Commit 3f38da8a authored by Dan Leinir Turthra Jensen's avatar Dan Leinir Turthra Jensen 🌈
Browse files

Add KPackage support to KNewStuffCore

Summary:
Adding support for KPackage directly to KNewStuff means that we are
able to deal more gracefully with things like Plasma's Global Themes
(and indeed any other kpackage based thing).

This is done by adding another archive specialisation to the installer
class, and by also adding a check to the cache to ensure that even
when a kpackage is removed from the system outside of KNewStuff,
it does not remain seemingly installed in the KNS lists.

* Make sure the cache gets written periodically
* Add KPackage support to KNSCore::Installation
* Introduce a getter (and enum) for the uncompress Installation setting
* Add a redirection to the knsrc documentation location
* Add a function to clean the cache of functionally stale entries
* Clean the cache when the uncompression method is set to kpackage
* Add a fallback for unconverted kpackage based knsrc files
* Clean up some of the error reporting, and reset the entry's state
* Check if installedFile is a file, if so bypass KPackage and delete
* Add a KPackageType property to Installation, for fallback purposes
* Add documentation for the new knsrc bits
* Handle adopting an already installed kpackage item
* Also uninstall not-adopted-but-there possibly installed kpackage bits
* Add a simple async job wrapper for KPackage operations (and use it for the installation handling tasks in Installation)

BUG:418466

Test Plan:
There are two options for testing out this patch:

1) Use an existing knsrc file which uses kpackage installation (such as plasma themes), which will use the fallback
2) Manually convert such a knsrc file, by removing the uninstall and installation commands from the knsrc file, and adding in an "Uncompress=kpackage" line instead

Both these should result in the KPackage path being used. You should see this on the command line when attempting to install an item, resulting in lines like "Using KPackage for installation", as well as more pleasant error reporting in the UI in the cases where something goes wrong.

To turn on debug output for KNewStuffCore, add QT_LOGGING_RULES="org.kde.knewstuff*=true" to your command line. For example, you can launch the test dialogue directly by launching the following from your build directory:

  QT_LOGGING_RULES="org.kde.knewstuff*=true" ./bin/khotnewstuff-dialog plasma-themes.knsrc

Reviewers: #plasma, #knewstuff, #frameworks, ngraham, mart, davidedmundson, broulik, bshah

Reviewed By: #plasma, mart

Subscribers: alex, ngraham, kde-frameworks-devel

Tags: #frameworks

Differential Revision: https://phabricator.kde.org/D28701
parent 293c87dc
......@@ -32,6 +32,7 @@ find_package(KF5I18n ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5IconThemes ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5KIO ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5ItemViews ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5Package ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5Service ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5TextWidgets ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5WidgetsAddons ${KF5_DEP_VERSION} REQUIRED)
......
......@@ -30,6 +30,9 @@ set(KNewStuffCore_SRCS
jobs/httpjob.cpp
jobs/httpworker.cpp
# A simple wrapper around KPackage operations, which allows for asynchronous interaction
jobs/kpackagejob.cpp
../attica/atticaprovider.cpp
../staticxml/staticxmlprovider.cpp
......@@ -73,6 +76,7 @@ target_link_libraries(KF5NewStuffCore
KF5::Archive # For decompressing archives
KF5::I18n # For translations
KF5::ConfigCore
KF5::Package
Qt5::Gui # For QImage
)
......
......@@ -21,6 +21,7 @@
#include <QFile>
#include <QDir>
#include <QFileInfo>
#include <QTimer>
#include <QXmlStreamReader>
#include <qstandardpaths.h>
#include <knewstuffcore_debug.h>
......@@ -241,6 +242,7 @@ void Cache::registerChangedEntry(const KNSCore::EntryInternal &entry)
{
setProperty("dirty", true);
cache.insert(entry);
QTimer::singleShot(1000, this, [this](){ writeRegistry(); });
}
void Cache::insertRequest(const KNSCore::Provider::SearchRequest &request, const KNSCore::EntryInternal::List &entries)
......@@ -261,3 +263,22 @@ EntryInternal::List Cache::requestFromCache(const KNSCore::Provider::SearchReque
return requestCache.value(request.hashForRequest());
}
void KNSCore::Cache::removeDeletedEntries()
{
QMutableSetIterator<KNSCore::EntryInternal> i(cache);
while (i.hasNext()) {
const KNSCore::EntryInternal &entry = i.next();
bool installedFileExists{false};
for (const auto &installedFile: entry.installedFiles()) {
if (QFile::exists(installedFile)) {
installedFileExists = true;
break;
}
}
if (!installedFileExists) {
i.remove();
setProperty("dirty", true);
}
}
writeRegistry();
}
......@@ -57,6 +57,19 @@ public:
void insertRequest(const KNSCore::Provider::SearchRequest &, const KNSCore::EntryInternal::List &entries);
EntryInternal::List requestFromCache(const KNSCore::Provider::SearchRequest &);
/**
* This will run through all entries in the cache, and remove all entries
* where all the installed files they refer to no longer exist.
*
* This cannot be done wholesale for all caches, as some consumers will allow
* this to happen (or indeed expect it to), and so we have to do this on a
* per-type basis
*
* This will also cause the cache store to be updated
*
* @since 5.71
*/
void removeDeletedEntries();
public Q_SLOTS:
void registerChangedEntry(const KNSCore::EntryInternal &entry);
......
......@@ -180,6 +180,9 @@ bool Engine::init(const QString &configfile)
qCDebug(KNEWSTUFFCORE) << "Cache is" << m_cache << "for" << configFileName;
connect(this, &Engine::signalEntryChanged, m_cache.data(), &Cache::registerChangedEntry);
m_cache->readRegistry();
if (m_installation->uncompressionSetting() == Installation::UseKPackageUncompression) {
m_cache->removeDeletedEntries();
}
m_initialized = true;
......
......@@ -89,6 +89,7 @@ public:
*
* @param configfile KNewStuff2 configuration file (*.knsrc)
* @return \b true if any valid configuration was found, \b false otherwise
* @see KNS3::DownloadDialog
*/
bool init(const QString &configfile);
......
This diff is collapsed.
......@@ -64,6 +64,13 @@ public:
ScopeUser,
ScopeSystem
};
enum UncompressionOptions {
NeverUncompress, ///@< Never attempt to decompress a file, whatever format it is. Matches "never" knsrc setting
AlwaysUncompress, ///@< Assume all downloaded files are archives, and attempt to decompress them. Will cause failure if decompression fails. Matches "always" knsrc setting
UncompressIfArchive, ///@< If the file is an archive, decompress it, otherwise just pass it on. Matches "archive" knsrc setting
UncompressIntoSubdir, ///@< As Archive, except that if there is more than an item in the file, put contents in a subdirectory with the same name as the file. Matches "subdir" knsrc setting
UseKPackageUncompression ///@< Use the internal KPackage support for installing and uninstalling the package. Matches "kpackage" knsrc setting
};
bool readConfig(const KConfigGroup &group);
bool isRemote() const;
......@@ -116,6 +123,14 @@ public Q_SLOTS:
*/
void uninstall(KNSCore::EntryInternal entry);
/**
* Returns the uncompression setting, in a computer-readable format
*
* @return The value of this setting
* @since 5.71
*/
UncompressionOptions uncompressionSetting() const;
// TODO KF6: remove, was used with deprecated Security class.
#if KNEWSTUFFCORE_ENABLE_DEPRECATED_SINCE(5, 31)
KNEWSTUFFCORE_DEPRECATED_VERSION(5, 31, "No longer use")
......
/*
Copyright (C) 2020 Dan Leinir Turthra Jensen <admin@leinir.dk>
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library. If not, see <http://www.gnu.org/licenses/>.
*/
#include "kpackagejob.h"
#include <knewstuffcore_debug.h>
#include <KLocalizedString>
#include <KPackage/PackageStructure>
#include <KPackage/Package>
#include <KPackage/PackageLoader>
#include <QCoreApplication>
#include <QRunnable>
#include <QStandardPaths>
#include <QThreadPool>
#include <QTimer>
using namespace KNSCore;
enum Operation {
UnknownOperation,
InstallOperation,
UpdateOperation,
UninstallOperation
};
class KPackageTask;
class KPackageJob::Private {
public:
Private() {}
QString package;
QString packageRoot;
QString serviceType;
Operation operation{UnknownOperation};
KPackageTask* runnable{nullptr};
};
class KPackageTask : public QObject, public QRunnable
{
Q_OBJECT
public:
QString package;
QString packageRoot;
QString serviceType;
Operation operation{UnknownOperation};
void run() override
{
qCDebug(KNEWSTUFFCORE) << "Attempting to perform an installation operation of type" << operation << "on the package" << package << "of type" << serviceType << "in the package root" << packageRoot;
KPackage::PackageStructure *structure = KPackage::PackageLoader::self()->loadPackageStructure(serviceType);
if (structure) {
qCDebug(KNEWSTUFFCORE) << "Service type understood";
KPackage::Package installer = KPackage::Package(structure);
if (installer.hasValidStructure()) {
qCDebug(KNEWSTUFFCORE) << "Installer successfully created and has a valid structure";
KJob *job{nullptr};
switch(operation)
{
case InstallOperation:
job = installer.install(package, packageRoot);
break;
case UpdateOperation:
job = installer.update(package, packageRoot);
break;
case UninstallOperation:
job = installer.uninstall(package, packageRoot);
break;
case UnknownOperation:
default:
// This should really not be happening, can't create one of these without going through one
// of the functions below, so how'd you get it in this state?
break;
};
if (job) {
qCDebug(KNEWSTUFFCORE) << "Created job, now let's wait for it to do its thing...";
QEventLoop loop;
connect(job, &KJob::result, this, [this,job,&loop](){
emit result(job);
loop.exit(0);
});
loop.exec();
} else {
qCWarning(KNEWSTUFFCORE) << "Failed to create a job to perform our task";
emit error(3, i18n("Failed to create a job for the package management task. This is usually because the package is invalid. We attempted to operate on the package %1", package));
}
} else {
qCWarning(KNEWSTUFFCORE) << "Failed to create package installer";
emit error(2, i18n("Could not create a package installer for the service type %1: The installer does not have a valid structure", serviceType));
}
} else {
qCWarning(KNEWSTUFFCORE) << "Service type was not understood";
emit error(1, i18n("The service type %1 was not understood by the KPackage installer", serviceType));
}
}
Q_SIGNAL void result(KJob* job);
Q_SIGNAL void error(int errorCode, const QString& errorText);
};
KPackageJob::KPackageJob(QObject* parent)
: KJob(parent)
, d(new Private)
{
}
KPackageJob::~KPackageJob()
{
delete d;
}
void KPackageJob::start()
{
if (d->runnable) {
// refuse to start the task more than once
return;
}
d->runnable = new KPackageTask();
d->runnable->package = d->package;
d->runnable->packageRoot = d->packageRoot;
d->runnable->serviceType = d->serviceType;
d->runnable->operation = d->operation;
connect(d->runnable, &KPackageTask::error, this, [this](int errorCode, const QString& errorText){
setError(errorCode);
setErrorText(errorText);
}, Qt::QueuedConnection);
connect(d->runnable, &KPackageTask::result, this, [this](KJob* job){
setError(job->error());
setErrorText(job->errorText());
emitResult();
}, Qt::QueuedConnection);
QThreadPool::globalInstance()->start(d->runnable);
}
KNSCore::KPackageJob * KNSCore::KPackageJob::install(const QString &sourcePackage, const QString &packageRoot, const QString &serviceType)
{
KPackageJob* job = new KPackageJob();
job->d->package = sourcePackage;
job->d->packageRoot = packageRoot;
job->d->serviceType = serviceType;
job->d->operation = InstallOperation;
QTimer::singleShot(0, job, &KPackageJob::start);
return job;
}
KPackageJob * KPackageJob::update(const QString &sourcePackage, const QString &packageRoot, const QString &serviceType)
{
KPackageJob* job = new KPackageJob();
job->d->package = sourcePackage;
job->d->packageRoot = packageRoot;
job->d->serviceType = serviceType;
job->d->operation = UpdateOperation;
QTimer::singleShot(0, job, &KPackageJob::start);
return job;
}
KPackageJob * KPackageJob::uninstall(const QString &packageName, const QString &packageRoot, const QString &serviceType)
{
KPackageJob* job = new KPackageJob();
job->d->package = packageName;
job->d->packageRoot = packageRoot;
job->d->serviceType = serviceType;
job->d->operation = UninstallOperation;
QTimer::singleShot(0, job, &KPackageJob::start);
return job;
}
#include "kpackagejob.moc"
/*
Copyright (C) 2020 Dan Leinir Turthra Jensen <admin@leinir.dk>
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef KPACKAGEJOB_H
#define KPACKAGEJOB_H
#include <KCoreAddons/KJob>
namespace KNSCore {
/**
* @brief A job for performing basic actions on KPackage packages asynchronously
*
* The internals of KPackage's Package functions are synchronous, which makes it easy to work with in some cases,
* but has the unfortunate side effect of blocking the UI. This job will perform those operations in a separate
* thread, which allows you to perform the work in a fire-and-forget fashion (as suggested by KJob's documentation).
*
* @since 5.71
*/
class KPackageJob : public KJob
{
Q_OBJECT
public:
/**
* Create a job for installing the given package into the package root, and treat it as the given service type.
*
* @param sourcePackage The full path name to the package you wish to install (e.g. /tmp/downloaded-archive.tar.xz)
* @param packageRoot The full path name to the location the package should be installed into (e.g. /home/username/.share/plasma/desktoptheme/)
* @param serviceType The name of the type of KPackage you intend to install (e.g. Plasma/Theme)
* @return A job which you can use to track the completion of the process (there will be useful details in error() and errorText() on failures)
*/
static KPackageJob *install(const QString &sourcePackage, const QString &packageRoot, const QString &serviceType);
/**
* Create a job for updating the given package, or installing it if it is not already, the given package into the
* package root, and treat it as the given service type.
*
* @param sourcePackage The full path name to the package you wish to update (e.g. /tmp/downloaded-archive.tar.xz)
* @param packageRoot The full path name to the location the package should be installed into (e.g. /home/username/.share/plasma/desktoptheme/)
* @param serviceType The name of the type of KPackage you intend to update (e.g. Plasma/Theme)
* @return A job which you can use to track the completion of the process (there will be useful details in error() and errorText() on failures)
*/
static KPackageJob *update(const QString &sourcePackage, const QString &packageRoot, const QString &serviceType);
/**
* Create a job for removing the given installed package
*
* @param packageName The name to the package you wish to remove (this is the plugin name, not the full path name, e.g. The.Package.Name)
* @param packageRoot The full path name to the location the package is currently installed (e.g. /home/username/.share/plasma/desktoptheme/The.Package.Name)
* @param serviceType The name of the type of KPackage you intend to remove (e.g. Plasma/Theme)
* @return A job which you can use to track the completion of the process (there will be useful details in error() and errorText() on failures)
*/
static KPackageJob *uninstall(const QString &packageName, const QString &packageRoot, const QString &serviceType);
virtual ~KPackageJob();
/**
* Start the process asynchronously
* @see KJob::start()
*/
Q_SLOT void start() override;
private:
explicit KPackageJob(QObject* parent = nullptr);
class Private;
Private* d;
};
}
#endif//KPACKAGEJOB_H
......@@ -63,6 +63,7 @@ class DownloadDialogPrivate;
* <li>never: never try to extract the file</li>
* <li>archive: if the file is an archive, uncompress it, otherwise just pass it on</li>
* <li>subdir: logic as archive, but decompress into a subdirectory named after the payload filename</li>
* <li>kpackage: require that the downloaded file is a kpackage, and use the kpackage framework for handling installation and removal (since 5.70)</li>
* </ol>
*
* You have different options to set the target install directory:
......@@ -72,6 +73,24 @@ class DownloadDialogPrivate;
* This is what QStandardPaths::writableLocation(QStandardPaths::GenericDataLocation) + QLatin1Char('/') + name will return.</li>
* </ol>
*
* \subsection KPackage Support
*
* To make use of the KPackage option described above, in addition to the Uncompress setting above, you should also specify
* the type of archive expected by KPackage. While it is possible to deduce this from the package metadata in many situations,
* it is not a requirement of the format that this information exists, and we need to have a fallback in the case it is not
* available there. As such, you will want to add a KPackageType entry to your knsrc file. The following example shows how this
* is done for Plasma themes:
*
* <pre>
ProvidersUrl=https://autoconfig.kde.org/ocs/providers.xml
Categories=Plasma Theme
StandardResource=tmp
TagFilter=ghns_excluded!=1,plasma##version==5
DownloadTagFilter=plasma##version==5
Uncompress=kpackage
KPackageType=Plasma/Theme
* </pre>
*
* @since 4.4
*/
class KNEWSTUFF_EXPORT DownloadDialog : public QDialog
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment