KDevelop Python Support merge requestshttps://invent.kde.org/kdevelop/kdev-python/-/merge_requests2024-03-18T20:35:27Zhttps://invent.kde.org/kdevelop/kdev-python/-/merge_requests/24Draft: Port to KF62024-03-18T20:35:27ZMorten VoldenDraft: Port to KF6This work is based on Aleix's kf6 KDevelop branch
I don't know if anyone more qualified than me has been working on the same. But I thought I'd put it out here in case someone finds it useful.
So far what has been done is.
* Port cmak...This work is based on Aleix's kf6 KDevelop branch
I don't know if anyone more qualified than me has been working on the same. But I thought I'd put it out here in case someone finds it useful.
So far what has been done is.
* Port cmake code to BUILD_WITH_QT6
- Use KF_MAJOR_VERSION, QT_MIN_VERSION, KF_MIN_VERSION variables
* Port foreach to range-based for
* QLatin1Char and QStringLiteral changes in a lot of places
* Port QRegExp -> QRegularExpression
* use Q_SLOTS instead of slots
Almost all UT's are currently passing (except a few that I'll take a look at in the near future)Morten VoldenMorten Voldenhttps://invent.kde.org/kdevelop/kdev-python/-/merge_requests/23Use FindPython3 instead of FindPythonInterp and FindPythonLibs2024-01-15T18:56:14ZCasian AndreiUse FindPython3 instead of FindPythonInterp and FindPythonLibsFixes CMake warning regarding CMP0148
The warning is about deprecation of FindPythonInterp
and FindPythonLibs. Use FindPython3 instead.
Require CMake 3.19 because of the usage of the version range.Fixes CMake warning regarding CMP0148
The warning is about deprecation of FindPythonInterp
and FindPythonLibs. Use FindPython3 instead.
Require CMake 3.19 because of the usage of the version range.https://invent.kde.org/kdevelop/kdev-python/-/merge_requests/22Release/23.08 - build on musl libc + adding manually generated PySide6 files2024-01-16T15:04:47ZBjörn StrömbergRelease/23.08 - build on musl libc + adding manually generated PySide6 files- quick fix to make it build on Alpine Linux (musl libc) due to missing sys/types.h header in asttransformer.cpp https://mail.kde.org/pipermail/kdevelop-devel/2024-January/063783.html with Sven Brauch
- adding of PySide6 generated suppo...- quick fix to make it build on Alpine Linux (musl libc) due to missing sys/types.h header in asttransformer.cpp https://mail.kde.org/pipermail/kdevelop-devel/2024-January/063783.html with Sven Brauch
- adding of PySide6 generated support files (manually generated every one on Alpine Linux with PySide6 v6.6.0) and moved local files into correct folder of sourcetree
- quick fix to get the musl patch and pyside6 support into 23.08 so it will get pushed out with next patch release if there will be any more..
- limited manual testing and PySide6 support seems to work from how i understand that the functions should work.https://invent.kde.org/kdevelop/kdev-python/-/merge_requests/21GIT_SILENT Port to new way of including CI templates2023-12-18T23:15:34ZAlbert Astals CidGIT_SILENT Port to new way of including CI templateshttps://invent.kde.org/kdevelop/kdev-python/-/merge_requests/20Adapt to kdevplatform API changes around TypePtr2023-01-24T15:10:06ZMilian WolffAdapt to kdevplatform API changes around TypePtrhttps://invent.kde.org/kdevelop/kdev-python/-/merge_requests/19Show extension modules that are present as not readable instead of not-found2023-01-06T18:06:07ZfrmdstryrShow extension modules that are present as not readable instead of not-foundThis at least indicates whether the extension is installed or not.This at least indicates whether the extension is installed or not.https://invent.kde.org/kdevelop/kdev-python/-/merge_requests/18Re-enable f-string visiting2023-01-06T18:05:56ZfrmdstryrRe-enable f-string visitingThe python ast.parser appears to properly set ranges in f-strings. Seems to work as expected in KDevelop.
The CI runner looks like it is not actually running any of the tests but still says it's passing.The python ast.parser appears to properly set ranges in f-strings. Seems to work as expected in KDevelop.
The CI runner looks like it is not actually running any of the tests but still says it's passing.https://invent.kde.org/kdevelop/kdev-python/-/merge_requests/17Fix possible crash in AstTransformer2022-11-24T15:40:55ZBernd BuschinskiFix possible crash in AstTransformerThis fixes the following crash for me (only in a very huge Python project)
```
Thread 28 "Queue(0x555559d" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff8133e6c0 (LWP 25746)]
Python::AstTransformer::getattr<int...This fixes the following crash for me (only in a very huge Python project)
```
Thread 28 "Queue(0x555559d" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff8133e6c0 (LWP 25746)]
Python::AstTransformer::getattr<int> (this=0x7fff8133c550,
attr=0x7fff9c2de165 "lineno", obj=Python Exception <class
'gdb.error'>: There is no member named ma_keys.
) at /var/tmp/portage/dev-util/kdevelop-python-9999/work/kdevelop-python-9999/parser/asttransformer.cpp:37
37 /var/tmp/portage/dev-util/kdevelop-python-9999/work/kdevelop-python-9999/parser/asttransformer.cpp:
No such file or directory.
(gdb) bt
#0 Python::AstTransformer::getattr<int>(_object*, char const*)
constPython Exception <class 'gdb.error'>: There is no member named
ma_keys.
(this=0x7fff8133c550, attr=0x7fff9c2de165 "lineno", obj=) at
/var/tmp/portage/dev-util/kdevelop-python-9999/work/kdevelop-python-9999/parser/asttransformer.cpp:37
#1 Python::AstTransformer::visitAliasNode(_object*,
Python::Ast*)Python Exception <class 'gdb.error'>: There is no member
named ma_keys.
(this=0x7fff8133c550, node=, parent=<optimized out>) at
/var/tmp/portage/dev-util/kdevelop-python-9999/work/kdevelop-python-9999/parser/asttransformer.cpp:194
#2 0x00007fff9c2db35e in
Python::AstTransformer::visitNodeList<Python::AliasAst>(_object*,
Python::Ast*)Python Exception <class 'gdb.error'>: There is no member
named ma_keys.
(this=this@entry=0x7fff8133c550, node=node@entry=,
parent=parent@entry=0x7fff680142d0)
at /var/tmp/portage/dev-util/kdevelop-python-9999/work/kdevelop-python-9999/parser/asttransformer.cpp:142
#3 0x00007fff9c2d7058 in
Python::AstTransformer::visitStmtNode(_object*, Python::Ast*)Python
Exception <class 'gdb.error'>: There is no member named ma_keys.
(this=0x7fff8133c550, node=, parent=0x7fff6810ad50) at
/var/tmp/portage/dev-util/kdevelop-python-9999/work/kdevelop-python-9999/parser/asttransformer.cpp:1155
#4 0x00007fff9c2db40e in
Python::AstTransformer::visitNodeList<Python::Ast>(_object*,
Python::Ast*)Python Exception <class 'gdb.error'>: There is no member
named ma_keys.
(this=this@entry=0x7fff8133c550, node=node@entry=,
parent=parent@entry=0x7fff6810ad50)
at /var/tmp/portage/dev-util/kdevelop-python-9999/work/kdevelop-python-9999/parser/asttransformer.cpp:142
#5 0x00007fff9c2d78bf in
Python::AstTransformer::visitModuleNode(_object*, Python::Ast*)Python
Exception <class 'gdb.error'>: There is no member named ma_keys.
(this=this@entry=0x7fff8133c550, node=node@entry=, parent=parent@entry=0x0)
at /var/tmp/portage/dev-util/kdevelop-python-9999/work/kdevelop-python-9999/parser/asttransformer.cpp:128
#6 0x00007fff9c2d0b3f in Python::AstTransformer::run(_object*,
QString)Python Exception <class 'gdb.error'>: There is no member named
ma_keys.
(moduleName=..., syntaxtree=, this=0x7fff8133c550) at
/var/tmp/portage/dev-util/kdevelop-python-9999/work/kdevelop-python-9999/parser/asttransformer.h:30
#7 Python::AstBuilder::parse(QUrl const&, QString&)
(this=this@entry=0x7fff8133c908, filename=..., contents=...) at
/var/tmp/portage/dev-util/kdevelop-python-9999/work/kdevelop-python-9999/parser/astbuilder.cpp:264
#8 0x00007fff9c2c0cce in Python::ParseSession::parse()
(this=0x7fff68002510) at
/var/tmp/portage/dev-util/kdevelop-python-9999/work/kdevelop-python-9999/parser/parsesession.cpp:65
#9 0x00007fff81ec0f94 in
Python::ParseJob::run(QSharedPointer<ThreadWeaver::JobInterface>,
ThreadWeaver::Thread*) (this=<optimized out>) at
/usr/include/qt5/QtCore/qshareddata.h:160
#10 0x00007ffff295d937 in
ThreadWeaver::IdDecorator::run(QSharedPointer<ThreadWeaver::JobInterface>,
ThreadWeaver::Thread*) () at /usr/lib64/libKF5ThreadWeaver.so.5
#11 0x00007ffff295d586 in
ThreadWeaver::Executor::run(QSharedPointer<ThreadWeaver::JobInterface>
const&, ThreadWeaver::Thread*) () at
/usr/lib64/libKF5ThreadWeaver.so.5
#12 0x00007ffff295e3ef in
ThreadWeaver::Job::execute(QSharedPointer<ThreadWeaver::JobInterface>
const&, ThreadWeaver::Thread*) () at
/usr/lib64/libKF5ThreadWeaver.so.5
#13 0x00007ffff2962651 in ThreadWeaver::Thread::run() () at
/usr/lib64/libKF5ThreadWeaver.so.5
#14 0x00007ffff62caea7 in () at /usr/lib64/libQt5Core.so.5
#15 0x00007ffff60b2845 in start_thread (arg=<optimized out>) at
pthread_create.c:442
#16 0x00007ffff613236c in clone3 () at
../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) p v
$1 = 0x0
```
After enabling the additional qDebug in the Code, the last output was:
```
getattr<int>: "<ast.Expr object at 0x7f446ff17310>" .
lineno v= "<NULL>"
```
And I currently have 3 python versions installed (gentoo)
- dev-lang/python-3.11.0_p2
- dev-lang/python-3.10.8_p3
- dev-lang/python-3.9.15_p3https://invent.kde.org/kdevelop/kdev-python/-/merge_requests/16Port to Python 3.102022-11-07T11:15:47ZSven BrauchPort to Python 3.10This is a somewhat working version which supports Python 3.10.
The general idea is that we somehow call `ast.parse` in Python and somehow transfer the information to C++. We can either do this call in-process, or in a separate process; ...This is a somewhat working version which supports Python 3.10.
The general idea is that we somehow call `ast.parse` in Python and somehow transfer the information to C++. We can either do this call in-process, or in a separate process; the latter would require a client/server interface though because starting a process for each parse job is too slow. So I went for the first option.
My first attempt was serializing it to XML, then reading it out again; this works, but I now noticed it might also be feasible to use `PyObject_GetAttrString`-style introspection on the Python objects instead. We can still switch over to this. @flherne What do you think?
Performance shouldn't be affected much, since the whole parsing step is < 20% of effort anyways.
![image](/uploads/106b976d90d68225ea1d557dd363bbaa/image.png)
There are some issues remaining:
- [ ] Some unit tests fail, these mostly seem to be somewhat recent changes to the AST, e.g. `f(**{3:5})` doesn't correctly expand the double-starred dict.
- [ ] Deserialization is missing some error handling, I think we want to enable exceptions and throw if the stream is ill-formed, then catch it and show an "internal parser error" or so.
- [ ] Since this was a somewhat desparate attempt to get the project back up *at all*, I dropped support for Python < 3.10 for now. @flherne What do you think? Should we make an effort to support at least 3.9 and 3.8 or so?
- [ ] I (intentionally) left out some cleanup of old AST nodes, maybe we need them for compatibility still. E.g. `NumberAst` is gone, it's replaced by `ConstantAst`.
There are some things which are not so nice but we probably can't do much:
- The way KPluginLoader loads us as a plugin requires us to re-open libpython so that executed python code can find its symbols
On the positive side, I mostly got rid of our custom attribute range parsing by using the new `endCol` and `endLine` information.Francis HerneFrancis Hernehttps://invent.kde.org/kdevelop/kdev-python/-/merge_requests/15Add CI2021-11-29T21:15:46ZAlbert Astals CidAdd CIhttps://invent.kde.org/kdevelop/kdev-python/-/merge_requests/14When codestyle.py outputs too slowly (or unexpectedly), restart it instead of...2021-10-28T21:47:16ZJonathan VernerWhen codestyle.py outputs too slowly (or unexpectedly), restart it instead of crashing.When the `codestyle.py` is too slow in outputting data, the
`processOutputStarted` function might miss them, release the
`m_mutex` lock and set `m_stylechecker` to nullptr.
When the data then later arrives, processOutputStarted is calle...When the `codestyle.py` is too slow in outputting data, the
`processOutputStarted` function might miss them, release the
`m_mutex` lock and set `m_stylechecker` to nullptr.
When the data then later arrives, processOutputStarted is called
again, however without `m_mutex` being held and `m_stylechecker`
no longer valid. This eventually leads to a crash when dereferencing
`m_stylechecker`.
The current commit tries to fix this by checking that `m_mutex` is
held at the start of `processOutputStarted`. If it is not, then we are
in the "late data case". However, in this situation, we do not know
the amount of data that should still arrive and basically the only
way to solve the situation, is to kill the server (and start it again
on the next run). So that is what we do.
BUG: 444484https://invent.kde.org/kdevelop/kdev-python/-/merge_requests/13Switch to Release Service driven versioning2021-11-08T17:05:09ZFriedrich W. H. KossebauSwitch to Release Service driven versioningSee kdevelop!269 for main discussinSee kdevelop!269 for main discussinhttps://invent.kde.org/kdevelop/kdev-python/-/merge_requests/12Show preview for autopep8 formatting style2022-06-25T07:06:17ZIgor KushnirShow preview for autopep8 formatting styleThe default value of `SourceFormatterStyle::m_usePreview` is `false`.
So, despite the overriding Python sample, the preview was not displayed.The default value of `SourceFormatterStyle::m_usePreview` is `false`.
So, despite the overriding Python sample, the preview was not displayed.https://invent.kde.org/kdevelop/kdev-python/-/merge_requests/11Lock the DUChain when retrieving the declaration for a StructureType2022-03-21T10:18:16ZJonathan VernerLock the DUChain when retrieving the declaration for a StructureTypeThe call to `KDevelop::IdentifiedType::declaration` requires the DUChain
to be at least read locked, since it calls `KDevelop::DUChain::chainForIndex`,
which is documented to require a read lock on the DUChain
The commit also removes a ...The call to `KDevelop::IdentifiedType::declaration` requires the DUChain
to be at least read locked, since it calls `KDevelop::DUChain::chainForIndex`,
which is documented to require a read lock on the DUChain
The commit also removes a (slightly) related misdocumentation of
`Helper::resolveAliasDeclaration`, which claimed to need a read
lock while doing the locking itself.
BUG: 444055https://invent.kde.org/kdevelop/kdev-python/-/merge_requests/10Adapt to kdevplatform's TopDUContext::Features type change2020-10-24T11:14:22ZIgor KushnirAdapt to kdevplatform's TopDUContext::Features type changehttps://commits.kde.org/kdevelop/8552fe7c68eeefa9ef2c2aa2cfebeaefdfbd7c59
made casting of the `operator|` return value to `TopDUContext::Features`
redundant => eliminate it.
https://commits.kde.org/kdevelop/08a82a9cfaf30657eae9ebb71de8f...https://commits.kde.org/kdevelop/8552fe7c68eeefa9ef2c2aa2cfebeaefdfbd7c59
made casting of the `operator|` return value to `TopDUContext::Features`
redundant => eliminate it.
https://commits.kde.org/kdevelop/08a82a9cfaf30657eae9ebb71de8fcb6dcfd7432
renamed `TopDUContext`'s `enum Features` to `Feature` and added
`QFlags<Feature> Features` => introduce a `constexpr features` variable in
`ParseJob::run()` to change the type of the first `operator|` argument from
`Feature` to `Features` and thus fix the following compilation error (GCC):
invalid user-defined conversion from ‘QIncompatibleFlag’ to ‘KDevelop::TopDUContext::Feature’ [-fpermissive]https://invent.kde.org/kdevelop/kdev-python/-/merge_requests/9Use C++17 to compile kdev-python2020-10-23T09:00:01ZIgor KushnirUse C++17 to compile kdev-pythonKDevelop uses this standard since
https://commits.kde.org/kdevelop/f2925b5c6dc258def26c5f6d6d11433f896452c1
Its plugins must follow suit or they won't compile once C++17 features
make it into kdevplatform headers.KDevelop uses this standard since
https://commits.kde.org/kdevelop/f2925b5c6dc258def26c5f6d6d11433f896452c1
Its plugins must follow suit or they won't compile once C++17 features
make it into kdevplatform headers.https://invent.kde.org/kdevelop/kdev-python/-/merge_requests/8Implement virtual DebugSession::killDebuggerNow()2020-11-28T19:17:16ZIgor KushnirImplement virtual DebugSession::killDebuggerNow()Once kdevelop!182 is merged into 5.6, this change will be necessary only in kdev-python master.
kdev-python compiles with this change. All 6 tests pass. Sorry, I haven't tested how this commit works, because I am not a python developer ...Once kdevelop!182 is merged into 5.6, this change will be necessary only in kdev-python master.
kdev-python compiles with this change. All 6 tests pass. Sorry, I haven't tested how this commit works, because I am not a python developer and have zero experience with kdev-python.
See the commit message for details.https://invent.kde.org/kdevelop/kdev-python/-/merge_requests/7Change PEP8 max line length configuration single steps to 12020-10-07T16:33:34ZBernd BuschinskiChange PEP8 max line length configuration single steps to 1My value was 119 (for some unknown reason) and I wanted 120, that is not (easily) possible with a value of 2 for singleStep and only scrolling.My value was 119 (for some unknown reason) and I wanted 120, that is not (easily) possible with a value of 2 for singleStep and only scrolling.https://invent.kde.org/kdevelop/kdev-python/-/merge_requests/6Fix ExtSlice crash with Python <= 3.82020-09-20T14:28:54ZFrancis HerneFix ExtSlice crash with Python <= 3.8Alternative to !4. See commit message for details. @brauch maybe you were right about just deleting the damn script.
Needs more testing, but I'm going to sleep now.Alternative to !4. See commit message for details. @brauch maybe you were right about just deleting the damn script.
Needs more testing, but I'm going to sleep now.Sven BrauchSven Brauchhttps://invent.kde.org/kdevelop/kdev-python/-/merge_requests/5WIP: Fix ExtSlice crash with Python <= 3.82020-09-20T00:00:10ZFrancis HerneWIP: Fix ExtSlice crash with Python <= 3.8Alternative to !4. See commit message for details. @brauch maybe you were right about just deleting the damn script.
Needs more testing, but I'm going to sleep now.Alternative to !4. See commit message for details. @brauch maybe you were right about just deleting the damn script.
Needs more testing, but I'm going to sleep now.