test_files-clang failure with Clang 13
test_files-clang consistently fails on KDevelop CI. It started failing on my system after Clang update to version 13.0. A relevant excerpt from the failing test's output:
QDEBUG : TestFiles::testFiles(basicdeclsandcontexts.cpp) "identifier" FAILED: Declaration's identifier ("(unnamed struct at /home/jenkins/workspace/KDevelop/kdevelop/kf5-qt5 SUSEQt5.15/plugins/clang/tests/files/basicdeclsandcontexts.cpp:97:1)") doesn't match test data (""). (Declaration on line 97 in /home/jenkins/workspace/KDevelop/kdevelop/kf5-qt5 SUSEQt5.15/plugins/clang/tests/files/basicdeclsandcontexts.cpp)
QDEBUG : TestFiles::testFiles(basicdeclsandcontexts.cpp) "identifier" FAILED: Declaration's identifier ("(unnamed union at /home/jenkins/workspace/KDevelop/kdevelop/kf5-qt5 SUSEQt5.15/plugins/clang/tests/files/basicdeclsandcontexts.cpp:100:1)") doesn't match test data (""). (Declaration on line 100 in /home/jenkins/workspace/KDevelop/kdevelop/kf5-qt5 SUSEQt5.15/plugins/clang/tests/files/basicdeclsandcontexts.cpp)
FAIL! : TestFiles::testFiles(basicdeclsandcontexts.cpp) 'validator.testsPassed()' returned FALSE. ()
Loc: [/home/jenkins/workspace/KDevelop/kdevelop/kf5-qt5 SUSEQt5.15/plugins/clang/tests/test_files.cpp(116)]
I attempted to fix the test failure, but realized that this is not a straightforward task. The declaration's identifier contains an absolute path, so cannot be included in a comment in the file kdevelop/plugins/clang/tests/files/basicdeclsandcontexts.cpp. Plus the identifier is empty in Clang versions < 13.
Fortunately, these failing test cases are the only ones that have an empty identifier. So the test cases can be referred to as pair("identifier", "")
and fixed or worked around.
Possible solutions:
- Remove the two unnamed-object declarations that cause the failure. This solution is trivial to implement, but reduces test coverage.
- Expand
DeclarationValidator
's and possiblyTestSuite
's interfaces to be able to pass test data fix-ups at runtime. This solution is much more tricky and time-consuming to implement. But if done right, could let us fix similar issues in the future - without removing tests. Unfortunately I don't know how to do this right and even if it is worth doing.