Skip to content

Fix running tests on macOS

Ivan Yossi requested to merge ivany/krita:macOS_buildTests into master

This adds some changes to how tests are build on macOS, cmake does not compile test libs as MODULE but as SHARED, this should not change anything in Linux but allows macs to link properly to tests to compile them.

I added new cmake method macos_test_fixrpath to add the correct rpath to the build tests, also all macos tests are now build as GUI, this is necessary for some tests but I added to every macos tests as it does not break the tests that don't need it and allows for finding each tests easily. The option can change binary behaviour on windows so it is only on for macOS compiles.

Make test now executes tests on macos and report pass fail but...

The number of failed tests on macOS is significantly higher as many tests require to be installed to be run, this does not mean the test is broken, but that we haven't come up with a good way to make sure Qt finds the bundle path for test apps running from the build directory. Such tests can be run if copied to the install dir and run manually.

Added script to run all tests as if ctest is run

This is a convenience script to run all compiled tests.app (this one of the reasons I decided to compile all tests on macOS as GUI bundles) from the current dir recursively. (you can run all tests found or just a set of argument given tests)

On macos some tests require to be run onn the install dir for Qt to find the bundle path, this tests are marked as broken to allow ctest to run with no errors.

The script will use BUILDROOT to se the install directory and to check if the directory you are running the tests is child of to the BUILDROOT directory. (this can be overriden with args --install_dir= and --work_dir=) From the run directory the script will find any .app bundle, install it, and run it, reporting if the test failed or passed.

The output is very similar to ctest, however this does not filter real broken tests than bundle_path broken tests.

NOTES

marked as broken but passes

/Volumes/Osiris/programs/krita-master/kisbuild/plugins/impex/ora/tests/KisOraTest.app

Jp2, png tests are marked as broken on both but passes. on macOS since they need kritaui for testing, they will be broken on macos until we fix the source issue.

All /Volumes/Osiris/programs/krita-master/kisbuild/plugins/paintops/libpaintop/ tests fail with:

QCoreApplication::applicationDirPath: Please instantiate the QApplication object first
QCoreApplication::arguments: Please instantiate the QApplication object first
QCoreApplication::applicationDirPath: Please instantiate the QApplication object first
Cannot calculate the bundle path from the app path

In this cases all tests are missing instantiation of QApplication.

Broken tests on macOS

The following tests fail when running ctest, I leave them as not broken tests as the problem seems to be RRU on 105, 109 and 114. For test #7, I started investigating the issue, but leave it since I neede first to make the thing work for all tests.

The following tests FAILED:
      7 - libs-widgetutils-TestKoProgressUpdater (Failed)
    105 - libs-kritaresources-TestTag (Failed)
    109 - libs-kritaresources-TestTagModel (Failed)
    114 - libs-kritaresources-TestTagResourceModel (Failed)

Formalities Checklist

  • I confirmed this builds.
  • I confirmed Krita ran and the relevant functions work.
  • I tested the relevant unit tests and can confirm they are not broken. (If not possible, don't hesitate to ask for help!)
  • I made sure my commits build individually and have good descriptions as per KDE guidelines.
  • I made sure my code conforms to the standards set in the HACKING file.
  • I can confirm the code is licensed and attributed appropriately, and that unattributed code is mine, as per KDE Licensing Policy.
Edited by Halla Rempt

Merge request reports