Commit c4f24526 authored by Anna Medonosová's avatar Anna Medonosová

Quality assurance pages

parent 8c291cf7
.. meta::
:description:
Quality Assurance.
.. metadata-placeholder
:authors: - Anna Medonosová <anna.medonosova@gmail.com>
:license: GNU free documentation license 1.3 or later.
.. _quality_assurance:
=================================
Introduction to Quality Assurance
=================================
We are users and developers systematically working on increasing quality of Krita and the process of it’s development. We help sustain the self-auditing culture of Krita’s community.
We
* Methodically assess functionality, usability and security.
* Hunt for bugs and cater for bugs already captured.
* Aid in quality management. Create tools to make developer’s life easier.
How To Help?
------------
The quality assurance field is really broad and diverse and we are always looking for people of all skills and talents. Below you will find a list of opportunities to help, so you can dive right into it. Also, don’t forget to visit us on the IRC, we will be happy to meet you.
Bug Triaging
~~~~~~~~~~~~
There is a great amount of incoming bug reports, more than the core team can handle. We are looking for volunteers who would go through the bug tracker and handle the reported bugs. This includes:
* Determining if a bug is really a bug or a new feature request
* Confirming bugs by reproducing
* Guiding reporters to provide all the information needed to fix the bug
* (Optional) Providing logs, backtraces, core dumps
Get Started
^^^^^^^^^^^
* :ref:`bugs_reporting` provides general information about bug reports and guidance for their creation
* Krita-specific guide to :ref:`triaging_bugs`
.. seealso::
* Guide to :ref:`developing_features`
* Hints for user support also apply here: :ref:`intro_user_support`
* Docs for gathering logs, backtraces, etc.
* https://docs.krita.org/en/KritaFAQ.html?highlight=mingw#how-can-i-produce-a-backtrace-on-windows
* https://docs.krita.org/en/reference_manual/dockers/log_viewer.html
Beta Testing
~~~~~~~~~~~~
To validate an upcoming stable version will work as expected, there is the beta version. You can help by dowloading the beta, trying it out and sharing your feedback. Every beta comes with a survey, which will ask for some basic information about your setup (all anonymized, of course) and guide you through testing latest features and bug fixes. You can find link to the survey on Krita's welcome page.
To know when there is a new beta, watch out for the news on the welcome page, or in the News section on Krita website.
For more information about the process refer to the :ref:`testing_strategy`.
Test Engineering
~~~~~~~~~~~~~~~~
The test suite is the safety net enabling the community to fearlessly move forward. We have a comprehensive testing strategy to help us find bugs early in the process and deliver the best user experience possible. But without people, the strategy is just a bunch of words. There are many ways you can help, for both technical and less technical people.
* If you like to experiment and try new things, consider exploratory testing. No coding skill required.
* Hone your analytical skills by designing end-to-end tests.
* Try your hand at unit testing. Design and implement the low level tests for both backend and UI code.
.. * Use your coding experience to implement the automatic test suite.
Check out :ref:`testing_strategy` for more information.
Enhancement Projects
~~~~~~~~~~~~~~~~~~~~
There is plenty of projects from small to big, some include writing and organizing, some require coding. We currently register following projects: https://phabricator.kde.org/T11218. Does something catch your eye?
Do you have something else in mind?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This list is not definite. We are always open to new ideas and approaches. Please, join us on the IRC (:ref:`the_krita_community`) to discuss the possibilities.
.. meta::
:description:
Testing strategy.
.. metadata-placeholder
:authors: - Anna Medonosová <anna.medonosova@gmail.com>
:license: GNU free documentation license 1.3 or later.
.. _testing_strategy:
================
Testing Strategy
================
Overview
--------
We’re always working on the next version of Krita. We fix bugs and implement new features. Every change to any software comes with a risk of introducing other issues. That's where testing comes in. The tester’s job is to uncover defects as early as possible in the development process: a bug caught early enough means easier fixing, better user experience and less load on user support.
The Functional Test Suite
-------------------------
When testing functionality we employ multiple strategies that translate into several layers of the test suite:
* Unit tests are our safeguard against breakage during development.
* End to end tests check that the basic high level workflows function properly.
* Exploratory testing experiments. Unexpected combinations, uncharted workflows.
.. * Acceptance tests help developing new features and make sure those features still work after changes.
.. We automate majority of the testing. Human time is precious. Use it where it is most beneficial. Computers are bad at intuition and drawing conclusions, but they are good at brute force checking.
Test Suite Layers
~~~~~~~~~~~~~~~~~
Unit Tests
^^^^^^^^^^
Unit tests are the base of our test suite. They are designed to ensure that every individual unit of source code (both backend and UI components) functions as expected. They are fully automated and fast to execute. They are run by developers during development. Also part of nightly testing suite.
In-depth unit testing doc: https://docs.krita.org/en/untranslatable_pages/unit_tests_in_krita.html
.. Acceptance and regression tests
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. Acceptance tests ensure that features work correctly for the user.
.. When new feature is developed, it’s user requirements are formalized into executable scenarios. These scenarios serve two purposes. First is to clearly communicate expected outputs of the feature under construction and give the developers a way to check compliance during their work. Second, these scenarios can be later used for regression testing - making sure that what worked before was not broken by another change.
.. Acceptance tests usually involve the UI of the application, however it is not a requirement. Design the tests for any layer that makes sense and ensures correct function for the user.
.. More information on https://docs.krita.org/en/untranslatable_pages/new_features.html and acceptance test writing [TODO].
End-to-end UI Tests
^^^^^^^^^^^^^^^^^^^
Formalized high level tests performed on the running application; carried out either by a computer or by a human.
End to end tests cover both the GUI and the command line interface.
Exploratory Testing
^^^^^^^^^^^^^^^^^^^
While the other layers of the test suite are composed of carefully curated scripted tests, balancing between coverage and efficiency, exploratory testing approaches testing quite differently. It’s purpose is to allow humans to apply their unique tools: learning, creativity, intuition. There are no suites, scenarios, defined steps. Just you and Krita. Explore and experiment. Try basic workflows. Try unexpected combinations. Try to break things. Then report bugs.
.. Get started docs [TODO].
When Do We Test
~~~~~~~~~~~~~~~
Continuous testing as part of continuous integration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Automatic test suite is run nightly against the last nightly build.
Beta Testing
^^^^^^^^^^^^
Beta testing is a type of user acceptance testing, where a subgroup of target users validates the upcoming release.
As a part of the release process, we collect features and bugs (mainly high impact bugs and those that benefit from testing in multiple different conditions) to test in a Phabricator task connected to the release. From that collection we create a survey on survey.kde.org and publicly release the beta version. Link to the survey is available on the welcome page of the beta release.
.. TODO: testing in the release cycle
.. TODO: UI tests during alpha and beta stages of a major release
.. * Draft for test scenarios: https://phabricator.kde.org/T8106
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment