Skip to content

Testing and development of KEcoLab

Project abstract

(Which KDE project do you want to do a proposal for, describe briefly what is your idea)

"Testing and development of KEcoLab"

Sustainability is a key goal for KDE over the next two years. One aspect of this goal is to measure the energy consumption of KDE softwares. To do this, it is necessary to access lab in KDAB, Berlin, which can be done remotely using KEcoLab.

The proposed proposal has these primary goals

  • Testing KEcoLab & bug squashing
  • Integrating measurement workflow in Okular repository
  • Create tests for KEcolab

Secondary goals

  • Measuring in WayLand
  • Usage Scenario scripting with Selenium

Proposal

(More details on what you plan to do, why is it useful to KDE, how you plan to do it and how do you plan to document and test your changes)

Achieving the high and medium-priority goals outlined above is crucial for KEcoLab, as they directly contribute to enhancing the capabilities of Remote EcoLab.

Testing KEcoLab & bug squashing

The goal of the issue "Testing KEcoLab & bug squashing" is to ensure the reliability, functionality, and performance of KEcoLab. The goal includes identifying and addressing any bugs or issues present in KEcoLab through a systematic testing process

Overview

Benchmarks: Identify benchmarks, which are specific criteria or standards against which we will be testing KEcoLab. Examples of benchmarks are performance metrics and compatibility with different application scripts or any type of tests to identify bugs with KEcolab.

Performance Testing: Consider testing the performance of KEcoLab. This involves assessing how well the software performs under different conditions, such as various workloads or scenarios. This could include measuring response times, resource usage, and overall system efficiency.

Compatibility Testing: Check if KEcoLab works seamlessly with different application scripts. This involves ensuring that KEcoLab is compatible with a variety of scripts and doesn't exhibit issues or errors when interacting with them.

Bug Identification: The main goal is to run various types of tests to identify any bugs or issues present in KEcoLab

How to Approach the Issue:

  1. Define Test Scenarios: Based on the provided information, create test scenarios that cover performance testing and compatibility testing. Think about different use cases and conditions under which KEcoLab might be used.
  2. Develop and Execute Test Cases: Write test cases that outline the steps to be taken, the expected outcomes. These should cover the identified benchmarks. Run the test cases and observe the behaviour of KEcoLab. Note any deviations from expected results.
  3. Report Findings: Document the findings, including successful tests and any bugs or issues encountered. Provide clear and concise information that can help developers understand and address the identified issues.
  4. Collaborate with Mentor and Community: Engage with the mentor and the KDE community to discuss the findings. Seek guidance, share progress, and collaborate with others who may be involved in testing or development.

Integrating measurement workflow in Okular repository

The goal of the issue “Integrating measurement workflow in Okular repository” is to create a GitLab CI pipeline that runs usage scenarios on Okular after each release. Instead of directly modifying the Okular CI pipeline, we will create a dedicated template for KEcolab. This template, similar to the ones found in the KDE CI Utilities Templates, will encapsulate the measurement workflow. The template will then be included in the Okular GitLab CI configuration.

Okular - It is the universal document viewer, also multi-platform, fast and packed with features, Okular allows user to read PDF documents, comics and EPub books, browse images, visualize Markdown documents, and much more

Workflow: Once a reviewer approves a merge request created with the app_name/module_name of the application to test, the CI job will run. This is to ensure the security of someone making an MR which runs arbitrary code.

Here's a general outline of the steps for GitLab CI template for Okular:

stages:
  - build
  - test
  - deploy

variables:
  OKULAR_VERSION: "latest"
  # Add other variables as needed
  
build:
  stage: build
  script:
    - git clone <Okular repository URL>
    - cd okular
    - ./build.sh

test:
  stage: test
  script:
    - cd okular
    - ./run_tests.sh
    
deploy:
  stage: deploy
  script:
    - cd okular
    - ./deploy.sh
  
  only:
    - master  # Deploy only when changes are pushed to the master branch

only:
  - tags  # Run only for tagged releases

Create tests for KEcolab

The goal of the issue “create tests for KEcolab” is to implement automated tests for KEcolab using the existing Kate test scripts. This will help ensure the stability and functionality of KEcolab after each merge request..

CI tests are automated tests that are run whenever code changes are made to ensure that the new code does not break existing functionality.

This can be achieved by implementing CI tests that automatically run against KEcolab after changes are introduced.

Steps:

  1. Identifying KEcolab Test Scenarios: Identify critical scenarios and functionalities in KEcolab that need testing. Examples: Basic code parsing and analysis. Handling different file types.
  2. Creating Test Cases: Develop test cases for the identified scenarios . Ensure that test cases cover a wide range of use cases and edge cases to guarantee robustness.
  3. Integrate Tests into CI/CD Pipeline: Integrate the newly created tests into the Continuous Integration (CI) pipeline to automatically trigger the tests after each merge request.
  4. Documentation: Document the newly created test cases, their purpose, and any specific instructions for developers to maintain and update them in the future.
  5. Test Maintenance: Regularly review and update test cases to adapt to changes in KEcolab's codebase. Ensure that new features and changes are accompanied by corresponding test updates.

Below is the gitlab CI template to achieve above goals.

stages:
  - test

variables:
APPLICATION_NAME: "org.kde.kate"

before_script:
  # Set up your environment before running the pipeline
  # Example: Install dependencies, set up the build environment, etc.

# Test Stage
test:
  stage: test
  script:
    # Execute test commands here
    # Example: Run your test suite or call test scripts
    - ./scripts/test_scripts/$APPLICATION_NAME/run_tests.sh

# Add additional stages as needed for deployment, etc.

Measuring in Wayland

The goal of issue "measuring in wayland" is to measure the energy consumption of KDE software when running on the Wayland display server.

How to approach the issue

To accomplish this, we'll need to write usage scenarios using a tool that runs on Wayland. The KdeEcoTest and the Selenium Webdriver using AT-SPI seem to be relevant tools for this purpose.

While Wayland support is available in Selenium, so we going to use it for script development!! Once scripts with wayland capabilities are ready (Selenium with at-spi), we can seamlessly integrate them into KEcolab for wayland environment testing.

While it's not possible to directly switch sessions from Xorg to Wayland using SSH or a single remote command, here are approaches for achieving similar outcomes:

  1. Logging out and selecting Wayland:

    Before logging in:

    • At the login screen, click the gear icon (usually in the bottom right corner).
    • Choose the "Ubuntu on Wayland" or similar option. Log in as usual.

    For remote switching:

    • Use SSH to log in to the remote machine.
    • Issue a command like gnome-session-quit --no-prompt or xfce4-session-logout to log out the current session.
    • The login screen will appear, allowing you to select Wayland as described above.

  2. Forwarding X11 applications over Wayland (SSH -X):

  • Requirements:
    • xorg-xauth installed on the remote machine.
    • xorg-xwayland installed on the Wayland machine.
    • X11Forwarding yes in the remote machine's /etc/ssh/sshd_config file.
  • Steps:
    • Use ssh -X to connect to the remote machine.
    • Run X11 applications on the remote machine; they'll be displayed on your Wayland machine.

Timeline

(What/when you plan to do the different tasks, potentially with milestones)

Project Plan Summary:

We will tackle high-priority issues first. If collaborating with the Okular team on "integrating the measurement workflow" in their repository remains unclear , we will shift our focus to "measuring in WayLand".

Community Bonding Period [17 Jan - 20 Jan]

  • Discuss the project idea as well as any improvements or edge cases with mentors
  • Set up the project repository
  • Learn more about unit testing, CI-CD process, Okular, Kde Eco.

Deliverables Timeline

1. Testing KEcoLab and bug squashing [21 Jan - 10 Feb]

  1.1. Benchmarks : Define and list the benchmarks for testing, with a focus on both performance and script 
       compatibility.

  1.2. Develop Test Cases: Begin drafting 1-2 detailed test cases, specifying inputs, execution steps, and 
       expected results for comprehensive testing.

  1.3. Test Script Compatibility: Concentrate on evaluating KEcoLab's compatibility with available 
       application scripts. Also, **collaborate with KDEEcotest and Selenium** to ensure compatibility with 
       their script execution methods.Run tests involving different script types and document the results.

  1.4. Address Compatibility Issues: Address any compatibility **issues discovered** during testing. 
       Document the issues and provide solutions or workarounds for each.

  1.5. Bug Identification and Stress Testing: Review the results of **previous tests** to identify any 
       reported bugs and their status. Conduct additional tests with a focus on intentionally stressing the 
       system to identify potential bugs.

  1.6. Report and Collaborate on Bug Resolution: Report all identified bugs, providing detailed information 
       and steps to reproduce. Collaborate actively with the mentor and the community to discuss and resolve 
       bug reports effectively.

2. Create tests for KEcolab [11 Feb - 28 Feb]

  2.1. Examine the current GitLab CI file and Kate test scripts. Understand how tests are organised and 
       executed.
  2.2. Work with mentors to identify critical scenarios and functionalities that should be covered by CI 
       tests.
  2.3. Create CI tests and ensure they will run as expected.   
  2.4. Run the CI pipeline with the hardcode application name to verify that tests are executed 
       successfully. Include critical scenarios in the initial testing, such as making a proposed change and 
       ensuring the pipeline works as expected. 
  2.5. Document the newly created test cases, their purpose, and instructions for developers on how to run 
       and maintain them.

3. Integrating measurement workflow in Okular repository [1 March- 24 March]

  3.1. Planning and Initial Setup: Review requirements and goals for integrating the measurement workflow in 
       Okular for KEcolab and discuss the task with the mentors to clarify any ambiguities and gather more 
       details.

  3.2. Engagement with Okular Team: Draft an initial communication to the Okular development team about the 
       measurement workflow integration. Actively participate in discussions with the Okular development 
       team based on their feedback and establish communication channels for collaboration and feedback.

  3.3. Build Stage and Initial Testing: Explore existing GitLab CI templates. Design and create a CI 
       template tailored for KEcolab measurements. Test the measurement workflow locally to identify any 
       issues or improvements.

  3.4. Outreach: Send the drafted communication to the Okular development team, seeking their input and 
       engagement in the measurement workflow integration.

  3.5. Deployment: Add a deploy stage to the CI pipeline. Test the complete CI pipeline locally, including 
       the deploy stage. Address any issues and ensure the pipeline completes successfully. Coordinate with 
       mentors for a final review.

4. Measuring in Wayland (when Wayland support is available in KDE Eco Tester or Selenium)

  4.1. Understand the current state of KDE software testing in wayland and identify specific KDE , 
       applications to test.
  4.2. Research about tool for testing (Selenium with AT-SPI).
  4.3. Create detailed usage scenarios for selected KDE applications using Selenium and ensure scripts are 
       compatible with both Xorg and Wayland.
  4.4. Update CI pipeline configuration file to include a section for session switching. 
  4.5. Add the necessary commands or scripts to enable Wayland in the CI environment and 
       ensure that the pipeline can seamlessly run tests in both Xorg and Wayland sessions. 
  4.6. Write a comprehensive report summarising my findings and share it with the mentors for 
       feedback.

5. Final-Term Evaluations [August 25 to August 31]

Foreseen challenges

(If your project may have challenges, feel free to add any info here and if you have any plan to resolve them if they appear)

  1. Lack of Diversity in Scripts: The absence of a diverse range of scripts may impact the thoroughness of our testing approach, especially in the areas of Benchmarks, Performance Testing, Compatibility Testing, and Bug Identification.

    Solution: Make the most of the available scripts within the KDE ecosystem. Understand their functionalities and use them to the fullest extent, even if the diversity is limited. For additional scripts we can collaborate with selenium and KdeEcotest team

  2. Complexity of Measurement Workflow: Okular developers may have ongoing priorities and commitments, and integrating a measurement workflow might not be their immediate focus

    Solution: Align the project goals with Okular's overarching objectives. Clearly articulate how the measurement workflow can benefit Okular's development and user experience.

  3. Integrating with CI/CD Infrastructure: Integrating the measurement workflow seamlessly into the GitLab CI infrastructure for Okular may require adjustments and compatibility considerations.

    Solution: Collaborate with the maintainers for Okular. Follow best practices for GitLab CI integration. Test the workflow in a controlled environment before production use.

References / relevant background info

(Contributions in KDE or other project with Gitlab/Github links, and should we choose you to carry this task)

I am a open source advocate and passionate about automation and devops. I have a good knowledge of building CI-CD pipelines, troubleshooting, writing unit test.

Recently, I worked as a LFX Mentee at Cloud Native Computing Foundation (KubeEdge) and also as a Google Arcade. I have been contributing to Free & Open Source projects for a long time, and have a keen interest in this field. I have been an active contributor to various open-source organizations, such as Kubernetes, Docker, Armada, Buildspack, Konveyor.

My journey with KDE started some months back. I found the community very helpful, and it helped me to enhance my developer skills. I am also part of the Open Source Society(OSDC) of my college, where I regularly give talks and create awareness about the open-source culture.

I also have a passion for writing blog, through which I share my learning and experiences

Here are some of my profiles on different platforms:

Here are some of my previous contributions to KDE: teams/eco/remote-eco-lab#9

How to reach you

(Element / IRC nickname / email)

You can reach to me at @s-thak:matrix.org on matrix or can drop a mail at sarthaknegi908@gmail.com. I will reply within 24 hrs.

Note: we don't need (yet) your personal address/phone, it will be necessary only if you succeed the SoK and wants to get goodies.

/cc @teams/season-of-kde

Edited by Sarthak Negi