diff --git a/handbook/README.md b/handbook/README.md new file mode 100644 index 0000000000000000000000000000000000000000..0e443557dba1c857c78e2abd9966b17170c1faa3 --- /dev/null +++ b/handbook/README.md @@ -0,0 +1,9 @@ +The handbook is broken up until smaller sections in the directory `sections/`. + +To build full handbook from the individual sections: + +$ `cat sections/0* sections/1* sections/2* sections/3* sections/4* > handbook.md` + +Alternatively one can build parts of the handbook by indicating which sections to include. For instance, the following command will only include the sections in part one. + +$ `cat sections/1* > handbook_partI.md` diff --git a/handbook/sections/0_front-matter.md b/handbook/sections/0_front-matter.md new file mode 100644 index 0000000000000000000000000000000000000000..afc1cfb5740ca67e0b11e25ffbd58a82829f790e --- /dev/null +++ b/handbook/sections/0_front-matter.md @@ -0,0 +1,8 @@ +--- +author: KDE Eco +date: 2022-09-09 +title: Applying the Blue Angel criteria to Free Software +--- + +== TODO == + diff --git a/handbook/sections/1.0_digitization-energy-consumption.md b/handbook/sections/1.0_digitization-energy-consumption.md new file mode 100644 index 0000000000000000000000000000000000000000..990ab952e2806a61e625878007af618a65c87c38 --- /dev/null +++ b/handbook/sections/1.0_digitization-energy-consumption.md @@ -0,0 +1,2 @@ +# Part I: The Energy Consumption Of Digitization + diff --git a/handbook/sections/1.1_oxymoron-tautology.md b/handbook/sections/1.1_oxymoron-tautology.md new file mode 100644 index 0000000000000000000000000000000000000000..b0773d706b9ce4b675743ca306830a8e21696084 --- /dev/null +++ b/handbook/sections/1.1_oxymoron-tautology.md @@ -0,0 +1,12 @@ +## Oxymoron or Tautology? + +In 2021 the Association of Computing Machinery (ACM) released a Technology Policy Council report entitled "[Computing And Climate Change](https://www.hpcwire.com/off-the-wire/acm-technology-policy-council-releases-techbrief-on-computing-and-carbon-emissions/)". Founded in 1947 the ACM is the oldest scientific and educational computing society in the world. In the policy report the researchers acknowledge an inherent contradiction of digitization: on the one hand digital technology "can help mitigate climate change", but to do so it "must first cease contributing to it" (p. 1). + +The report's estimates are alarming. At the time of the report's publication, the Information and Communication Technology (ICT) sector is already estimated to contribute between 1.8-3.9% of global carbon emissions. To put this into perspective, the aviation industry is estimated to contribute 2.5% of all emissions. Most alarmingly, the report warns: if nothing is changed by 2050 the carbon emissions attributable to the ICT sector will rise to more than 30% of all emissions globally. + +As digital technology has revolutionized how we live, it has often been extolled for replacing its analogue and more wasteful counterparts. We were reading about the benefits of future paperless offices at the end of the 20th century. And in the first decades of the 21st century, we have been able to replace -- with the global Coronavirus pandemic pushing things rapidly forward -- in person activities with video meetings for everything, from mundane office meetings to global academic conferences to local piano recitals. With virtual/augmented reality in the metaverse these online gatherings will soon become full digital immersion experiences. + +At first glance it would appear that, vis-a-vis the analogue, digital technology will more efficiently achieve the same, or at least comparable, result. By sending colleagues an email you use less resources than sending the same information by mail. By attending your work meeting online you will contribute fewer greenhouse gas emissions than by driving -- or even worse, by flying -- to attend in person. + +Second glances, however, are when reality becomes more complicated. The title of a [talk](https://www.tele-task.de/lecture/video/9104/) by researchers Henning Heitkötter and Nico Wottke frames the issue concisely: "Digital Systems and Energy Efficiency: Oxymoron or Tautology?" + diff --git a/handbook/sections/1.2_material-footprint.md b/handbook/sections/1.2_material-footprint.md new file mode 100644 index 0000000000000000000000000000000000000000..1fad0287f8d7747d7627bf5b4645ac8bd67d7584 --- /dev/null +++ b/handbook/sections/1.2_material-footprint.md @@ -0,0 +1,26 @@ +## Energy Consumption & Carbon Emissions + +Before continuing, it is crucial to point out that energy consumption is not the same as carbon emissions. Carbon emissions will depend on the mix of fuels used for generating electricity in a particular region, referred to as the [electricity or power generation mix](https://www.planete-energies.com/en/medias/close/what-power-generation-mix). + +As an example, for the European Union's 28 member states' [energy supply](www.bpb.de/nachschlagen/zahlen-und-fakten/europa/75140/themengrafik-energiemix-nach-staaten) in 2016 -- according to Germany's Federal Agency for Civic Education ('Bundeszentrale für politische Bildung') -- the power generation mix included 32.9% based on oil, 23.9% gas, 14.9% coal, 13.7% nuclear, and 14.5% from renewables. It is necessary to estimate the carbon emission intensity (or CEI) calculated from this mix to calculate the carbon footprint from electric energy consumption. As an example, using 100% carbon-neutral energy sources means having a 0% CEI, which no matter how much energy is consumed means 0 grams carbon per kilowatt-hour energy consumed. + +Note that here I will refer to greenhouse gas emissions or energy consumption depending on the information provided in the cited text. + +## Material Footprint Of Digital Technology + +Unlike the physical materials comprising the mechanical and analogue world, in the public's imagination digital technology is often, and erroneously, regarded as being immaterial. But there is a very real and hard material aspect to digitization: devices such smartphones and laptops, the processing plants to mine the metals and the ships to transport the products, and the cables and data centers connecting the world. The report ["Lean ICT: Towards Digital Sobriety"](https://theshiftproject.org/wp-content/uploads/2019/03/Lean-ICT-Report_The-Shift-Project_2019.pdf) from the SHIFT Project describes it so: + +> [T]he material footprint of digital technology is largely underestimated by its users, given the miniaturization of equipment and the "invisibility" of the infrastructures used. This phenomenon is reinforced by the widespread availability of services on the "Cloud", which makes the physical reality of uses all the more imperceptible and leads to underestimating the direct environmental impacts of digital technology. (p. 10) + +So what is contributing to this atmospheric rise of greenhouse gas emissions from the ICT sector? The ACM policy report above points out that between 2012 and 2018 the energy demands of artificial intelligence (AI) has increased 300,000 times. And today it currently doubles every few months. To capture the energy consumption of AI in more relatable terms: [one study](https://doi.org/10.48550/arXiv.1906.02243) found that training a single AI model, in this case a model used in machine translation and language modeling, required the energy equivalent of flying from New York to San Francisco round-trip ... 300 times -- or about 626,000 pounds of carbon dioxide equivalent. Such high energy demand contributes to the doubling in energy consumption of data centers over the past 10 years, currently reported by the ACM report to consume about 3% of all eletricity supply. Another notorious contributor to this energy consumption is blockchain technology, in particular proof of work systems such as Bitcoin, which [reportedly](https://hbr.org/2021/05/how-much-energy-does-bitcoin-actually-consume) consume as much energy as entire countries like Sweden or Malaysia. + +Moreover, the number of internet-connected devices---including not only laptops and smartphones but also smart TVs, home assistants as well as IoT devices---is exploding. By 2025 the number is expected to surpass 75 billion, or about 9-10 devices for every person on earth. (Of course, the distribution is far from even, with many devices per person in certain parts of the world and digital deserts in others. ) Production and transportation of these devices---not to mention end of life treatment---as well as the network traffic they contribute to all demand energy. According to the SHIFT report (2019), the number of smartphones in particular is growing rapidly, as is the number of features requiring "increasingly diverse range of metals" and the energy intensive processes to mine them. + +Indeed, what does the distribution of energy consumption look like exactly? The Shift Project, a French nonprofit aiming to transition away from fossil fuels, released an analysis in 2019 entitle "[Lean IT--Towards Digital Sobriety]()". Consider the pie chart below from the report. Although the data was collected before the pandemic in 2017, and current numbers may look different, the report provides a relevant starting point to discuss energy consumption in more detail. + +![Energy Consumption Distribution](images/sec1_distribution-energy-consumption.png) + +In this chart, one can see that production and usage divide the pie in half, with 45% of energy consumption attributed to the production of personal computers, mobile phones, TVs and home entertainment systems (top right) and 55% to their usage (bottom left). It is important to note that this report does not include numbers related to transportation nor end of life treatment of these devices.[^1] + +[^1]: Moreover, the devices not included in these numbers are: printers and multi-function devices; digital and video cameras; music players and similar digital media and stand-alone video player devices; network connected white goods; smart thermostats; home energy management; security systems; satellites; personal drones; robots; driverless automotives; and portable batteries, 'power banks'. "While the excluded categories are not currently a significant contributor to electricity usage, compared to the ones included, some of them and others might emerge over the coming decade" (Andrae & Edler 2015: pp. 118--9). + diff --git a/handbook/sections/1.3_rebound-effects.md b/handbook/sections/1.3_rebound-effects.md new file mode 100644 index 0000000000000000000000000000000000000000..bb0967f9b5f0be761e423a1041dcab22d20cd4be --- /dev/null +++ b/handbook/sections/1.3_rebound-effects.md @@ -0,0 +1,10 @@ +## "The Very Contrary Is The Truth": Rebound Effects and Jevon's Paradox + +Rebound, also known as take-back effects, is when efficiency gains can decrease or even negate efficiency benefits. The concept is straightforward: imagine that a change in software results in a 5% improvement in energy efficiency; however, now image that given these increased savings you use the software more. As a result, there is, in total, only a 1% drop in overall energy consumption. The [rebound effect](https://energyskeptic.com/2014/jevons-paradox-the-rebound-effect/) is thus 80% ((5-1)/5): the conservation gains from the 5% efficiency increase are reduced by 80% given the increased usage as a result. + +If the rebound effect goes over 100%, meaning more energy is now used than before, this is referred to as the [Jevon's Paradox](http://en.wikipedia.org/wiki/Jevons_paradox), or 'back-fire'. The paradox comes from the English economist William Stanley Jevons, who in 1865 recognized that technological improvements in coal-use actually increased coal consumption across industries. Jevon's conclusion: + +> "It is a confusion of ideas to suppose that the economical use of fuel is equivalent to diminished consumption. The very contrary is the truth." + +One practical interpretation of this paradox: efficiency gains must be combined with conservation to have a meaningful effect. The ACM report (p. 1) makes a similar point: "Computing-enabled efficiencies must be coupled with slashed energy demand to reduce ICT sector carbon emissions." + diff --git a/handbook/sections/1.4_blowback.md b/handbook/sections/1.4_blowback.md new file mode 100644 index 0000000000000000000000000000000000000000..813950673a8420aceac1ab908052fa204f373e1d --- /dev/null +++ b/handbook/sections/1.4_blowback.md @@ -0,0 +1,22 @@ +## Thinking About The Pushback + +Considering the bigger picture, software's contribution to global greenhouse gas emissions may seem insignificant if compared to other industries. + +There are a few issues with this critique. The first is what is referred to as the ["not as bad as"](https://rationalwiki.org/wiki/Not_as_bad_as) fallacy, also known as "fallacy of relative privation". That software's contributions to GHG is not as bad as another industry's contributions does not negate software's contributions. + +![Bigger Problem](images/sec1_bigger_problem.png) + +Moreover, we can reduce software's contributions AND reduce another industry's, so the argument suggests a false choice. As has been shown here, software can have huge impacts on energy consumption, especially when considered at scale. + +What is more, in order to make an argument about relative harm it is necessary to have calculations about the actual effects. We do not always have the data, or it is challenging to get an accurate estimate. With this handbook, we hope to help change that! + +One could argue that the discussion here is too focused on individuals, since combatting the climate crisis requires structural changes. What is worse, [evidence](https://doi.org/10.1016/j.oneear.2021.04.014) suggests that major contributors to global greenhouse gas emissions like ExxonMobile embrace a rhetoric of individual responsibility to deflect from its own role in the crisis. + +A zero-emissions future will require fundamental shifts in how we live, and that responsbility should not be individualized. However, as anthropologist Margaret Mead said: + +> "Never doubt that a small group of thoughtful, committed citizens can change the world; indeed, it’s the only thing that ever has." + +Structural change will happen when committed individuals organize to combat the issues facing society. + +Combatting software-driven energy consumption is what we can do, together. + diff --git a/handbook/sections/2.0_BE-case-study_okular.md b/handbook/sections/2.0_BE-case-study_okular.md new file mode 100644 index 0000000000000000000000000000000000000000..3e5f2d20a6cbcc118e64ee5d9e5c5c86b8c6d1f2 --- /dev/null +++ b/handbook/sections/2.0_BE-case-study_okular.md @@ -0,0 +1,5 @@ +# PART II: Okular, A *Blue Angel* Case Study + +== TODO == + +![Okular awarded the Blue Angel ecolabel](images/sec2_okular-BE-logo.png) \ No newline at end of file diff --git a/handbook/sections/3.0_how-to.md b/handbook/sections/3.0_how-to.md new file mode 100644 index 0000000000000000000000000000000000000000..aa9fdb4eeb19067619b9b60d3c493d7044f51b96 --- /dev/null +++ b/handbook/sections/3.0_how-to.md @@ -0,0 +1,8 @@ +# Part III: How To Measure Your Software's Energy Consumption + +The lab setup consists of a power meter, a computer to aggregate and evaluate the power meter output, and a desktop computer for the system under test where user behavior is emulated. + +The setup described here follows the specifications from the [Blue Angel Basic Award Criteria for Resource and Energy-Efficient Software Products](https://produktinfo.blauer-engel.de/uploads/criteriafile/en/DE-UZ%20215-202001-en-Criteria-2020-02-13.pdf). + +Terminology is in part from Kern et al. (2018): *[Sustainable software products--Towards assessment criteria for resource and energy efficiency](https://doi.org/10.1016/j.future.2018.02.044). + diff --git a/handbook/sections/3.1_workflow.md b/handbook/sections/3.1_workflow.md new file mode 100644 index 0000000000000000000000000000000000000000..0ccbfdd2e608d0b64e74e0bd693566580cd4d666 --- /dev/null +++ b/handbook/sections/3.1_workflow.md @@ -0,0 +1,306 @@ +## Overview Of Laboratory Setup + +The laboratory setup requires 1 power meter and at least 2 computers: + +- **Power Meter** (PM) + + One of the recommended devices is the [Gude Expert Power Control 1202 Series](https://www.gude.info/en/power-distribution/switched-metered-pdu/expert-power-control-1202-series.html) ([manual](https://gude-systems.com/app/uploads/2022/05/manual-epc1202-series.pdf)). It provides plugs for powering the computer and measures the current during operation. The device can be controlled and read via cabled Ethernet. There is a web-based user interface, a [REST API](http://wiki.gude.info/EPC_HTTP_Interface), and the device supports various protocols such as SNMP or syslog. + + - **Computer 1: Data Aggregator & Evaluator** (DAE) + + The computer for collecting and evaluating results from the power meter. + + A Python script (MIT) to read out the data from the Gude Expert Power Control 1202 Series with SNMP is available at the [FEEP repository](https://invent.kde.org/teams/eco/feep/-/blob/master/tools/GUDEPowerMeter/check_gude_modified.py). + + ![Lab Setup](images/sec3_gude-pm.jpg) + + Other PMs may require non-Free software, e.g., Janitza's [GridVis Power Grid Monitoring Software](https://www.gridvis.com/gridvis-overview.html).[^1] + +[^1]: In Mai (2021), Seiwert & Zaczyk (2021), and the OSCAR Manual the visualization software GridVis is used to read out the data recorded by the Janitza UMG 604 power meter, which required Windows OS. Note that with the Janitza UMG 604 power meter it is possible also to download the power measurement data directly from the power meter, but it is recommended to monitor progress live with the second computer in order to ensure everything is proceeding smoothly. + + - **Computer 2: Reference System / System Under Test** (SUT) + + The reference system is the hardware used to measure the energy consumption of the system under test, or SUT. The SUT includes the operating system (**OS**) and software installed for (i) testing the software product, (ii) emulating the standard usage scenario[^2] and (iii) collecting the hardware performance results. Note the following: + +[^2]: In Kern et al. (2018) they describe a setup using 3 computers, with the standard usage scenario emulation generated on a computer independent of the SUT. Details of a similar setup for an external workload generator can be found [here](https://invent.kde.org/teams/eco/feep/-/tree/master/tools/arduino-board_external-workload-generator.md). + + For GNU/Linux systems, the [Blauer Engel criteria (Section 1.1)](https://produktinfo.blauer-engel.de/uploads/criteriafile/en/DE-UZ%20215-202001-en-Criteria-2020-02-13.pdf) recommend Fujitsu computers as the reference system. + + For emulating activity in the standard usage scenario, Free Software task automation tools such as [*xdotool*](https://github.com/jordansissel/xdotool), [*KDE Eco Tester* (in progress)](https://invent.kde.org/teams/eco/feep/-/tree/master/tools/KdeEcoTest), or [*Actiona*](https://wiki.actiona.tools/doku.php?id=en:start) (GPLv3) can be used. + + For collecting hardware performance data (e.g., processor utilization, RAM utilization, hard disk activity, network traffic), a Free Software tool such as [*Collectl*](https://sourceforge.net/projects/collectl/) (GPLv2/Artistic License) can be used. + +![Lab Setup](images/sec3_lab-setup_modified.png) + + +## Preparation Of The Lab + +### Reference System + +The Fujitsu Esprimo P920 Desktop-PC proGreen selection (Intel Core i5-4570 3,6GHz, 4GB RAM, 500GB HDD) is one of the recommended reference systems. On the reference system you can set up the System Under Test. + +### System Under Test + +The System Under Test (SUT) must reduce unrelated energy consumption and have a standardized configuration. This includes the following: + + - Overwriting the entire hard drive of the machine with a standardized OS. + + - Deactivating all possible background processes (automatic updates, backups, indexing, etc.). + + - Installing the necessary software (i.e., the application under consideration as well as the emulation and data collection software). + +### Standard Usage Scenario + +Preparing the Standard Usage Scenario requires the following: + + - Identifying tasks users typically carry out when using the application under consideration. + + - Identifying functionalities which require high energy demand or high resource utilization. + + - Based on the above, scheduling a flow chart of individual actions and emulating these actions with a task automation tool. + + - Note that between runs the cache should be cleared and any new files deleted before starting the next measurement. + +An example standard usage scenario for KMail includes: searching for an email, writing a reply or forwarding the email, saving an attachment, deleting a folder in the mail client, etc. See the [Actiona scripts](https://invent.kde.org/cschumac/feep/-/tree/master/measurements) used to test KMail, Krita, and Okular for examples (see the OSCAR Manual for details about using WinAutomation on Windows). + +Note: If the task automation tool uses pixel coordinates to store the position of the automated clicks (e.g., Actiona) and, moreover, the screen resolution of the computer used in preparation differs from that of the laboratory computer, all pixel coordinates will eventually have to be set anew for the laboratory environment; see Seiwert & Zaczyk (2021: p. 12) for details about their experience. + +### Emulation Tools For Usage Scenarios + +An automation tool is required which can run the usage scenarios and does not need human intervention, so it can be run repeatedly in a well-defined way to provide accurate measurements. + +There are some candidates for tools which might meet the requirements: + + - [Actiona](https://github.com/Jmgr/actiona) + + - [xdotool](https://github.com/jordansissel/xdotool) + + - [Atbswp](https://github.com/RMPR/atbswp) + + - [SikuliX](https://github.com/RaiMan/SikuliX1) + +See a list from David Hurka here: [Visual Workflow Automation Tools](https://invent.kde.org/teams/eco/feep/-/tree/master/tools/presentation_Visual_Workflow_Automation_Tools). + +Most of these tools use X11-specific features and thus do not work on Wayland systems. There are a few possible approaches here: + + - [The XDG RemoteDesktop portal](https://docs.flatpak.org/en/latest/portal-api-reference.html#gdbus-org.freedesktop.portal.RemoteDesktop) + + - Various Wayland protocols: + - + - + + Support varies between compositors. + + - [libinput user devices](https://lwn.net/Articles/801767/) + +## Reading Data From Gude Expert Power Control 1202 Series Power Meter + +There is a script available to read data from the device via SNMP: + +### Other Devices + +As an alternative it is possible to [repurpose cheap switchable power plugs as measurement devices](https://volkerkrause.eu/2020/10/17/kde-cheap-power-measurement-tools.html); see Section [2.6](#sec:gosundPM){reference-type="ref" reference="sec:gosundPM"} for set up instructions. + +## Measurement Process + +The measurement process is defined in Appendix A of the [Basic Award Criteria](https://produktinfo.blauer-engel.de/uploads/criteriafile/en/DE-UZ%20215-202001-en-Criteria-2020-02-13.pdf). It requires one to record energy data and performance indicators with a granularity of 1 second and log it so it can be processed and average values can be calculated. + +Some general comments: + + - Times between the PM and DAE must be syncronized. + + - On the DAE confirm that the desired power outlet is read out (e.g., via the live graph created when using LabPlot). + + - When using Collectl to collect performance load, ensure it is running in the console of the reference system; furthermore, it is recommended to check that the required csv file is also correctly generated. + + - Since each run of the usage scenarios results in changes to the standard operating system, clearing the cache between runs is recommended. + + - All runs (Baseline, Idle Mode, Standard Usage Scenario) must be for the same length of time, based on the time needed to run the Standard Usage Scenario script. + +During the energy measurement we also need to record a set of performance indicators: processor utilisation, RAM utilisation, hard disk activity and network traffic. Tool candidate: + +- [Collectl](https://sourceforge.net/projects/collectl/) + +Mai (2021: p. 15) recommends the following command for obtaining hardware performance data with Collectl: + +$ `collectl -s cdmn -i1 -P --sep 59 -f /var/log/performanceMeasures.csv` + +The specified options are: + + - `-s cdmn` + + collect CDU, Disk, memory, and network data + + - `-i1` + + sampling interval of 1 second + + - `-P` + + output in plot format (separated data which consists of a header with one line per sampling interval) + + - `--sep 59` + + semicolon separator for -P option + + - `-f /PATH/TO/FILE.csv` + + save file at specified path + +### Baseline & Idle Mode + +- **Baseline**: *Operating System* (OS) + + To establish the baseline electrical power usage and hardware performance data of the system under test, a scenario is measured in which the OS is booted but no action is taken. + +- **Idle Mode**: *OS + Application While Idle* + + To establish the electrical power usage and hardware performance data of the application while idle, a scenario is measured in which the application under consideration is opened but no action is taken. + +Important: the baseline and idle mode are run for the same time needed to carry out the standard usage scenario. In Seiwert & Zaczyk (2021) both measurements were repeated 10 times, taking about 70 minutes each (140 minutes in total). Since the power consumption for the baseline and idle scenario is relatively uniform, 10 repetitions for each is considered sufficient to obtain a representative sample. + +### Standard Usage Scenario + +Standard Usage Scenario: To measure the electrical power usage and hardware performance data of the application under consideration in use, the standard usage scenario is run. In Seiwert and Zaczyk (2021), the measurement of the standard usage scenario was repeated 30 times, taking about 4 hours in total. This higher number of repetitions was necessary to obtain a representative sample, as the electrical power usage and performance data may vary across measurements. + +## Analysis Of The Results With OSCAR + +There is a tool available from Umwelt-Campus Birkenfeld which generates reports from measurement data, called OSCAR (Open Source Software Consumption Analysis): + + - [Source Code](https://oscar.umwelt-campus.de/) + + - [Running instance](https://gitlab.umwelt-campus.de/y.becker/oscar-public). + +At the project website there is the Oscar Manual with detailed instructions including screenshots on how to use OSCAR. + +### Files Needed + +The analysis with OSCAR requires uploading the following files to the [OSCAR website](https://oscar.umwelt-campus.de/): (i) a [log file of actions](#orga5061f0) taken, (ii) the [electrical power usage](#orgc039d51), and (iii) the [hardware performance data](#org3261c21). All files are .csv files, examples of which are provided below. Important: some preprocessing of the raw data may be necessary (e.g., performance data measured by Collectl; see [this section](#org7c54e7e) for details). + +#### Log File Of Actions + +A log file of actions taken will have the following format (OSCAR Manual: Section 3.1.1): + +| | | | +|--|--|--| +| timestamp; | startTestrun | | +| timestamp; | startAction; | action1 | +| timestamp; | stopAction | | +| timestamp; | startAction; | action2 | + +#### Electrical Power Usage + +The electrical power usage will have the following format (OSCAR Manual: Section 3.1.2): + +| | | +|------------|--------| +| timestamp; | value1 | +| timestamp; | value2 | +| timestamp; | value3 | +| timestamp; | value4 | + +Note: (i) the timestamp increases in one-second increments and (ii) *value* stands for 'averaged measured value per second in watts'. + +Below is an example with header from the OSCAR Manual (measurements are from the Janitza UMG 604 power meter exported on the DAE using the GridVis interface): + +| Row-Number; | Time; | Valueavg[W]; | Value-min[W]; | Value-max[W]; | +|-------------|-------|--------------|---------------|---------------| +| 1; | 18.01.12 17:00:01; | 78,573; | 78,281; | 78,982; | +| 2; | 18.01.12 17:00:02; | 78,611; | 78,390; | 78,972; | +| 3; | 18.01.12 17:00:03; | 78,556; | 78,353; | 78,887; | +| 4; | 18.01.12 17:00:04; | 78,589; | 78,391; | 78,882; | +| 5; | 18.01.12 17:00:05; | 78,546; | 78,288; | 79,057; | +| 6; | 18.01.12 17:00:06; | 78,580; | 78,252; | 78,941; | +| 7; | 18.01.12 17:00:07; | 80,712; | 78,330; | 87,391; | +| 8; | 18.01.12 17:00:08; | 78,545; | 78,275; | 79,202; | + +Note: The columns above containing min and max are ignored by OSCAR. + +#### Performance Data (Raw) + +The hardware performance data will have the following format (OSCAR Manual: Section 3.1.3): + +== INSERT EXAMPLE DATA == + +Note: The timestamp again increases in one-second increments. + +Below is an example of the preprocessed results from Collectl running on Ubuntu 16.04 LTS (OSCAR Manual). The measurements that need to be specified in OSCAR are: [CPU]Totl = Processor; [MEM]Used = Main memory - used kilobytes; [NET]RxKBTot = Network - Kilobytes received/s; [NET]TxKBTot = Network - Kilobytes sent/s; [DSK]ReadKBTot = Disk - Kilobytes read/s; and [DSK]WriteKBTot = Disk - kilobytes written/s. + +See below for notes about preprocessing Collectl results for use with OSCAR. + +== INSERT EXAMPLE DATA == + +### Preprocessing Data + +Before using OSCAR some preprocessing of the data may be necessary. For example, when using Collectl for performance data of the SUT it is necessary to do the following before uploading the data to OSCAR (see Seiwert & Zaczyk 2021: p. 13 for details; see also Appendix A 2 on p. 46 for a Python script to automate some of these tasks): + + - All information above the header row can be removed. + + - Remove all \# characters from the file. + + - In the first column the semicolon between the data/time and data must be removed or the date and time will be interpreted as two separate columns. + + - Moreover, the date should have a character inserted between YYYYMMDD, e.g., YYYY/MM/DD. Whatever character is used must be specified in OSCAR. + + - The file must be saved in csv format. + +### Analysis With OSCAR + +#### Uploading The Results + +Once the necessary files have been preprocessed, the evaluation of the baseline, idle mode, and standard usage scenario can begin. Note that the baseline measurements are uploaded separately from the idle mode/standard usage scenario measurements. + +When uploading the idle mode measurements, for *Art der Messung* ('Type of Measurement') select *Leerlauf* ('Idle Mode'); when uploading the standard usage scenario measurements select *Nutzungsszenario* ('Use Scenario'). + +For each measurement, the log file of actions taken, the electrical power usage, and the hardware performance data are uploaded. + +![OSCAR screenshot](images/sec3_oscar.png) + +In the OSCAR interface, note the following: + +- The duration of the individual measurements (for instance, the duration of a measurement in seconds) must be specified. + +- The formatting of the data must be specified. + +- Only the semicolon must be specified as separator. + +- For the performance output of the hardware, it is necessary to select the columns that are relevant for evaluation. + +- As a last step, the correct formatting of the time stamp must be specified for each of the uploaded files. + +An example of time stamp formatting specified in OSCAR is as follows: + +*Aktionen* + +%Y-%m-%d %H:%M:%OS + +*Elektrische Leistung* + +%d.%m.%y %H:%M:%OS + +*Hardware-Auslastung* + +%Y/%m/%d %H:%M:%OS + +Note that in the above example the character used for the date in the preprocessing of the Collectl hardware performance data (*Hardware-Auslastung*) is a slash, i.e., YYYY/MM/DD. + +For more details, including screenshots, see the OSCAR Manual (Section 3.3). + +#### Downloading The Reports + +After completing the above, the report can be generated and downloaded, resulting in two documents: one is a report for the baseline and idle mode, and the other is a report for the baseline and standard usage scenario. + +## Notes + +This workflow is a summary of the process described in detail in the following documents: + +- [Resource and Energy-Efficient Software Products: Basic Award Criteria (Edition January 2020, Version 1)](https://www.blauer-engel.de/en/productworld/resources-and-energy-efficient-software-products) + +- Mai, Franziska (2021) *Vergleichende Analyse und Bewertung von Betriebssystemen hinsichtlich ihrer Energieeffizienz* (German only), + +- Seiwert & Zaczyk (2021) [*Projektbericht: Ressourceneffiziente Softwaresysteme am Beispiel von KDE-Software*](https://invent.kde.org/cschumac/feep/-/blob/master/measurements/abschlussbericht-kmail-krita.pdf) (German only), + +- [OSCAR Manual](https://gitlab.umwelt-campus.de/y.becker/oscar-public/blob/master/OSCAR/static/OSCAR_Gesamt.pdf) (German only), and + +- Section 4.1 of Kern et al. (2018), *[Sustainable software products--Towards assessment criteria for resource and energy efficiency](https://doi.org/10.1016/j.future.2018.02.044)*. + diff --git a/handbook/sections/3.2_gosund.md b/handbook/sections/3.2_gosund.md new file mode 100644 index 0000000000000000000000000000000000000000..abf37c0d3b6635211ee7838da7813e9ea43edafb --- /dev/null +++ b/handbook/sections/3.2_gosund.md @@ -0,0 +1,142 @@ +## Alternative PM: Gosund SP111 Setup {#sec:gosundPM} + + - (0) Prerequisite: already flashed with a sufficiently new Tasmota version. + + - (1) Firmware Reset. + + If the device had previously been connected to another Wifi it might need a full reset before being able to connect to a new one. + + If the device did open a WiFi access point named "tasmota-XXXXX" this is not needed, continue at (2) directly. + + Press the button for 40 seconds. + + The device will restart and you should be able to continue at (2). + + - (2) WiFi Setup + + The device opens a WiFi access point named "tasmota-XXXXX", connect to that. + + Open in a browser. + + The device asks you for the WiFi name and password to connect to after entering those, the device will reconnect to that WiFi and disable its access point. + + While doing that it should show you its new address in the browser, make a note of that. + + In case that didn't happen, check your WiFi router for the address of the device. + + - (3) Tasmota Setup + + Open the address from step (2) in a browser. + + You should see the Tasmota web UI (a big "ON/OFF" text and a bunch of blue and one red button). + + Click "Configuration". + + Click "Configure Other". + + Copy + + {"NAME":"Gosund SP111 2","GPIO": + [56,0,57,0,132,134,0,0,131,17,0,21,0],"FLAG":0, + "BASE":18} + + into the template input field. + + Tick the "Activate" checkbox. + + Click "Save". + + The device will restart, connect to it again. + + The UI should now also contain text fields showing electrical properties, and the "Toggle" button should now actually work. + + - (4) Calibration + + Open the address from step (2) in a browser. + + Connect a purely resistive load with a known wattage, such as a conventional light bulb (not an LED or energy saving bulb). + + Switch on power by clicking "Toggle" if needed. + + Verify that the "Power Factor" value is shown as 1 (or very close to 1); if it is lower the current load is not suited for calibration. + + Click "Console". + + Enter the following commands one at a time and press enter: + + AmpRes 3 + VoltRes 3 + EnergyRes 3 + WattRes 3 + FreqRes 3 + SetOption21 1 + VoltageSet 230 + + Enter the command PowerSet XXX with XXX replaced by the wattage specified for the test load (e.g. "40" for a 40W light bulb). + + Click "Main Menu". + + The main page now should show correct power readings with several decimals precision + + - (5) MQTT Broker Setup + + The only known way for high-frequency automatic readouts so far is polling over MQTT. This is not ideal and needs additional setup unfortunately. + + If you happen to have a MQTT Broker around already, skip to step (6), otherweise you need to set up one. The below assumes Mosquitto is available packaged on your distribution, and doesn't configure any security, so only do this in your own trusted network and switch it off when not needed. + + - install the `mosquitto` package + - add a file `/etc/mosquitto/conf.d/listen.conf` with the following content: + + listener 1883 + allow_anonymous true + + - start Mosquitto using `systemctl start mosquitto.service` + + - (6) MQTT Tasmota Setup + + Connect to the Tasmota device using a web browser, and open the MQTT configuration page via Configuration > Configure MQTT. + Enter the IP address of the MQTT broker into the "Host" field. + + Note down the value shown right of the "Topic" label in parenthesis (typically something like "tasmota_xxxxxx"). This will be needed later on in addressing the device via MQTT. You can also change the default value to something easier to remember, but this has to be unique if you have multiple devices. + + Press Save. + + The device will restart and once it's back you should see output in its Console prefixed with "MQT". + + - (7) Verifying MQTT Communication + + This assumes you have the Mosquitto client tools installed, which are usually available as distribution packages. + + You need two terminals to verify MQTT communication works as intended. + + - In terminal 1, run `mosquitto_sub -t 'stat//STATUS10'` + - In terminal 2, run `mosquitto_pub -t 'cmnd//STATUS' -m '10'` + + Replace `` with the value noted down in step (6). + + Everytime you run the second command, you should see a set of values printed in the first terminal. + + - (8) Continuous Power Measurements + + See scripts in + + - (9) Switching WiFi Networks + + Once connected to a WiFi network Tasmota will not let you get back to step (2) by default for security reasons, without hard resetting the device (40sec button press). That however also removes all settings and the calibration. If you need to move to a different network, there are less drastic options available though, but those can only be taken inside the network you originally connected to: + + Under Configuration > Configure WiFi you can add details for a second WiFi access point. Those will be tried alternatingly with the first configuration by default. This doesn't compromise security, but requires you to know the details for the network you want to connect to. + + You can configure Tasmota to open an access point as in step (2) by default for a minute or so after boot, and then trying to connect to the known configurations. This makes booting slower in known networks, and opens the potential for hijacking the device, but it can be convenient when switching to unknown networks. This mode can be enabled in the Console by the command `WifiConfig 2`, and disabled by the command `WifiConfig 4`. + + For Tasmota version 11 the 40 sec button press reset can leave the device in a non-booting state, resetting from the Console using `Reset 1` doesn't have that problem, but has to be done before disconnecting from the known WiFi as well. + + - (10) Recovering non-booting devices + + With Tasmota 11 you can end up in a non-booting state by merely resetting the device using the 40 sec button press. This is not permanently damaging the device, but can be fixed with reflashing via a serial adapter. + + The basic process is described in , the PCB layout of the Gosund SP 111 can be seen on . + + In order for this to work, you need to connect GPIO0 (second pin on bottom left in the above image) to GND **before** powering up (ie. before connecting USB). The device LEDs (red and blue) are a useful indicator whether you ended up in the right boot mode, red flashing quickly or red and blue being on is wrong, just red being on is correct. Once in that state the connection can be removed (e.g. if you just hold a jumper cable to the pin), it remains in the right mode until a reboot. + + Most importantly: **DO NOT CONNECT THE DEVICE TO MAIN POWER**! That would be life-threatening, the entire flashing process in solely powered from 3.3V supplied by the serial adapter. Do not do any of this without having read . + diff --git a/handbook/sections/4_back-matter.md b/handbook/sections/4_back-matter.md new file mode 100644 index 0000000000000000000000000000000000000000..8b172e22a3957a5aed1170cfab1998d785c79b2b --- /dev/null +++ b/handbook/sections/4_back-matter.md @@ -0,0 +1,2 @@ +== TODO == + diff --git a/handbook/sections/images/sec1_bigger_problem.png b/handbook/sections/images/sec1_bigger_problem.png new file mode 100644 index 0000000000000000000000000000000000000000..cb9f74de373721c8e56cb38138b9116f4e6f1feb Binary files /dev/null and b/handbook/sections/images/sec1_bigger_problem.png differ diff --git a/handbook/sections/images/sec1_distribution-energy-consumption.png b/handbook/sections/images/sec1_distribution-energy-consumption.png new file mode 100644 index 0000000000000000000000000000000000000000..65f143ca022968bb192bb6f0a20e8ce552766f1d Binary files /dev/null and b/handbook/sections/images/sec1_distribution-energy-consumption.png differ diff --git a/handbook/sections/images/sec2_okular-BE-logo.png b/handbook/sections/images/sec2_okular-BE-logo.png new file mode 100644 index 0000000000000000000000000000000000000000..b3da6f0d8b1a89e6fbb0eeffb61da48a57e84a07 Binary files /dev/null and b/handbook/sections/images/sec2_okular-BE-logo.png differ diff --git a/handbook/sections/images/sec3_gude-pm.jpg b/handbook/sections/images/sec3_gude-pm.jpg new file mode 100644 index 0000000000000000000000000000000000000000..4092799635cade0aeccb63cadb31be68fe8ffb4e Binary files /dev/null and b/handbook/sections/images/sec3_gude-pm.jpg differ diff --git a/handbook/sections/images/sec3_lab-setup_modified.png b/handbook/sections/images/sec3_lab-setup_modified.png new file mode 100644 index 0000000000000000000000000000000000000000..7f8623f39c381fc228ab43574f61a0ba2e080bee Binary files /dev/null and b/handbook/sections/images/sec3_lab-setup_modified.png differ diff --git a/handbook/sections/images/sec3_oscar.png b/handbook/sections/images/sec3_oscar.png new file mode 100644 index 0000000000000000000000000000000000000000..2f1b25dbb0f28abeba29c1615565191efb494e42 Binary files /dev/null and b/handbook/sections/images/sec3_oscar.png differ