Unable to run wordpress-official-data

[manuelleduc]

Hello,
In the process of understanding how green metrics tool is working to evaluate a web application, I started studying the examples.
Sadly, I did not succeed in running the wordpress-official-data example.
See below the steps I followed, based on my current understanding of the tool.

cd /tmp/wordpress-official-data
git init
git add .
git commit -m "first commit"
cd ~/green-metrics-tool # where the tool is installed on my laptop
source ./venv/bin/activate
python3 runner.py --uri /tmp/wordpress-official-data --name wordpress-official-data  --dev-no-sleeps  --allow-unsafe

And see below the logs of the last command

 
Running:  usage_scenario.yml 
 
Checking system 
 
Running System Checks - Mode: start 
Checking db online                            : OK
Checking gmt directory dirty                  : WARN (The GMT directory contains untracked or changed files - These changes will not be stored and it will be hard to understand possible changes when comparing the measurements later. We recommend only running on a clean dir.)
Checking single energy scope machine provider : OK
Checking tmpfs mount                          : OK
Checking ntp                                  : WARN (You have NTP time syncing active. This can create noise in runs and should be deactivated.)
Checking < 5% CPU utilization                 : WARN (Your system seems to be busy. Utilization is above 5%. Consider terminating some processes for a more stable measurement.)
Checking high sampling rate                   : OK
Checking 1 GiB free hdd space                 : OK
Checking 1 GiB free memory                    : OK
Checking docker daemon                        : OK
Checking running containers                   : WARN (You have other containers running on the system. This is usually what you want in local development, but for undisturbed measurements consider going for a measurement cluster [See https://docs.green-coding.io/docs/installation/installation-cluster/].)
Checking utf file encoding                    : OK
Checking swap disabled                        : WARN (Your system uses a swap filesystem. This can lead to very instable measurements. Please disable swap.)
 
Checking out repository 
 
Having Usage Scenario  Wordpress Data Puppeteer Scenario 
From:  Arne Tarara <arne@green-coding.berlin>
Description:  Wordpress sample with pre-populated DB (pages and plugins) where a simple Puppeteer flow is executed 

 

>>>> Warning: Runner is running in unsafe mode <<<<

 
 
Importing metric providers 
Importing CpuUtilizationProcfsSystemProvider from metric_providers.cpu.utilization.procfs.system.provider
Configuration is {'sampling_rate': 99}
Importing CpuUtilizationCgroupContainerProvider from metric_providers.cpu.utilization.cgroup.container.provider
Configuration is {'sampling_rate': 99}
Importing MemoryUsedCgroupContainerProvider from metric_providers.memory.used.cgroup.container.provider
Configuration is {'sampling_rate': 99}
Importing NetworkIoCgroupContainerProvider from metric_providers.network.io.cgroup.container.provider
Configuration is {'sampling_rate': 99}
Importing DiskIoCgroupContainerProvider from metric_providers.disk.io.cgroup.container.provider
Configuration is {'sampling_rate': 99}
 
Removing all temporary GMT images 
 

>>>> Warning: GMT is not instructed to prune docker images and build caches. 
We recommend to set --docker-prune to remove build caches and anonymous volumes, because otherwise your disk will get full very quickly. If you want to measure also network I/O delay for pulling images and have a dedicated measurement machine please set --full-docker-prune <<<<

 
 
Downloading dependencies 
latest: Pulling from kaniko-project/executor
Digest: sha256:4e7a52dd1f14872430652bb3b027405b8dfd17c4538751c620ac005741ef9698
Status: Image is up to date for gcr.io/kaniko-project/executor:latest
gcr.io/kaniko-project/executor:latest
 
Initializing run 
 
Starting metric providers 
stdbuf -o0 taskset -c 0 /home/mleduc/green-metrics-tool/metric_providers/cpu/utilization/procfs/system/metric-provider-binary -i 99 > /tmp/green-metrics-tool/cpu_utilization_procfs_system.log
 
Waiting for Metric Providers to boot ... 
Stderr check on CpuUtilizationProcfsSystemProvider
 
Starting phase [BASELINE]. 

Force-sleeping for 1s
 
Starting phase [INSTALLATION]. 

Force-sleeping for 1s
 
Building Docker images 
Building gcb_wordpress_mariadb
docker run --rm -v /workspace -v /tmp/wordpress-official-data:/tmp/repo:ro -v /tmp/green-metrics-tool/docker_images:/output gcr.io/kaniko-project/executor:latest --dockerfile=/tmp/repo//./Dockerfile-mariadb --context dir:///tmp/repo//. --destination=gcb_wordpress_mariadb_gmt_run_tmp --tar-path=/output/gcb_wordpress_mariadb_gmt_run_tmp.tar --registry-mirror index.docker.io --cleanup=true --no-push
docker load -q -i /tmp/green-metrics-tool/docker_images/gcb_wordpress_mariadb_gmt_run_tmp.tar
Building gcb_wordpress_apache
docker run --rm -v /workspace -v /tmp/wordpress-official-data:/tmp/repo:ro -v /tmp/green-metrics-tool/docker_images:/output gcr.io/kaniko-project/executor:latest --dockerfile=/tmp/repo//./Dockerfile-wordpress --context dir:///tmp/repo//. --destination=gcb_wordpress_apache_gmt_run_tmp --tar-path=/output/gcb_wordpress_apache_gmt_run_tmp.tar --registry-mirror index.docker.io --cleanup=true --no-push
docker load -q -i /tmp/green-metrics-tool/docker_images/gcb_wordpress_apache_gmt_run_tmp.tar
Pulling greencoding/puppeteer-chrome
['docker', 'pull', 'index.docker.io/greencoding/puppeteer-chrome']
 
Saving image and volume sizes 
 
Starting phase [BOOT]. 

Force-sleeping for 1s
 
Setting up networks 
Creating network:  gcb-wordpress-mariadb-network
 
Setting up services 
Startup order:  ['gcb-wordpress-mariadb', 'gcb-wordpress-apache', 'gcb-puppeteer']
 
Setting up container for service: gcb-wordpress-mariadb 
Container name: gcb-wordpress-mariadb
Resetting container
Creating container
Setting ports:  ['3306:3306']
Applying Memory Limit from deploy
Applying CPU Limit from deploy
Checking image architecture compatibility...
Architecture compatible: amd64 (native)
Running docker run with: docker run -it -d --name gcb-wordpress-mariadb -v /tmp/wordpress-official-data:/tmp/repo:ro -p 3306:3306 -e MYSQL_ROOT_PASSWORD=somewordpress -e MYSQL_DATABASE=wordpress -e MYSQL_USER=wordpress -e MYSQL_PASSWORD=wordpress --net gcb-wordpress-mariadb-network --memory 500M --cpus 1 gcb_wordpress_mariadb_gmt_run_tmp
Stdout: bb9ba427d8d0ccaa59d21fb72860b7d9bbcc493e20ad7ad925651f88337d4077
Running commands
Running command:  docker exec gcb-wordpress-mariadb sleep 20
 
Setting up container for service: gcb-wordpress-apache 
Container name: gcb-wordpress-apache
Resetting container
Creating container
Setting ports:  ['9875:9875']
Applying Memory Limit from deploy
Applying CPU Limit from deploy
Checking image architecture compatibility...
Architecture compatible: amd64 (native)
Container state of dependent service 'gcb-wordpress-mariadb': running
Running docker run with: docker run -it -d --name gcb-wordpress-apache -v /tmp/wordpress-official-data:/tmp/repo:ro -p 9875:9875 -e WORDPRESS_DB_HOST=gcb-wordpress-mariadb -e WORDPRESS_DB_USER=wordpress -e WORDPRESS_DB_PASSWORD=wordpress -e WORDPRESS_DB_NAME=wordpress --net gcb-wordpress-mariadb-network --memory 500M --cpus 1 gcb_wordpress_apache_gmt_run_tmp
Stdout: 2f03669015c52c0d1a3277eec2ef88404901c5a4f522f5aac9b3475e8e3899df
Running commands
 
Setting up container for service: gcb-puppeteer 
Container name: gcb-puppeteer
Resetting container
Creating container
Applying Memory Limit from deploy
Applying CPU Limit from deploy
Checking image architecture compatibility...
Architecture compatible: amd64 (native)
Running docker run with: docker run -it -d --name gcb-puppeteer -v /tmp/wordpress-official-data:/tmp/repo:ro --net gcb-wordpress-mariadb-network --memory 500M --cpus 1 greencodingpuppeteerchrome_gmt_run_tmp
Stdout: 1e874b61943a204fb7b44a6f9f8ea2efa605eacd0d5b926b8a6184d9a9d9d043
Running commands
Running command:  docker exec gcb-puppeteer cp /tmp/repo/wordpress-official-data/puppeteer-flow.js /var/www/puppeteer-flow.js
 
Marking run as failed 
 
Updating start and end measurement times 
 
Capturing container logs 
 Stopping metric providers and parsing measurements 
Trying to kill metric-provider-binary with PGID: 25697
Imported  14400  metrics from  CpuUtilizationProcfsSystemProvider
 
Reading process stdout/stderr (if selected) and cleaning them up 
 
Updating phases in DB 
 
Saving notes:   [{'note': 'Start of measurement', 'detail_name': '[NOTES]', 'timestamp': 1758392680292063}, {'note': 'Starting phase [BASELINE]', 'detail_name': '[NOTES]', 'timestamp': 1758392687058691}, {'note': 'Ending phase [BASELINE] [UNPADDED]', 'detail_name': '[NOTES]', 'timestamp': 1758392687058744}, {'note': 'Ending phase [BASELINE] [PADDED]', 'detail_name': '[NOTES]', 'timestamp': 1758392687157744}, {'note': 'Starting phase [INSTALLATION]', 'detail_name': '[NOTES]', 'timestamp': 1758392687158782}, {'note': 'Building gcb_wordpress_mariadb', 'detail_name': '[NOTES]', 'timestamp': 1758392687193920}, {'note': 'Building gcb_wordpress_apache', 'detail_name': '[NOTES]', 'timestamp': 1758393179133378}, {'note': "Pulling {service['image']}", 'detail_name': '[NOTES]', 'timestamp': 1758394093626661}, {'note': 'Ending phase [INSTALLATION] [UNPADDED]', 'detail_name': '[NOTES]', 'timestamp': 1758394094851557}, {'note': 'Ending phase [INSTALLATION] [PADDED]', 'detail_name': '[NOTES]', 'timestamp': 1758394094950557}, {'note': 'Starting phase [BOOT]', 'detail_name': '[NOTES]', 'timestamp': 1758394095148801}, {'note': 'End of measurement', 'detail_name': '[NOTES]', 'timestamp': 1758394120976150}]
 
Saving logs to DB 
 
Calculating and storing phases data. This can take a couple of seconds ... 
 
Starting cleanup routine 
Stopping metric providers
Stopping containers
bb9ba427d8d0ccaa59d21fb72860b7d9bbcc493e20ad7ad925651f88337d4077
2f03669015c52c0d1a3277eec2ef88404901c5a4f522f5aac9b3475e8e3899df
1e874b61943a204fb7b44a6f9f8ea2efa605eacd0d5b926b8a6184d9a9d9d043
Removing network
gcb-wordpress-mariadb-network
 
Removing all temporary GMT images 
Untagged: gcb_wordpress_apache_gmt_run_tmp:latest
Deleted: sha256:5ee334375501745ec7807ee724d98d89ce0ec51c104e532f914801deaee016fc
Deleted: sha256:2915292bc1eb5820b46fc5dea8f35407be2d9802876dee5f8c3bf3880a4828e6
Deleted: sha256:7610895a4d3fd4cfc7b4d856f139458a99bd5db4e3ac0e2400c1391d9e2bebe1
Deleted: sha256:ddf819173ad8693dc59b22acd12f0199a258238e49a62e86a590db731cb4165f
Deleted: sha256:6429a12c6ddb406d998711c88da1e9216e05c15cc06fd5b8af802ba324b7d382
Deleted: sha256:001c2e6e776453e4bfb9538cde62b1f3e022ef01788af5188d193ca9091439cf
Deleted: sha256:ee5cac6ba279f7e3a252c3f5115bd4882e9a46c05c35f2efab9eaedacd73364e
Deleted: sha256:b832f30a961d093924aa6f42eaeb9ad2549434d1069e863d251ec525aba2a5ed
Deleted: sha256:ed7ca789ccbefe0728e915f7a8c650b1fce4a63a66853f051d616a47e728b1b5
Deleted: sha256:50bfc56dc3cf11857df5c25706b800e33f251afd2289d7bc2d0efc50245e2e70
Deleted: sha256:16ab6c97639d61a7bdb32f394210b63f0149f86acbbbe93fa81f2035a0496b27
Deleted: sha256:5e8e0667341ef04f352681cbefef5c7265c12eff0626ee33417dd5072575796a
Deleted: sha256:593bedb123e9b3e25507a0265df321dbfcd2043d0a1a746105b36c8040c24a55
Deleted: sha256:1b970319fcaf413cda14110f0dad7f10694bcf365558fc7da1999a359113f3e2
Deleted: sha256:3d420c6c70a635b60a7685ace3ffeddccb578b4ddf48d5599bcd615e526615ad
Deleted: sha256:106ef46f8a0c6237cb34e23dcbbf3330a5c590aceec117f35474b7a0752f6859
Deleted: sha256:33f2a61a6811589e52b94d91f1362698be0f105f4b9bdc0f5645ec7478846a3e
Deleted: sha256:d700ba334af64b7d9e5e4f398695e54591172038596d68df1ad49d9ef3193362
Deleted: sha256:bd1ca1112454c7564fd93b2f61537ba7a4eba1136fc78d7fed533e7ad4f34f15
Deleted: sha256:87ee6327dc3aa71cae0fff76278a8d0c3eb828e2d624d8b18de092ff841b3c7f
Deleted: sha256:3d24f832b1a3f695aeab79c51d1941850b01adc26cc0e2e60c92009279c6b6ab
Deleted: sha256:879fa38d47bcc11920de17dc6973bfda2635e1b1d44e506c9502a36ca2f056e4
Deleted: sha256:988bebd5d3f4d8a7fc4dfcccabbc284cdc697b60e0a9a0d1b6aed48b775bdeca
Untagged: gcb_wordpress_mariadb_gmt_run_tmp:latest
Deleted: sha256:c8239aa4332dcee24944d3d234a007249ee97d2c35ace8fb697400fcccf95d64
Deleted: sha256:78d8088120f6dd47f2b3ba2620bfdc1222aea38b77f718117b026047ae7b3d49
Deleted: sha256:ffb9fb143b85a783ff0e83cf0c594c8fd6838b889dee7d14c0a2a3e0c9df419d
Deleted: sha256:067860988e6658468a17d2d0d57fb3e74ef45bf3b38167a796f5bfa61d14ed4d
Deleted: sha256:52da81964394b8f090e4e3405eb3478cbb274d84e023680b5eb21be463d4fb72
Deleted: sha256:4bd99e1fe00faaa2737267cb775982ad444ad62999d37df6a9fa92f458d9f10c
Deleted: sha256:9f4f0dd33c9816176e9bd3077cee199d0f14bd05ef5b7e2f01a80a1ea3d48316
Deleted: sha256:46c59a61ed6209c1e6a55e8811345216f5dfe704d4411ba3e37e1cb345d7deeb
Deleted: sha256:4bf277548ca07781855a8ce8c8b299a89fd5742151003f83053026e0b01f8470
Deleted: sha256:715a1b962166ede06c7a0e87d068a4b686e6066e0eca5ecab6f4d6cfab2121fe
Deleted: sha256:97ab3baee34d0c75ee10e65c63a06cbc87d20d695c17d14ad565d4ff1b8dc2ca
Deleted: sha256:9f54eef412758095c8079ac465d494a2872e02e90bf1fb5f12a1641c0d1bb78b
Untagged: greencodingpuppeteerchrome_gmt_run_tmp:latest
 

>>>> Warning: GMT is not instructed to prune docker images and build caches. 
We recommend to set --docker-prune to remove build caches and anonymous volumes, because otherwise your disk will get full very quickly. If you want to measure also network I/O delay for pulling images and have a dedicated measurement machine please set --full-docker-prune <<<<

 
 -Cleanup gracefully completed 
 

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 0_o >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Traceback (most recent call last):
  File "/home/mleduc/green-metrics-tool/runner.py", line 183, in <module>
    run_id = runner.run()  # Start main code
             ^^^^^^^^^^^^
  File "/home/mleduc/green-metrics-tool/lib/scenario_runner.py", line 2287, in run
    raise exc
  File "/home/mleduc/green-metrics-tool/lib/scenario_runner.py", line 2231, in run
    self._setup_services()
  File "/home/mleduc/green-metrics-tool/lib/scenario_runner.py", line 1432, in _setup_services
    raise RuntimeError(f"Process {d_command} failed.\n\nStdout: {ps.stdout}\nStderr: {ps.stderr}")
RuntimeError: Process ['docker', 'exec', 'gcb-puppeteer', 'cp', '/tmp/repo/wordpress-official-data/puppeteer-flow.js', '/var/www/puppeteer-flow.js'] failed.

Stdout: 
Stderr: cp: can't stat '/tmp/repo/wordpress-official-data/puppeteer-flow.js': No such file or directory



<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 0_o >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Error: RuntimeError occured in runner.py

Exception_context (NoneType): None
Final_exception (RuntimeError): Process ['docker', 'exec', 'gcb-puppeteer', 'cp', '/tmp/repo/wordpress-official-data/puppeteer-flow.js', '/var/www/puppeteer-flow.js'] failed.

Stdout: 
Stderr: cp: can't stat '/tmp/repo/wordpress-official-data/puppeteer-flow.js': No such file or directory

Run_id (UUID): 1931dd7d-f22a-4079-b3f5-faaaf7facd02
Traceback_first (bool): True
Run-ID Link: http://metrics.green-coding.internal:9142/stats.html?id=1931dd7d-f22a-4079-b3f5-faaaf7facd02

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 0_o >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

The issue seems to be with the absence of a /tmp/repo/wordpress-official-data/puppeteer-flow.js file.
I do have a /tmp/wordpress-official-data/puppeteer-flow.js file, but I fail to understand if I need to change the hardcoded /tmp/repo prefix, or to move the files somewhere else locally, define a file mount somewhere with docker, or maybe something else.
Thanks!
PS: I started studying the examples because I failed to define my own configuration with puppeteer, following https://docs.green-coding.io/explanations.

[ribalba]

Hey, thank you for trying out the GMT. We mount the repo you supply with --uri into /tmp/repo in the container. So when you supply --uri /tmp/wordpress-official-data this will the /tmp/repo/ so you don't need to add the wordpress-official-data part. You can just do /tmp/repo/puppeteer-flow.js if this is in your dir you supplied with --uri.

Can you maybe go into more details why you failed to configure your own puppeteer flow?

Also the GMT now offers Playwright commands natively. It hasn't been added to the documentation but might help:
green-coding-solutions/green-metrics-tool#1334

Feel free to ask if you encounter any more problems.

[manuelleduc]

Thanks @ribalba, with your explanations I managed to run the wordpress example!

I had a look again at my own experiment and my issues are related to the use of the sleep setup commands (I believe). I have attached the two files of my own benchmark below.

When running with the sleep commented, I get:

[...] 
 -Cleanup gracefully completed 
 

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 0_o >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Traceback (most recent call last):
  File "/home/mleduc/green-metrics-tool/lib/scenario_runner.py", line 2287, in run
    raise exc
  File "/home/mleduc/green-metrics-tool/lib/scenario_runner.py", line 2241, in run
    self._start_metric_providers(allow_container=True, allow_other=False)
  File "/home/mleduc/green-metrics-tool/lib/scenario_runner.py", line 1557, in _start_metric_providers
    raise RuntimeError(f"Stderr on {metric_provider.__class__.__name__} was NOT empty: {stderr_read}")
RuntimeError: Stderr on CpuUtilizationCgroupContainerProvider was NOT empty: Error - Could not open container for reading: c5864be7b0628005b27cbb86c48bbd41ad5fc1daac6f98bed2eb6bad5a3eadf6. Maybe the container is not running anymore? Errno: 2

And with the sleep uncommented:

-Cleanup gracefully completed 
 

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 0_o >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Traceback (most recent call last):
  File "/home/mleduc/green-metrics-tool/runner.py", line 183, in <module>
    run_id = runner.run()  # Start main code
             ^^^^^^^^^^^^
  File "/home/mleduc/green-metrics-tool/lib/scenario_runner.py", line 2287, in run
    raise exc
  File "/home/mleduc/green-metrics-tool/lib/scenario_runner.py", line 2231, in run
    self._setup_services()
  File "/home/mleduc/green-metrics-tool/lib/scenario_runner.py", line 1432, in _setup_services
    raise RuntimeError(f"Process {d_command} failed.\n\nStdout: {ps.stdout}\nStderr: {ps.stderr}")
RuntimeError: Process ['docker', 'exec', 'xwiki-postgres-db', 'sleep', '50'] failed.

Stdout: 
Stderr: 


<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 0_o >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Error: RuntimeError occured in runner.py

Exception_context (NoneType): None
Final_exception (RuntimeError): Process ['docker', 'exec', 'xwiki-postgres-db', 'sleep', '50'] failed.

Stdout: 
Stderr: 
Run_id (UUID): 5705e008-70f1-4510-828e-ea2d4349dc79
Traceback_first (bool): True
Run-ID Link: http://metrics.green-coding.internal:9142/stats.html?id=5705e008-70f1-4510-828e-ea2d4349dc79

puppeteer-flow.js
usage_scenario.yml

[ribalba]

Hey, so I spotted a few things that were in the way of running the scenario. Here is a fixed version which works on my Linux Laptop. Please note that this will not work on Mac as it is not x86 and has no X.

I copied most of the code from

https://github.com/green-coding-solutions/example-applications/tree/main/puppeteer-firefox-chrome

• One thing that also catches me out sometimes is that if you set a container name this will be the network name. So the key in they yml will not resolve.
• You also need to setup puppeteer in your js file

Nearly there :) Feel free to reach out if you have any more problems.

Archive.zip

[manuelleduc]

Sorry for the late reply.
Thank you so much for the working example; that's much appreciated!

I think I can now close this issue, and I'll need to invest more time to define scenarios before being able to have actual numbers.

PS: @ribalba I just listened to your interview with Chris Adams on Environment Variables, very interesting episode, I must say.

[ribalba]

Happy we could help. And thanks for listing to the podcast :)