diff --git a/backlog-set-due-date b/backlog-set-due-date new file mode 100755 index 00000000..f2cc7e12 --- /dev/null +++ b/backlog-set-due-date @@ -0,0 +1,27 @@ +#!/bin/sh -e + +redmine_api_key="${redmine_api_key:?"Need redmine API key"}" +project_id="${project_id:-18}" +ticket_limit="${ticket_limit:-200}" +host="${host:-"https://progress.opensuse.org"}" +issues=$(mktemp) +issue=$(mktemp) + +curl -s -H "X-Redmine-API-Key: $redmine_api_key" "$host/issues.json?project_id=$project_id&limit=$ticket_limit" | jq -r '.issues | .[] | select(.priority.name!="Low") | select(.due_date==null)' >"$issues" + +for id in $(jq .id "$issues"); do + jq "select(.id==$id)" "$issues" >"$issue" + case $(jq ".priority.name" "$issue"| tr -d '"') in + Normal) days=356;; + High) days=30;; + Urgent) days=7;; + Inmediate) days=1;; + *) days=0;; + esac + + start_date=$(jq ".created_on" "$issue" | tr -d '"') + due_date=$(date -d"$start_date + $days days" +%Y-%m-%dT%TZ) + echo "$due_date" + #TODO Setup the due date using the Progress API +done + diff --git a/issues.json b/issues.json new file mode 100644 index 00000000..f50b5fcf --- /dev/null +++ b/issues.json @@ -0,0 +1,1199 @@ +{ + "id": 78390, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 12, + "name": "Workable" + }, + "priority": { + "id": 5, + "name": "High" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 134, + "name": "Concrete Bugs" + }, + "fixed_version": { + "id": 418, + "name": "Ready" + }, + "subject": "worker is \"broken\" and not taking jobs if one of multiple configured webUI hosts is down", + "description": "## Observation\r\n\r\nON 2020-11-20 all worker instances on grenache-1 show up as \"broken\" and checked on grenache-1 with `systemctl status openqa-worker@39` which says:\r\n\r\n```\r\nNov 20 06:40:46 grenache-1 worker[6888]: [info] [pid:6888] Registering with openQA baremetal-support.qa.suse.de\r\nNov 20 06:40:46 grenache-1 worker[6888]: [warn] [pid:6888] Failed to register at baremetal-support.qa.suse.de - connection error: Can't connect: Name or service not known - trying again in 10 seconds\r\nNov 20 06:40:56 grenache-1 worker[6888]: [info] [pid:6888] Registering with openQA baremetal-support.qa.suse.de\r\nNov 20 06:40:56 grenache-1 worker[6888]: [warn] [pid:6888] Failed to register at baremetal-support.qa.suse.de - connection error: Can't connect: Name or service not known - trying again in 10 seconds\r\n```\r\n\r\nso it seems like baremetal-support.qa.suse.de is down while openqa.suse.de should have been taken but the worker is stuck in a retry loop maybe for way too long or forever\r\n\r\n## Steps to reproduce\r\n\r\nLikely can be reproduced by configuring a worker to connect to two web UI hosts where one can not be reached at all (e.g. valid DNS entry but host not up)\r\n\r\n## Acceptance criteria\r\n* **AC1:** The configured and reachable webUI hosts are reached while the down host is ignored as long as it is down\r\n* **AC2:** webUI does not show up as \"broken\" on a reachable webUI\r\n* **AC3:** worker still retries for multiple minutes when a webUI is temporarily down, e.g. during reboot\r\n\r\n## Suggestions\r\nMaybe when we extended the waiting periods while a webUI is down we introduced this regression\r\n* check history of our changes\r\n* crosscheck and extend tests\r\n* fix behaviour\r\n* confirm working in OSD infrastructure\r\n* remove workarounds, e.g. ensure baremetal-support.qa.suse.de is added back to salt-pillars-openqa for grenache-1\r\n\r\n## Workaround\r\nRemove webUI hosts that are down from /etc/openqa/workers.ini and restart worker services", + "start_date": null, + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-11-20T05:56:02Z", + "updated_on": "2020-11-20T05:56:02Z", + "closed_on": null +} +{ + "id": 78169, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 15, + "name": "Blocked" + }, + "priority": { + "id": 5, + "name": "High" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "assigned_to": { + "id": 22072, + "name": "mkittler" + }, + "category": { + "id": 134, + "name": "Concrete Bugs" + }, + "fixed_version": { + "id": 418, + "name": "Ready" + }, + "subject": "after osd-deploy 2020-11-18 incompletes with auto_review:\"Cache service (status error from API|.*error 500: Internal Server Error)\":retry", + "description": "## Observation\r\n\r\nFor example https://openqa.suse.de/tests/5020506 showing\r\n\"Reason: setup failure: No workers active in the cache service\"\r\n\r\nwith autoinst-log.txt:\r\n\r\n```\r\n[2020-11-18T06:40:55.0106 CET] [info] [pid:12060] +++ setup notes +++\r\n[2020-11-18T06:40:55.0106 CET] [info] [pid:12060] Running on openqaworker9:17 (Linux 4.12.14-lp151.28.79-default #1 SMP Wed Nov 11 08:17:16 UTC 2020 (472d149) x86_64)\r\n[2020-11-18T06:40:55.0147 CET] [info] [pid:12060] +++ worker notes +++\r\n[2020-11-18T06:40:55.0147 CET] [info] [pid:12060] End time: 2020-11-18 05:40:55\r\n[2020-11-18T06:40:55.0147 CET] [info] [pid:12060] Result: setup failure\r\n[2020-11-18T06:40:55.0152 CET] [info] [pid:13158] Uploading autoinst-log.txt\r\n```\r\n\r\nand worker-log.txt:\r\n\r\n```\r\n[2020-11-18T06:40:55.0106 CET] [debug] [pid:12060] Preparing Mojo::IOLoop::ReadWriteProcess::Session\r\n…\r\n[2020-11-18T06:40:55.0111 CET] [error] [pid:12060] Unable to setup job 5020506: No workers active in the cache service\r\n[2020-11-18T06:40:55.0111 CET] [debug] [pid:12060] Stopping job 5020506 from openqa.suse.de: 05020506-sle-12-SP2-Server-DVD-Incidents-Kernel-KOTD-x86_64-Build4.4.121-261.1.g13f6b6d-ltp_syscalls_pre12sp4@64bit - reason: setup failure\r\n[2020-11-18T06:40:55.0112 CET] [debug] [pid:12060] REST-API call: POST http://openqa.suse.de/api/v1/jobs/5020506/status\r\n[2020-11-18T06:40:55.0152 CET] [info] [pid:13158] Uploading autoinst-log.txt\r\n[2020-11-18T06:40:55.0207 CET] [info] [pid:13158] Uploading worker-log.txt\r\n```\r\n\r\n## Steps to reproduce\r\n\r\nFind jobs referencing this ticket with the help of\r\nhttps://raw.githubusercontent.com/os-autoinst/scripts/master/openqa-query-for-job-label ,\r\nfor example to look this ticket #78169 call `openqa-query-for-job-label poo#78169`\r\n\r\n\r\n## Suggestions\r\n\r\n* Crosscheck what the osd deployment 2020-11-18 could have brought in as changes explaining the problems\r\n* Lookup in source code what this message could mean", + "start_date": "2020-11-18", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-11-18T08:40:55Z", + "updated_on": "2020-11-19T20:38:54Z", + "closed_on": null +} +{ + "id": 77905, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 4, + "name": "Feedback" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "assigned_to": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 418, + "name": "Ready" + }, + "parent": { + "id": 77698 + }, + "subject": "CI pipeline proof-of-concept running isotovideo", + "description": "## Acceptance criteria\r\n* **AC1:** A proof-of-concept CI configuration and run showing how isotovideo executes tests exists\r\n\r\n## Suggestions\r\n* *DONE*: Install and run isotovideo for os-autoinst-distri-example -> see https://github.com/os-autoinst/os-autoinst-distri-example/pull/4\r\n* *WIP*: Optional: Containerize isotovideo, e.g. based on https://build.opensuse.org/package/show/home:okurz:container/isotovideo , to make startup easier and faster in repeated CI runs\r\n * created https://build.opensuse.org/package/show/home:okurz:container/isotovideo-qemu-x86 and https://build.opensuse.org/package/show/home:okurz:container/isotovideo-qemu-kvm\r\n * @waitfor https://github.com/os-autoinst/os-autoinst/pull/1567\r\n * then create package in build.opensuse.org/project/show/devel:openQA with _service that extracts the Dockerfile from os-autoinst and builds and publishes images\r\n * use that image in https://github.com/os-autoinst/os-autoinst-distri-example/pull/4\r\n* Add to documentation, e.g. https://github.com/os-autoinst/os-autoinst/blob/master/README.asciidoc in https://github.com/os-autoinst/os-autoinst/pull/1563\r\n* Optional: Try in a CI that supports nested virt, e.g. travis CI in parallel to github actions. Then likely https://build.opensuse.org/package/show/home:okurz:container/isotovideo-qemu-kvm is a better selection", + "start_date": "2020-11-15", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-11-15T07:59:14Z", + "updated_on": "2020-11-18T09:49:34Z", + "closed_on": null +} +{ + "id": 77698, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 6, + "name": "coordination" + }, + "status": { + "id": 15, + "name": "Blocked" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "assigned_to": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 418, + "name": "Ready" + }, + "subject": "[epic] synchronous qemu based system level test in pull request CI runs, e.g. standalone isotovideo or openQA tests", + "description": "## User Story\r\nAs a software stack developer I want to run synchronous qemu based tests in pull request CI runs to have system level test coverage before accepting PRs\r\n\r\n## Acceptance criteria\r\n* **AC1:** Easy instructions how to run qemu based system level tests in any github project exist\r\n* **AC2:** Scalable for multiple projects\r\n* **AC3:** One example github PR or gitlab MR shows a successful test in action for any product or repository\r\n\r\n## Suggestions\r\n* Experiment with local runs of isotovideo, maybe based on github.com/os-autoinst/os-autoinst-distri-example\r\n * isotovideo supports setting all necessary test parameters on the command line but a file `vars.json` can be used as well\r\n* Look into github actions and publish on github marketplace for easy inclusion\r\n * you can start with https://docs.github.com/en/free-pro-team@latest/actions/creating-actions/creating-a-docker-container-action and https://github.com/actions/hello-world-docker-action\r\n * read how to publish on marketplace: https://docs.github.com/en/free-pro-team@latest/actions/creating-actions/publishing-actions-in-github-marketplace#publishing-an-action\r\n* If the above is not feasible or as alternative create documentation describing how to use the proof of concept\r\n* Try to have a generic solution that is not dependant on one specific CI provider but consider \"polishing\" it for at least one to be super-easy :)\r\n* Also upload test artifacts, e.g. serial terminal log and video.ogv , e.g. see https://docs.github.com/en/free-pro-team@latest/actions/guides/storing-workflow-data-as-artifacts#uploading-build-and-test-artifacts . was suggested by pdostal in https://github.com/os-autoinst/os-autoinst-distri-example/pull/4#issuecomment-727800015", + "start_date": "2020-11-15", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-11-11T09:51:22Z", + "updated_on": "2020-11-16T18:35:50Z", + "closed_on": null +} +{ + "id": 76984, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 12, + "name": "Workable" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 418, + "name": "Ready" + }, + "subject": "Automatically remove assets+results based on available free space", + "description": "## Motivation\r\n\r\nSee examples like #76822 : openQA has automatic removal of assets+results but the sum of all configured retention periods and asset quotas can still exceed the available space so that manual administration is required. In case the cleanup based on these parameters can not free enough space we should do the next step and remove more until we have enough free space again. We already do something similar in https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/etc/master/cron.d/SLES.CRON#L18 to remove videos of older test jobs which we identified as a big contributor to space usage.\r\n\r\n\r\n## Acceptance criteria\r\n* **AC1:** the filesystem including the openQA results directory is ensured to have at least a configured amount of free space\r\n\r\n## Suggestions\r\n* Read and understand https://gitlab.suse.de/openqa/salt-states-openqa/-/blob/master/etc/master/cron.d/SLES.CRON#L18\r\n* Extend the existing asset+result cleanup to\r\n * check the free space of the filesystem including the results directory\r\n * compare the free space against a configured value, e.g. in openqa.ini\r\n * if free space is below limit after results cleanup remove more data from results checking in each step until free space limit is reached, e.g.\r\n * videos from oldest, non-important jobs first (\"oldest first\" can mean simply job id numbers ascending order)\r\n * other results from oldest, non-important jobs\r\n * videos from oldest, important jobs\r\n * other results from oldest, important jobs\r\n * if after all steps free space limit could still not be reached, i.e. if all result data was removed, raise error\r\n * the above can be configured as well, e.g. \"results_free_space_cleanup_components=non-important-results-videos,non-important-results-other,important-results-videos,important-results-other\"\r\n* can use https://software.opensuse.org/package/perl-Filesys-Df?search_term=perl-FileSys-Df\r\n* can mock \"df\" in tests to simply give back what we want, e.g. \"enough free space available\" or \"free space exceeded\"\r\n* Optional: Extend to assets as well\r\n\r\n\r\n## Impact\r\nThis can also greatly help us as administrators of osd to ensure that /results limits are not exceeded which repeatedly caused us additional administration work.\r\n\r\n## Workaround\r\nHave a periodic job calling \"df\" and checking against limit, remove results otherwise", + "start_date": null, + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-11-04T16:10:15Z", + "updated_on": "2020-11-04T20:27:56Z", + "closed_on": null +} +{ + "id": 76978, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 1, + "name": "New" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 490, + "name": "future" + }, + "subject": "How to run an openQA test in 5 minutes", + "description": "## User story\r\n\r\nAs a potential user of openQA I want to run an openQA test in 5 minutes to be hooked\r\n\r\n\r\n## Acceptance criteria\r\n* **AC1:** An openQA test can be installed and is used to execute a new (empty) test within 5 minutes (not counting download times on slow connections)\r\n\r\n## Suggestions\r\n\r\n* We already have a quick bootstrap method but it might not be prominent enough and we can present it as a \"first-class citizen\"\r\n* Have a workflow (or test or something) that uses the quick bootstrap or comparable approach and runs tests based on github.com/os-autoinst/os-autoinst-distri-example\r\n* record a video and make some noiz about it :)\r\n\r\n\r\n## Further details\r\nInspired by talking to bzoltan :)\r\n\r\n", + "start_date": "2020-11-04", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-11-04T14:11:46Z", + "updated_on": "2020-11-04T14:12:26Z", + "closed_on": null +} +{ + "id": 75265, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 12, + "name": "Workable" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "assigned_to": { + "id": 23018, + "name": "kraih" + }, + "category": { + "id": 134, + "name": "Concrete Bugs" + }, + "fixed_version": { + "id": 418, + "name": "Ready" + }, + "subject": "sporadic errors in test suite of perl-Mojo-IOLoop-ReadWriteProcess", + "description": "## Observation\r\n\r\nhttps://build.opensuse.org/package/live_build_log/devel:openQA:Leap:15.1/perl-Mojo-IOLoop-ReadWriteProcess/openSUSE_Leap_15.1/x86_64 shows\r\n\r\n```\r\n[ 97s] TEST error print\r\n[ 97s] TEST error print\r\n[ 97s] t/01_run.t ............... ok\r\n[ 108s] t/02_parallel.t .......... ok\r\n[ 109s] Can't use an undefined value as filehandle reference at lib/Mojo/IOLoop/ReadWriteProcess.pm line 298.\r\n[ 126s] t/03_func.t .............. ok\r\n[ 127s] t/04_queues.t ............ ok\r\n[ 127s] t/05_serialize.t ......... ok\r\n[ 131s] t/06_events.t ............ ok\r\n[ 191s] \r\n[ 191s] # Failed test 'collect_status fired 8 times'\r\n[ 191s] # at t/07_autodetect.t line 411.\r\n[ 191s] # got: undef\r\n[ 191s] # expected: '1'\r\n[ 191s] \r\n[ 191s] # Failed test 'new_subprocess fired 7 times'\r\n[ 191s] # at t/07_autodetect.t line 412.\r\n[ 191s] # got: '9'\r\n[ 191s] # expected: '8'\r\n[ 191s] \r\n[ 191s] # Failed test 'detection works'\r\n[ 191s] # at t/07_autodetect.t line 414.\r\n[ 191s] # got: '9'\r\n[ 191s] # expected: '8'\r\n[ 191s] # bless( {\r\n[ 191s] # '_status' => -1,\r\n[ 191s] # 'args' => [],\r\n[ 191s] # 'error' => bless( [], 'Mojo::Collection' ),\r\n[ 191s] # 'error_stream' => bless( \\*Symbol::GEN122, 'IO::Handle' ),\r\n[ 191s] # 'events' => {},\r\n[ 191s] # 'execute' => '/home/abuild/rpmbuild/BUILD/Mojo-IOLoop-ReadWriteProcess-0.28/t/data/subreaper/roulette.sh',\r\n[ 191s] # 'process_id' => 1744,\r\n[ 191s] # 'read_stream' => bless( \\*Symbol::GEN120, 'IO::Handle' ),\r\n[ 191s] # 'separate_err' => 1,\r\n[ 191s] # 'session' => bless( {\r\n[ 191s] # 'collect_status' => 1,\r\n[ 191s] # 'events' => {\r\n[ 191s] # 'collected' => [\r\n[ 191s] # sub { \"DUMMY\" }\r\n[ 191s] # ],\r\n[ 191s] # 'collected_orphan' => [\r\n[ 191s] # sub { \"DUMMY\" }\r\n[ 191s] # ]\r\n[ 191s] # },\r\n[ 191s] # 'handler' => undef,\r\n[ 191s] # 'orphans' => {\r\n[ 191s] # '1744' => bless( {\r\n[ 191s] # '_status' => 256,\r\n[ 191s] # 'process_id' => 1744\r\n[ 191s] # }, 'Mojo::IOLoop::ReadWriteProcess' ),\r\n[ 191s] # '1748' => bless( {\r\n[ 191s] # '_status' => 256,\r\n[ 191s] # 'process_id' => 1748\r\n[ 191s] # }, 'Mojo::IOLoop::ReadWriteProcess' ),\r\n[ 191s] # '1749' => bless( {\r\n[ 191s] # '_status' => 256,\r\n[ 191s] # 'process_id' => 1749\r\n[ 191s] # }, 'Mojo::IOLoop::ReadWriteProcess' ),\r\n[ 191s] # '1755' => bless( {\r\n[ 191s] # '_status' => 256,\r\n[ 191s] # 'process_id' => 1755\r\n[ 191s] # }, 'Mojo::IOLoop::ReadWriteProcess' ),\r\n[ 191s] # '1756' => bless( {\r\n[ 191s] # '_status' => 256,\r\n[ 191s] # 'process_id' => 1756\r\n[ 191s] # }, 'Mojo::IOLoop::ReadWriteProcess' ),\r\n[ 191s] # '1762' => bless( {\r\n[ 191s] # '_status' => 0,\r\n[ 191s] # 'process_id' => 1762\r\n[ 191s] # }, 'Mojo::IOLoop::ReadWriteProcess' ),\r\n[ 191s] # '1763' => bless( {\r\n[ 191s] # '_status' => 0,\r\n[ 191s] # 'process_id' => 1763\r\n[ 191s] # }, 'Mojo::IOLoop::ReadWriteProcess' ),\r\n[ 191s] # '1773' => bless( {\r\n[ 191s] # '_status' => 0,\r\n[ 191s] # 'process_id' => 1773\r\n[ 191s] # }, 'Mojo::IOLoop::ReadWriteProcess' ),\r\n[ 191s] # '1774' => bless( {\r\n[ 191s] # '_status' => 0,\r\n[ 191s] # 'process_id' => 1774\r\n[ 191s] # }, 'Mojo::IOLoop::ReadWriteProcess' )\r\n[ 191s] # },\r\n[ 191s] # 'process_table' => {\r\n[ 191s] # '1744' => \\$VAR1\r\n[ 191s] # },\r\n[ 191s] # 'subreaper' => 1\r\n[ 191s] # }, 'Mojo::IOLoop::ReadWriteProcess::Session' ),\r\n[ 191s] # 'set_pipes' => 1,\r\n[ 191s] # 'subreaper' => 1,\r\n[ 191s] # 'write_stream' => bless( \\*Symbol::GEN121, 'IO::Handle' )\r\n[ 191s] # }, 'Mojo::IOLoop::ReadWriteProcess' )\r\n[ 191s] # Looks like you failed 3 tests of 4.\r\n[ 191s] \r\n[ 191s] # Failed test 'subreaper_bash_roulette'\r\n[ 191s] # at t/07_autodetect.t line 418.\r\n[ 191s] 0 at t/07_autodetect.t line 414.\r\n[ 191s] # Tests were run but no plan was declared and done_testing() was not seen.\r\n[ 191s] # Looks like your test exited with 255 just after 7.\r\n[ 191s] t/07_autodetect.t ........ \r\n[ 191s] Dubious, test returned 255 (wstat 65280, 0xff00)\r\n[ 191s] Failed 1/7 subtests \r\n[ 191s] \t(less 2 skipped subtests: 4 okay)\r\n[ 193s] t/08_ioloop.t ............ ok\r\n[ 194s] t/09_session.t ........... ok\r\n[ 195s] t/10_cgroupv1.t .......... ok\r\n[ 196s] t/10_cgroupv2.t .......... ok\r\n[ 197s] t/11_containers.t ........ skipped: This test works only if you have cgroups permissions\r\n[ 255s] t/12_mocked_container.t .. ok\r\n[ 256s] t/13_shared.t ............ skipped: Skipped unless TEST_SHARED is set\r\n[ 256s] \r\n[ 256s] Test Summary Report\r\n[ 256s] -------------------\r\n[ 256s] t/07_autodetect.t (Wstat: 65280 Tests: 7 Failed: 1)\r\n[ 256s] Failed test: 7\r\n[ 256s] Non-zero exit status: 255\r\n[ 256s] Parse errors: No plan found in TAP output\r\n[ 256s] Files=15, Tests=62, 162 wallclock secs ( 0.25 usr 0.15 sys + 11.89 cusr 2.20 csys = 14.49 CPU)\r\n[ 256s] Result: FAIL\r\n[ 256s] Failed 1/15 test programs. 1/62 subtests failed.\r\n```\r\n\r\nso unhandled output in t/01_run.t, perl warning \"Can't use an undefined value as filehandle reference at lib/Mojo/IOLoop/ReadWriteProcess.pm line 298.\" in t/03_func.t and errors in t/07_autodetect.t\r\n\r\nThen in https://build.opensuse.org/package/live_build_log/devel:openQA:Leap:15.2/perl-Mojo-IOLoop-ReadWriteProcess/openSUSE_Leap_15.2/aarch64\r\n\r\n```\r\n[ 187s] t/11_containers.t ........ skipped: This test works only if you have cgroups permissions\r\n[ 211s] \r\n[ 211s] # Failed test 'procs interface contains the added pids'\r\n[ 211s] # at t/12_mocked_container.t line 37.\r\n[ 211s] # got: ''\r\n[ 211s] # expected: '1785\r\n[ 211s] # '\r\n[ 211s] # 1785\r\n[ 244s] # Looks like you failed 1 test of 43.\r\n[ 244s] \r\n[ 244s] # Failed test 'container_3'\r\n[ 244s] # at t/12_mocked_container.t line 258.\r\n[ 244s] # Looks like you failed 1 test of 3.\r\n[ 244s] t/12_mocked_container.t .. \r\n[ 244s] Dubious, test returned 1 (wstat 256, 0x100)\r\n[ 244s] Failed 1/3 subtests\r\n```\r\n\r\nand in\r\nhttps://build.opensuse.org/package/live_build_log/devel:openQA:Leap:15.2/perl-Mojo-IOLoop-ReadWriteProcess/openSUSE_Leap_15.2/x86_64\r\n\r\n```\r\n[ 70s] t/03_func.t .............. ok\r\n[ 71s] t/04_queues.t ............ ok\r\n[ 3679s] qemu-system-x86_64: terminating on signal 15 from pid 18026 ()\r\n\r\n\r\nJob seems to be stuck here, killed. (after 3600 seconds of inactivity)\r\n```\r\n\r\nso stuck in tests.\r\n\r\n## Acceptance criteria\r\n\r\n* **AC1:** All tests included in OBS checks running stable, i.e. either tests stabilized or excluded from running\r\n\r\n## Suggestions\r\n* Try to reproduce problems locall and try to fix them or exclude them from checks within the spec file with according comments", + "start_date": "2020-10-25", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-10-25T13:44:15Z", + "updated_on": "2020-11-13T08:36:44Z", + "closed_on": null +} +{ + "id": 75214, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 12, + "name": "Workable" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 418, + "name": "Ready" + }, + "subject": "openqa-review fails to post reminder comments on bugzilla, errors in log \"Encountered error trying to post a reminder comment on issue\"", + "description": "## Observation\r\n\r\nfrom https://gitlab.suse.de/openqa/openqa-review/-/jobs/275810/raw\r\n\r\n```\r\n++ /usr/bin/openqa-review --host https://openqa.suse.de -n -r -T --query-issue-status --no-empty-sections --include-softfails --running-threshold=2 --exclude-job-groups '^(Released|Development|old)' --reminder-comment-on-issues --save --save-dir /tmp/tmp.pZbTFYl7W0\r\n........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ERROR:openqa_review.openqa_review:Encountered error trying to post a reminder comment on issue '* [jeos-base+phub_hyperv@svirt-hyperv-uefi](https://openqa.suse.de/tests/4846363 \"Failed modules: firstrun\"), [jeos-filesystem_hyperv@svirt-hyperv-uefi](https://openqa.suse.de/tests/4846366 \"Failed modules: bootloader_uefi\") -> [bsc#1176134](https://bugzilla.suse.com/show_bug.cgi?id=1176134 \"[Build 20.3] JeOS images 15sp3 do not boot in uefi mode\") (Ticket status: VERIFIED (FIXED), prio/severity: P2/Normal, assignee: ms@suse.com)\r\n': 500 Server Error: Internal Server Error for url: https://apibugzilla.suse.com/jsonrpc.cgi. Skipping.\r\n```\r\n\r\n## Steps to reproduce\r\n\r\n* See the instructions called in the gitlab CI definitions in https://gitlab.suse.de/openqa/openqa-review/-/blob/master/.gitlab-ci.yml#L44\r\n\r\nBe aware that the gitlab CI pipeline uses a dedicated user account as visible in https://gitlab.suse.de/openqa/openqa-review/-/blob/master/.gitlab-ci.yml#L9 and https://gitlab.suse.de/openqa/openqa-review/-/settings/ci_cd (see \"Variables\")\r\n\r\n## Problem\r\n\r\nPosting reminder comments from account \"okurz\" works. This is what we use so far on lord.arch.suse.de. Also it worked to post a reminder comment manually in https://bugzilla.suse.com/show_bug.cgi?id=1005824#c10 but so far posting from gitlab CI did not work\r\n\r\n## Suggestions\r\n\r\n* Try again manually with the credentials of the bot account\r\n* Try to read/post bugs over API as openqa-review, e.g. in test bug https://bugzilla.suse.com/show_bug.cgi?id=1005824\r\n* Crosscheck on https://bugzilla.suse.com/buglist.cgi?quicksearch=commenter%3Aopenqa-review if comments are posted correctly\r\n\r\n\r\n## Further details\r\nInitially reported in #52655#note-10", + "start_date": "2020-10-23", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-10-23T18:25:45Z", + "updated_on": "2020-10-23T18:25:45Z", + "closed_on": null +} +{ + "id": 75091, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 1, + "name": "New" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 134, + "name": "Concrete Bugs" + }, + "fixed_version": { + "id": 490, + "name": "future" + }, + "subject": "incomplete jobs with one of the isotovideo sub-processes receiving a signal and just terminating, no clue why or who/what triggered the termination", + "description": "## Observation\r\n\r\nFor example\r\nhttps://openqa.suse.de/tests/4864829/file/autoinst-log.txt\r\nshowing\r\n\r\n```\r\n[2020-10-21T19:31:38.817 CEST] [debug] <<< testapi::wait_still_screen(similarity_level=47, timeout=30, stilltime=3)\r\n\u001b[37m[2020-10-21T19:31:39.498 CEST] [debug] autotest received signal TERM, saving results of current test before exiting\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:39.509 CEST] [debug] [autotest] process exited: 1\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:39.509 CEST] [debug] THERE IS NOTHING TO READ 5 5 4\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:39.510 CEST] [debug] stopping command server 10509 because test execution ended\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:39.510 CEST] [debug] isotovideo: informing websocket clients before stopping command server: http://127.0.0.1:20123/wOZ5cESySrbitZid/broadcast\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:39.526 CEST] [debug] commands process exited: 0\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:39.627 CEST] [debug] done with command server\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:39.627 CEST] [debug] stopping autotest process 10516\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:39.627 CEST] [debug] done with autotest process\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:39.627 CEST] [debug] isotovideo failed\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:39.628 CEST] [debug] stopping backend process 10575\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:39.628 CEST] [debug] backend got TERM\r\n\u001b[0m\u001b[33m[2020-10-21T19:31:39.629 CEST] [info] ::: OpenQA::Qemu::Proc::save_state: Saving QEMU state to qemu_state.json\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:40.697 CEST] [debug] Passing remaining frames to the video encoder\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:40.728 CEST] [debug] Waiting for video encoder to finalize the video\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:40.728 CEST] [debug] The built-in video encoder (pid 10634) terminated\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:40.730 CEST] [debug] QEMU: qemu-system-x86_64: terminating on signal 15 from pid 10575 (/usr/bin/isotovideo: backen)\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:40.731 CEST] [debug] sending magic and exit\r\n\u001b[0m\u001b[37m[2020-10-21T19:31:40.930 CEST] [debug] done with backend process\r\n\u001b[0m10493: EXIT 1\r\n```\r\n\r\nso autotest reports that it \"received signal TERM\", no clue what sent that signal to autotest.\r\n\r\n## Workaround\r\nretrigger", + "start_date": "2020-10-22", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "hard" + } + ], + "created_on": "2020-10-22T10:05:24Z", + "updated_on": "2020-10-22T10:59:17Z", + "closed_on": null +} +{ + "id": 73468, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 2, + "name": "In Progress" + }, + "priority": { + "id": 5, + "name": "High" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "assigned_to": { + "id": 34361, + "name": "ilausuch" + }, + "category": { + "id": 208, + "name": "Organisational" + }, + "fixed_version": { + "id": 418, + "name": "Ready" + }, + "parent": { + "id": 69310 + }, + "subject": "SUSE QA tools team ticket process helpers: Set due date on tickets in redmine based on SLOs", + "description": "## Motivation\r\nSee https://progress.opensuse.org/issues/69310#Motivation\r\n\r\n## Acceptance criteria\r\n* **AC1:** due dates in progress tickets are automatically set based on our SLOs\r\n\r\n## Suggestions\r\n* Research how we are already conducting checks on our backlog from github actions: https://github.com/os-autoinst/scripts/blob/master/.github/workflows/backlog.yml\r\n* Consider an additional GitHub action or similar workflow\r\n* Automatically fetch ticket queries corresponding to SLOs in https://progress.opensuse.org/projects/qa/wiki#How-we-work-on-our-backlog to find tickets which are exceeding the limits\r\n* For each ticket set a due date automatically, e.g. to next day or current day with a comment pointing to SLOs", + "start_date": "2020-10-17", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "easy" + } + ], + "created_on": "2020-10-17T22:01:35Z", + "updated_on": "2020-11-20T13:52:57Z", + "closed_on": null +} +{ + "id": 73369, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 12, + "name": "Workable" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 22072, + "name": "mkittler" + }, + "category": { + "id": 134, + "name": "Concrete Bugs" + }, + "fixed_version": { + "id": 490, + "name": "future" + }, + "subject": "Job incompletes with auto_review:\"(?s)backend died: runcmd .*qemu-img create -f qcow2 .* failed with exit code 1: 'Formatting .*\" on o3", + "description": "### observation\r\n\r\n```\r\n[2020-10-11T16:08:25.517 CEST] [debug] qemu version detected: 3.1.1.1\r\n[2020-10-11T16:08:25.519 CEST] [debug] running /usr/bin/chattr -f +C /var/lib/openqa/pool/3/raid\r\n[2020-10-11T16:08:25.541 CEST] [debug] running /usr/bin/qemu-img info --output=json /var/lib/openqa/pool/3/openSUSE-Leap-15.1-DVD-x86_64.iso\r\n[2020-10-11T16:08:25.571 CEST] [debug] running /usr/bin/qemu-img create -f qcow2 /var/lib/openqa/pool/3/raid/hd0 40G\r\n[2020-10-11T16:08:25.600 CEST] [debug] Formatting '/var/lib/openqa/pool/3/raid/hd0', fmt=qcow2 size=42949672960 cluster_size=65536 lazy_refcounts=off refcount_bits=16\r\n[2020-10-11T16:08:25.600 CEST] [debug] Backend process died, backend errors are reported below in the following lines:\r\nruncmd '/usr/bin/qemu-img create -f qcow2 /var/lib/openqa/pool/3/raid/hd0 40G' failed with exit code 1: 'Formatting '/var/lib/openqa/pool/3/raid/hd0', fmt=qcow2 size=42949672960 cluster_size=65536 lazy_refcounts=off refcount_bits=16' at /usr/lib/os-autoinst/osutils.pm line 121.\r\n```\r\n\r\nhttps://openqa.opensuse.org/tests/1428212\r\n\r\n## Acceptance criteria\r\n* **AC1:** Errors returned by qemu-img either are handled internally, e.g. with retry, or clear error messages are escalated to test reviewers\r\n\r\n## Suggestions\r\n\r\n* Research if we do not already have a retry for qemu-img commands in os-autoinst, fix if broken", + "start_date": "2020-10-14", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-10-14T11:13:01Z", + "updated_on": "2020-10-29T22:04:45Z", + "closed_on": null +} +{ + "id": 73309, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 12, + "name": "Workable" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 490, + "name": "future" + }, + "subject": "every time a direct dependency is updated in Factory our CI jobs fail until the package is updated", + "description": "## Observation\r\n\r\nhttps://github.com/os-autoinst/openQA/pull/3463 is a very simple change. The CI jobs fail immediately on circleCI in the \"cache\" step with:\r\n\r\n```\r\n+ sudo zypper -n install --download-only aspell-0.60.6.1…ShellCheck-0.6.0\r\nLoading repository data...\r\nReading installed packages...\r\nPackage 'perl-Mojolicious-8.61' not found.\r\n```\r\n\r\nbecause the package perl-Mojolicious-8.61 does not exist anymore in this version in devel:openQA, devel:openQA:Leap:15.1 and openSUSE:Factory as it is already at 8.62 so the old files vanished. Retriggering the PR as well as rebasing the PR would not help because only any next nightly job would update the version specific dependency list.\r\n\r\n# Problem\r\n\r\nThe problem was introduced when we moved to circleCI together with a \"new\" approach to hardcode versions of packages so that pull requests would not suddenly fail if any updated dependency would by coincidence appear the first time. However it feels like now we have more or less the same problem, less subtle failures maybe but even more likely than what we wanted to avoid.\r\n\r\n## Acceptance criteria\r\n* **AC1:** Pull requests do not fail tests just because suddenly a package has a new version\r\n\r\n## Suggestions\r\n* Research how other projects cope with this\r\n* Try to find an approach to make our dependencies we test consistent but available. How about having *all* dependencies pre-installed in a container and test based on that?", + "start_date": "2020-10-13", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-10-13T21:41:51Z", + "updated_on": "2020-10-29T22:04:44Z", + "closed_on": null +} +{ + "id": 73285, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 15, + "name": "Blocked" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "assigned_to": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 418, + "name": "Ready" + }, + "parent": { + "id": 62420 + }, + "subject": "test incompletes with auto_review:\"(?s)Download of.*processed[^:].*Failed to download\":retry , not helpful details about reason of error", + "description": "## Observation\r\n\r\nhttps://openqa.suse.de/tests/4811018/file/autoinst-log.txt\r\n\r\nshows just\r\n\r\n```\r\n[2020-10-12T17:00:42.0701 CEST] [info] [pid:36912] Downloading SLES-12-SP5-x86_64-mru-install-minimal-with-addons-Build:16741:php7-Server-DVD-Incidents-64bit.qcow2, request #42551 sent to Cache Service\r\n[2020-10-12T17:00:47.0773 CEST] [info] [pid:36912] Download of SLES-12-SP5-x86_64-mru-install-minimal-with-addons-Build:16741:php7-Server-DVD-Incidents-64bit.qcow2 processed\r\n[2020-10-12T17:00:47.0778 CEST] [error] [pid:36912] Failed to download SLES-12-SP5-x86_64-mru-install-minimal-with-addons-Build:16741:php7-Server-DVD-Incidents-64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/SLES-12-SP5-x86_64-mru-install-minimal-with-addons-Build:16741:php7-Server-DVD-Incidents-64bit.qcow2\r\n[2020-10-12T17:00:47.0826 CEST] [info] [pid:36912] +++ worker notes +++\r\n[2020-10-12T17:00:47.0826 CEST] [info] [pid:36912] End time: 2020-10-12 15:00:47\r\n[2020-10-12T17:00:47.0826 CEST] [info] [pid:36912] Result: setup failure\r\n```\r\n\r\nand the worker log https://openqa.suse.de/tests/4811018/file/worker-log.txt\r\n\r\n```\r\n[2020-10-12T17:00:47.0778 CEST] [error] [pid:36912] Unable to setup job 4811018: Failed to download SLES-12-SP5-x86_64-mru-install-minimal-with-addons-Build:16741:php7-Server-DVD-Incidents-64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/SLES-12-SP5-x86_64-mru-install-minimal-with-addons-Build:16741:php7-Server-DVD-Incidents-64bit.qcow2\r\n[2020-10-12T17:00:47.0778 CEST] [debug] [pid:36912] Stopping job 4811018 from openqa.suse.de: 04811018-sle-12-SP5-Server-DVD-Incidents-x86_64-Build:16741:php7-mau-extratests-kdump@64bit - reason: setup failure\r\n[2020-10-12T17:00:47.0778 CEST] [debug] [pid:36912] REST-API call: POST http://openqa.suse.de/api/v1/jobs/4811018/status\r\n[2020-10-12T17:00:47.0859 CEST] [info] [pid:25653] Uploading autoinst-log.txt\r\n[2020-10-12T17:00:47.0932 CEST] [info] [pid:25653] Uploading worker-log.txt\r\n```\r\n\r\nthere is no clue on why \"SLES-12-SP5-x86_64-mru-install-minimal-with-addons-Build:16741:php7-Server-DVD-Incidents-64bit.qcow2\" could not be downloaded, from where to where it should have been downloaded, etc. There is no http status code or anything, like no \"404 Not Found\" or similar.\r\n\r\n## Acceptance criteria\r\n* **AC1:** The real reason for the download error is accessible in the job details (either \"reason\" or autoinst-log.txt), e.g. http status code with message\r\n\r\n## Suggestions\r\n\r\n* Research what the actual error could be and add it as a \"likely explanation\" in the existing error string\r\n* Try to include the actual error details into the affected jobs logs\r\n* If possible, adapt the \"reason\" to be more specific with details about the actual error", + "start_date": "2020-07-30", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-10-13T10:27:39Z", + "updated_on": "2020-11-02T16:47:21Z", + "closed_on": null +} +{ + "id": 71809, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 12, + "name": "Workable" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 23378, + "name": "asmorodskyi" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 490, + "name": "future" + }, + "subject": "Enable multi-machine jobs trigger without \"isos post\"", + "description": "In case of single job you have two options to trigger it : \r\n1.) isos post ( requires certain setup on target openQA ) \r\n2.) jobs post ( can be done on any \"random\" openQA server because all variables are not calculated from flavors , job groups , machines etc. but coming with a call ) \r\n\r\nIn case of multi-machine job you have only first option . \r\nWe need to find a way to trigger multi-machine jobs on target openQA host without require certain setup on it", + "start_date": "2020-09-24", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-09-24T08:46:28Z", + "updated_on": "2020-10-16T10:12:29Z", + "closed_on": null +} +{ + "id": 71794, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 1, + "name": "New" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 25092, + "name": "pcervinka" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 490, + "name": "future" + }, + "subject": "[powervm] Support images in HMC backend on PowerVM", + "description": "There is no image/snapshot support on PowerVM backend and it must be treated like bare metal and it has following limitations:\r\n- tests can be scheduled in one job with installation\r\n- tests can chained with START_DIRECTLY_AFTER variable, which will make sure that tests will be scheduled on the same worker\r\n\r\nThere were already some discussions in the past with several people around, it seems to be possible, but it didn't end in any ticker or action.\r\n\r\nCould you please evaluate it?\r\n\r\n## Expected result\r\n- installation on PowerVM HMC backend will produce image\r\n- image will be reused by tests\r\n- tests can be scheduled on any available worker\r\n- tests can be restarted in solo way (without installation job)", + "start_date": "2020-09-24", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-09-24T07:25:49Z", + "updated_on": "2020-09-24T07:41:12Z", + "closed_on": null +} +{ + "id": 71758, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 12, + "name": "Workable" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 418, + "name": "Ready" + }, + "parent": { + "id": 58184 + }, + "subject": "[spike][timeboxed:20h] complete test definition from a yaml schedule file in local test distribution folder", + "description": "## Motivation\r\nSee #58184 . As a test maintainer using openQA I want openQA to be able to load the complete test definition from a yaml schedule in a local test distribution folder so that all the necessary configuration can come from a central place and no configuration needs to be added to the openQA database beforehand\r\n\r\n\r\n## Acceptance criteria\r\n* **AC1:** proof-of-concept exists for a complete test definition consisting of at least two scenarios (1) loadable from text files within the local test distribution folder\r\n\r\n## Suggestions\r\n\r\n* See https://github.com/os-autoinst/openQA/pull/2706 for a draft of the idea\r\n\r\n\r\n## References\r\n\r\n1. Note 1 in AC1: A single job can be completely specified by test parameters on job creation time already. So the necessary challenge is that we have test definitions as commonly done by the combination of medium types, test suites, job templates, job groups (and machines and worker config) to form a test schedule consisting of multiple scenarios, i.e. at least two", + "start_date": null, + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-09-23T17:34:55Z", + "updated_on": "2020-11-19T10:32:40Z", + "closed_on": null +} +{ + "id": 70927, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 1, + "name": "New" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 32261, + "name": "tuanpembual" + }, + "category": { + "id": 208, + "name": "Organisational" + }, + "fixed_version": { + "id": 490, + "name": "future" + }, + "subject": "Testing Outbound Amail", + "description": "Testing if redmine@o.o will work", + "start_date": "2020-09-04", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-09-03T23:53:10Z", + "updated_on": "2020-09-19T03:19:15Z", + "closed_on": null +} +{ + "id": 70915, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 1, + "name": "New" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 20030, + "name": "favogt" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 490, + "name": "future" + }, + "subject": "Bugref comments carried over even if status changed failed -> incomplete", + "description": "See https://openqa.opensuse.org/tests/1381130#next_previous\r\n\r\nThe comment from the last failing test was carried over to the current incomplete one as the set of failing modules is equal.\r\nHowever, in the case of the incomplete test some modules were not executed at all, so their state is unknown at that point.\r\n", + "start_date": "2020-09-03", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-09-03T13:22:30Z", + "updated_on": "2020-09-09T07:18:10Z", + "closed_on": null +} +{ + "id": 70876, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 12, + "name": "Workable" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 22072, + "name": "mkittler" + }, + "category": { + "id": 134, + "name": "Concrete Bugs" + }, + "fixed_version": { + "id": 490, + "name": "future" + }, + "subject": "Job incompletes with \"failed to start VM\" and no further detail", + "description": "### observation\r\n\r\nNot sure why this job incompleted:\r\n\r\n```\r\n\u001b[37m[2020-09-02T00:15:10.754 CEST] [debug] Current version is 4.6.1598623827.924c995d [interface v20]\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:10.840 CEST] [debug] git hash in /var/lib/openqa/cache/openqa1-opensuse/tests/opensuse: ab7782333c508e136760d4299a0c32297a5c38fc\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:11.276 CEST] [debug] usingenv DESKTOP=kde\r\n…\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:11.278 CEST] [debug] usingenv NOIMAGES=1\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:11.294 CEST] [debug] scheduling isosize tests/installation/isosize.pm\r\n…\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:11.520 CEST] [debug] scheduling shutdown tests/shutdown/shutdown.pm\r\n\u001b[0m[2020-09-02T00:15:11.577 CEST] [info] cmdsrv: daemon reachable under http://*:20033/FDK5NTQyiHRYvqGw/\r\n\u001b[37m[2020-09-02T00:15:11.579 CEST] [debug] git hash in /var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles: 73ebc3acc3c3cd24f09d9f6c15edc470eba6e5eb\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:11.580 CEST] [debug] init needles from /var/lib/openqa/cache/openqa1-opensuse/tests/opensuse/products/opensuse/needles\r\n\u001b[0m[2020-09-02T00:15:11.583 CEST] [info] Listening at \"http://[::]:20033\"\r\nWeb application available at http://[::]:20033\r\n\u001b[37m[2020-09-02T00:15:12.201 CEST] [debug] inst-console-20200224 contains inst-console twice\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:13.370 CEST] [debug] loaded 8184 needles\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:13.634 CEST] [debug] Blocking SIGTERM\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:13.721 CEST] [debug] Unblocking SIGTERM\r\n\u001b[0m5575: channel_out 15, channel_in 14\r\n\u001b[37m[2020-09-02T00:15:13.990 CEST] [debug] Blocking SIGTERM\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:14.122 CEST] [debug] Unblocking SIGTERM\r\n\u001b[0m5966: cmdpipe 13, rsppipe 16\r\n\u001b[37m[2020-09-02T00:15:14.125 CEST] [debug] started mgmt loop with pid 5966\r\n\u001b[0mremove_tree qemuscreenshot\r\n\u001b[37m[2020-09-02T00:15:14.382 CEST] [debug] qemu version detected: 3.1.1.1\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:14.709 CEST] [debug] running /usr/bin/chattr -f +C /var/lib/openqa/pool/3/raid\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:15.238 CEST] [debug] running /usr/bin/qemu-img info --output=json /var/lib/openqa/pool/3/openSUSE-Jump-15.2-DVD-aarch64-Build72.2-Media.iso\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:15.443 CEST] [debug] Driver backend collected unknown process with pid 6596 and exit status: 0\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:15.477 CEST] [debug] backend process exited: 0\r\n\u001b[0mfailed to start VM at /usr/lib/os-autoinst/backend/driver.pm line 128.\r\n\u001b[37m[2020-09-02T00:15:15.485 CEST] [debug] stopping command server 5612 because test execution ended through exception\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:16.690 CEST] [debug] done with command server\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:16.690 CEST] [debug] stopping autotest process 5645\r\n\u001b[0m\u001b[37m[2020-09-02T00:15:17.893 CEST] [debug] done with autotest process\r\n\u001b[0m5575: EXIT 1\r\n[2020-09-02T00:15:18.0121 CEST] [info] Isotovideo exit status: 1\r\n[2020-09-02T00:15:18.0153 CEST] [info] +++ worker notes +++\r\n[2020-09-02T00:15:18.0154 CEST] [info] End time: 2020-09-01 22:15:18\r\n[2020-09-02T00:15:18.0154 CEST] [info] Result: died\r\n[2020-09-02T00:15:18.0238 CEST] [info] Uploading video.ogv\r\n[2020-09-02T00:15:18.0319 CEST] [info] Uploading video_time.vtt\r\n[2020-09-02T00:15:18.0355 CEST] [info] Uploading vars.json\r\n[2020-09-02T00:15:18.0381 CEST] [info] Uploading autoinst-log.txt\r\n```\r\n\r\nThe message `isotovideo died: failed to start VM at /usr/lib/os-autoinst/backend/driver.pm line 128.` is saved as reason but I'm not sure whether it is really the source of the problem or just a symptom.\r\n\r\n---\r\n\r\njob reference: https://openqa.opensuse.org/tests/1380901\r\n\r\n## Steps to reproduce\r\n\r\nFind reproducing jobs on production with\r\n\r\n```\r\nfor i in o3 osd; do ssh $i \"sudo -u geekotest psql openqa -c \\\"select id,t_finished,test,reason from jobs where t_finished >= '2020-09-01' and reason ~ 'isotovideo died: failed to start VM' order by t_finished desc limit 20;\\\"\"; done\r\n```\r\n\r\n## Expected result\r\nProblem does not happen anymore or if it happens there is more helpful output.\r\n\r\n## Suggestions\r\n* Check current test code coverage in os-autoinst for the error handling of backend/driver.pm:128 with the line `$self->_send_json({cmd => 'start_vm'}) || die \"failed to start VM\";`\r\n* Ensure the code is properly covered with tests\r\n* Try to reproduce the above problem\r\n* Improve feedback in these cases", + "start_date": "2020-09-02", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "hard" + } + ], + "created_on": "2020-09-02T15:48:01Z", + "updated_on": "2020-10-29T22:04:42Z", + "closed_on": null +} +{ + "id": 70840, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 6, + "name": "coordination" + }, + "status": { + "id": 1, + "name": "New" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 490, + "name": "future" + }, + "subject": "[epic] further features for the \"search\" feature", + "description": "## Motivation\r\n\r\n\r\n## Acceptance criteria\r\n\r\n\r\n## Suggestion\r\n\r\n* clickable search results. We should link results, e.g. a job template links to the job group.\r\n* names of test suite database #71422\r\n* https://github.com/okurz/scripts/blob/master/openqa-db_query_last_use_of_module provides an external tool relying on database access but it can help to find out scenarios running a specified test module.\r\n* add keyboard shortcut to reach search field, again maybe just like github\r\n* optional: include job groups in search as well\r\n* optional: add search qualifiers, e.g. `testsuite:` or `finished:` (for jobs)\r\n* see https://developer.github.com/v3/search/ for reference\r\n* optional: extend for other stuff to search for, e.g. users, audit events, needles, assets, bugs, everything else we have in openQA … except maybe API keys and secrets ;)\r\n* mkittler also suggested we display the type of result, maybe as an icon.\r\n* optional: Include search field in the tutorial popup \"feature tour\"", + "start_date": "2020-09-16", + "due_date": null, + "done_ratio": 100, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-09-02T09:20:00Z", + "updated_on": "2020-10-12T13:45:06Z", + "closed_on": null +} +{ + "id": 70612, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 1, + "name": "New" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 33956, + "name": "MDoucha" + }, + "category": { + "id": 142, + "name": "Feature requests" + }, + "fixed_version": { + "id": 490, + "name": "future" + }, + "subject": "better error handling in testapi function script_output (was: script_output() ignores parse errors)", + "description": "## Motivation\r\nI've just spent 2 hours staring into code trying to figure out why `get_patches()` in lib/qam.pm rejected an update that has the right incident ID and is marker as needed.\r\nhttps://openqa.suse.de/tests/4596773#step/update_kernel/63\r\n\r\nThen I've noticed that the leading output marker is malformed due to VNC typing issue so `script_output()` simply returned an empty string to `get_patches()`.\r\n\r\nWhen `script_output()` fails to parse the output, it shouldn't silently return an empty string. Suggestion: It should throw an exception.\r\n\r\n\r\n## Acceptance criteria\r\n* **AC1:** Parsing errors in script_output can be easily distinguished from a false boolean result from the internal called script command\r\n\r\n\r\n## Suggestions\r\n* Review and potentially extend os-autoinst t/03-testapi.t for how script_output behaves on an error like lost characters leading to unparseable responses", + "start_date": "2020-08-27", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "easy" + } + ], + "created_on": "2020-08-27T14:16:53Z", + "updated_on": "2020-09-03T06:50:26Z", + "closed_on": null +} +{ + "id": 69451, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 6, + "name": "coordination" + }, + "status": { + "id": 15, + "name": "Blocked" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "assigned_to": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 134, + "name": "Concrete Bugs" + }, + "fixed_version": { + "id": 418, + "name": "Ready" + }, + "parent": { + "id": 62420 + }, + "subject": "[epic] test incompletes with \"(?s)Download.*successful.*Failed to download\":retry, not helpful details", + "description": "## Observation\r\n\r\nhttps://openqa.suse.de/tests/4504422 is incomplete with reason \"setup failure: Failed to download SLEHPC-15-SP2-aarch64-GM-SERVER-gnome.qcow2 to /var/lib/openqa/cache/openqa.suse.de/SLEHPC-15-SP2-aarch64-GM-SERVER-gnome.qcow2\" so no explanation of the real reason in there. The log content of autoinst-log.txt as shown on the details page shows:\r\n\r\n```\r\n[info] [#2427] Download of \"/var/lib/openqa/cache/openqa.suse.de/SLEHPC-15-SP2-aarch64-GM-SERVER-gnome.qcow2\" successful, new cache size is 50GiB\r\n…\r\n[2020-07-30T04:39:00.0302 CEST] [error] [pid:16217] Failed to download SLEHPC-15-SP2-aarch64-GM-SERVER-gnome.qcow2 to /var/lib/openqa/cache/openqa.suse.de/SLEHPC-15-SP2-aarch64-GM-SERVER-gnome.qcow2\r\n```\r\n\r\nwhich is certainly weird and not expected.\r\n\r\n\r\n## Acceptance criteria\r\n* **AC1:** No \"download successful\" should end with \"Failed to download\" referencing the same asset\r\n\r\n\r\n## Suggestions\r\n\r\n* Compare to #69448\r\n* Research what the actual error could be and add it as a \"likely explanation\" in the existing error string\r\n* Try to include the actual error details into the affected jobs logs\r\n* If possible, adapt the \"reason\" to be more specific with details about the actual error", + "start_date": "2020-07-30", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-07-30T07:51:46Z", + "updated_on": "2020-10-27T08:28:38Z", + "closed_on": null +} +{ + "id": 69310, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 6, + "name": "coordination" + }, + "status": { + "id": 15, + "name": "Blocked" + }, + "priority": { + "id": 5, + "name": "High" + }, + "author": { + "id": 32300, + "name": "cdywan" + }, + "assigned_to": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 208, + "name": "Organisational" + }, + "fixed_version": { + "id": 418, + "name": "Ready" + }, + "subject": "[epic] SUSE QA tools team ticket process helpers", + "description": "## Motivation\r\nWe want to follow the https://progress.opensuse.org/projects/openqav3/wiki#ticket-workflow including https://progress.opensuse.org/projects/qa/wiki#How-we-work-on-our-backlog which define some SLOs for ourselves which are hard and tedious to track as humans but bots would be happy to do that job.\r\n\r\n## Ideas\r\n\r\n* Query Redmine via GitLab CI bot to update due date: On GitHub we have bots checking that certain parameters are met, such as stale PRs or conflicts. It would be great to update the Due Date automatically according to tickets that need updates, taking priority into account, which then allow Redmine to send out notifications. As an alternative write comments directly.", + "start_date": "2020-07-24", + "due_date": null, + "done_ratio": 50, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-07-24T10:18:02Z", + "updated_on": "2020-10-29T15:17:50Z", + "closed_on": null +} +{ + "id": 68413, + "project": { + "id": 18, + "name": "openQA Project" + }, + "tracker": { + "id": 4, + "name": "action" + }, + "status": { + "id": 12, + "name": "Workable" + }, + "priority": { + "id": 4, + "name": "Normal" + }, + "author": { + "id": 17668, + "name": "okurz" + }, + "category": { + "id": 134, + "name": "Concrete Bugs" + }, + "fixed_version": { + "id": 490, + "name": "future" + }, + "subject": "De-duplicate definitions of needle soft-fail markers (was: screenshot shows soft-fail border for a needle with bug reference but the module turns out to be \"passed\", expected \"softfailed\")", + "description": "## Observation\r\n\r\nsee https://openqa.opensuse.org/tests/1306944#step/wine/26\r\n\r\n\r\n## Reproducible\r\n\r\nReproduced, e.g. in the same scenario in other jobs the same\r\n\r\n\r\n## Expected result\r\n\r\nThe softfailed needle should make the module result turn \"softfailed\" as well\r\n\r\n\r\n## Problem\r\n\r\nLikely a regression that was introduced within the last year with some changes regarding the handling of softfails\r\n\r\n\r\n## Further details\r\n\r\nAlways latest result in this scenario: [latest](https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=opensuse&flavor=DVD&machine=64bit&test=extra_tests_on_gnome&version=15.2)\r\n", + "start_date": "2020-06-24", + "due_date": null, + "done_ratio": 0, + "is_private": false, + "estimated_hours": null, + "custom_fields": [ + { + "id": 16, + "name": "Difficulty", + "value": "" + } + ], + "created_on": "2020-06-24T20:17:12Z", + "updated_on": "2020-10-29T22:04:41Z", + "closed_on": null +}