LP1712301: Need S3 stress test for discrete graphic #60
Labels
Comments
|
This thread was migrated from launchpad.net https://launchpad.net/~kissiel wrote on 2017-12-03 17:12:18:Fix landed here: https://code.launchpad.net/~kissiel/plainbox-provider-checkbox/+git/plainbox-provider-checkbox/+merge/333607 https://launchpad.net/~ray.chen wrote on 2019-01-18 09:13:00:This improvement was verified and test on Somerville-graphic test plan after manual replace all eg: https://launchpad.net/~ray.chen wrote on 2019-04-25 08:55:25:The fix was only fixed dual graphic auto switch, OEM projects need a suspend_30_cycles_with_reboots/logs for dual graphic like
https://launchpad.net/~ray.chen wrote on 2019-04-25 08:56:13:reopen this issue based on https://docs.google.com/document/d/1Cj-HMAnY-W1uwoYIgTW36aFz3fIbQzVzjKpLktIt9Is/edit#heading=h.4lu4w12akx1hhttps://launchpad.net/~jocave wrote on 2019-04-25 09:08:41:Discussion on irc, the effective operations that checkbox should carry out are: switch to iGPU, reboot, 30 S3, reboot, 30 S3, reboot, 30 S3, attach logs |
|
It's related to old way of running tests so I think this is no longer relevant. |
pieqq
pushed a commit
that referenced
this issue
Jan 12, 2024
Some of the project devices' caam_hwrng have buffer, so previously the dd size is increased to 10M to exceed the buffer size to make the interrupt increase. It will make the device without buffer take a very long time (over 30 mins) to do the dd command. Therefore, this commit separated the size to 512K and looped it 20 times. On the device without buffer it could do 512K and detect the increased interrupt to get the pass result, which is much faster. On the device with buttfer it could also exceed the buffer by dd 20 times and detect the increased interrupt to get the pass result.
pieqq
pushed a commit
that referenced
this issue
Jan 12, 2024
Some of the project devices' caam_hwrng have buffer, so previously the dd size is increased to 10M to exceed the buffer size to make the interrupt increase. It will make the device without buffer take a very long time (over 30 mins) to do the dd command. Therefore, this commit separated the size to 512K and looped it 20 times. On the device without buffer it could do 512K and detect the increased interrupt to get the pass result, which is much faster. On the device with buttfer it could also exceed the buffer by dd 20 times and detect the increased interrupt to get the pass result.
kissiel
pushed a commit
that referenced
this issue
Jan 12, 2024
Some of the project devices' caam_hwrng have buffer, so previously the dd size is increased to 10M to exceed the buffer size to make the interrupt increase. It will make the device without buffer take a very long time (over 30 mins) to do the dd command. Therefore, this commit separated the size to 512K and looped it 20 times. On the device without buffer it could do 512K and detect the increased interrupt to get the pass result, which is much faster. On the device with buttfer it could also exceed the buffer by dd 20 times and detect the increased interrupt to get the pass result.
kissiel
pushed a commit
that referenced
this issue
Jan 12, 2024
* Initial commit * Migrate ARM generic jobs from launchpad * Initial commit * Clean up this repo, and up it to date. * Change: job id for cpu cases * Change: id of rpmsg * Add: manage.py Since we need this repo as a provider * Change: separate jobs into generic and non-generic * Add: checkbox-provider-arm folder Following the structure of checkbox * Change: the access of tool not created by us To prevent those tools been packaged into checkbox * Add: test plan for ARM generic jobs * Change: test method for NPU TensorFlow Change the test method by using test snap build by Isaac * Change: MTD test method Since we don't have reliable way to indentify the type of MTD. * Change: test method for buzzer Change the test method for buzzer that probe as an input PC speaker * Change: chunkfs test method * Add: dump DTB file * Add: manifest for the jobs * Add: rs485 remote test * Remove: non-generic jobs Since we need to move generic parts to github Co-authored-by: Nara Huang <nara.huang@canonical.com> * Add snapcraft yaml for uc20 (#2) * Add: snapcraft.yaml for uc20 * Add: README.md * Fix: export PYTHONPATH in override-build section * Add: libc6-dev for build-packages of input-pcspkr Since snapcraft7.x will need extra lib from libc6-dev * Fix cpu and device tree cases (#3) * Fix: cpu offlining test * Change: requires of cpu offlineing test * Add: stage-package linuxptp * Fix: test plan for dtb * Add eeprom test and change name of the provider (#6) * Add: eeprom test cases * Change: provider name from arm to ce-oem * Change name space and README.md (#9) * Change: name space * Change: README.md * Fix: wording of checkbox interface snap * Remove: snapcraft.yaml (#8) * Add: led generic test (#7) * Add: support cold reboot tests by pdu (#10) * Add: support cold reboot tests by pdu add cold reboot tests by network PDU for those system which not able to wakeup by rtc * Fix: fix bugs add following changes in this commit - check supported pdu type inside of main function - sync data after dump device information - correct command for init-boot-loop-data job * Fix: fix bug - fix the error reported by shellcheck * Change prefixes (#11) * Change: prefixes for plans and jobs * Change: rename test plan and add PDU related plan * Change: name of category * Fix: missing prefixes of serial test job and plan * Add: informations about used config in readme (#13) * Change: sysfs led method (#12) * Fix: rtc wakealarm checking to right attribute (#14) * Fix: remove ce-oem-power-automated-by-pdu Remove the ce-oem-power-automated-by-pdu from the ce-oem-automated The ce-oem-power-automated-by-pud will always be executed even the user not intend to reboot the DUT by PDU. And the checkbox session will stuck after send out a snmpset command. So remove this test first, and we need to integrate the cold boot related tests to PPC to support cold reboot test for those DUT which not able to wake up by rtc. * Add: add a generic iot test plans Adding a generic IoT test plans, it's a full testing set include tests in certification test plan and ce-oem-generic test plan * Improve the MTD script Before doing the read and write testing of specific mtd partition, we need to know the MTD is read-only or writable, therefore, a new variable called is_mtd_writable to help us to know it. If this mtd is read-only, I only read its content to a file. If this mtd is writable, backup the its original content to a file, then do the compare procedure, finally, recover its content. * Add: CAAM related test (#21) * Add: CAAM related test * Add: generic crypto test * Fix naming with ce-oem * Add: add digital-io tests (#24) * Add: add digital-io tests add digital-io tests * Fix: fixed scripts and test plan - fixed the format string style - fixed the typo in docstring - fixed the variable type for TEST_STATES - remove useless test jobs * Fix: fixed typo fixed typo * Add: add readme (#23) * Fix: updated README.md add a section about how to use checkbox-ce-oem as a checkbox launcher * Fix: updated the installation steps updated the installation steps for Ubuntu classic and Ubuntu core * fix: fixed typo fixed typo * Fix: updated digital-io tests (#25) - corrected digital-io tests in post-suspend stage - import manifest job * Add: cpufreq governors test for 6 governors (#19) * Add: cpufreq governors test for 6 governors Add a resource job to check system supported governors, and test jobs for all 6 governors. These tests are only applicable to the platform using acpi_cpufreq related drivers. Newer intel CPU using intel_pstate driver is NOT applicable. * Fix: typo and apply suggestions * Remove: ce-oem-cpu/offlining_test from the plan Since there is already a fix for cpu/offlining_test in checkbox base provider, this ce-oem-cpu/offlining_test is no longer required. #522 * Modify CAAM and installation time related jobs. (#26) * Modify: modify hwrng and caam algo check to require manifest * Add: requires for installation related jobs Make those jobs run only in ubuntu core * Fix: revert summary section for snapd_installation_time The previous commit accidentally removed the section * Fix script for pep8 check (#27) * Remove: test job/script that not in use. Since we have new script for cpu freq governors. And also cpu_offline test has been updated in base provider. * Fix: python script to fit flake8 * Fix: manifest shouldn't place in resource job Move the manifest to test job. To prevent manifest being executed at bootstrap stage. * Add code check by tox just like checkbox did (#28) * Add code check by tox just like checkbox did This commit following what checkbox did, and add pep8-naming check for python 3.8+. * Change the fork checkbox git repo url to offical one * Fix tox test environment error (#32) * Fix tox test environment error Each tox test environment won't be reseted, therefore removing all providers before test is needed. After checking with checkbox repo, checkbox-ng has to use different install method to install. * Follow checkbox repo to update pyYAML version to 6.0.1 for python 3.10+ to solve tox issue * Fix for shellcheck (#29) * Fix: shell scrip to fit shellcheck * Fix: audio jobs to fit shellcheck * Fix: caam jobs to fit shellcheck * Fix: device jobs to fit shellcheck * Fix: digital-io jobs to fit shellcheck * Change: nested EEPROM test from checkbox base Since EEPROM jobs merged back to the base provider already. * Fix: power-manager jobs to fit shellcheck * Change: nested RTC test from checkbox base Since RTC jobs merged back to the base provider already. * Fix: stress jobs to fit shellcheck * Fix: theraml-sensor jobs to fit shellcheck * Fix: Nested plan to ce-oem Follow the structure as a client-cert plan * Add multiple usb otg test (#33) * Remove: OTG related jobs Those OTG jobs have been migrated back to base provider. * Add: multiple OTG test * Add: readme.md * Refactor cpufreq governor test (#36) * Change: refactor cpufreq governor test CQT-2906 The original tests did not handle the following 2 conditions: 1. CPUfreq scaling driver not found (no files under cpufreq folder) 2. On ARM device, system using multiple policies to handle different CPUs group This commit refactor the tests to make them policies-based tests. Also adding a driver detect job that can be use on all platforms. * Add optee test (#34) * Add: optee test * Add: README.md * Include led test from base (#35) * Add: nested led job from checkbox-base * Change: README.md for led * Change: PTP command (#37) * Change: PTP command lp:2030942, workaround for the current version of ptp4l * Fix: apply musical scales for pwm buzzer test (#40) * Fix: apply musical scales for pwm buzzer test apply musical scales for pwm buzzer test * Fix: added description for test_pwm_buzzer func added description for test_pwm_buzzer function * Fix: add touchscreen tests (#39) * Fix: add touchscreen tests validate the required touch events for touchscreen also add new test to cover maximum fingers support on touchscreen * Change: hardware PTP support keywords For PTP capabilities, ethtool v5.4 (on Ubuntu 20.04) shows `hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE)` But ethtool v5.16 (on Ubuntu 22.04) only shows `hardware-transmit` The test case was using `SOF_TIMESTAMPING_TX_HARDWARE` as keywords, This commit changes it to use `hardware-transmit` for adapting new version of ethtool. * Remove: nested ptp-manual test plans The current manual test job steps are duplicated with the auto test job. Currently we only need to run the auto test job. The manual test steps here is used as a reference. If there are new jobs in th future, the ptp-manual test plan can be nested back. * Add: LD_PATH information in readme.md (#41) * Change: additional patterns to map UDC and USB node (#42) * Change: additional patterns to map UDC and USB node * Fix: the if else logic for map UDC * Fix: include python3-systemd in tox.ini (#48) include the python3-systemd package in tox.ini * Fix: updated the related packages for tox (#50) updated the related packages for tox * Fix: nested plan for server manual (#44) * Fix: nested plan for server manual * Fix: errors when mtd count is 0 (#55) * Fix: errors when mtd count is 0 * Add shellcheck ignore 2126 * Add: rpmsg tests for i.MX and TI processors (#46) * Fix: updated the related packages for tox updated the related packages for tox * Add: rpmsg tests for i.MX and TI processors adding rpmsg tests for i.MX and TI processors * Fix: updated rpmsg_tests.py scripts updated rpmsg_tests.py scripts * Fix: fixed bugs fixed bugs * Modify optee test run out of gadget (#45) * Add: support outsourcing optee-test * 230921 crypto test (#52) * Fix: updated the related packages for tox updated the related packages for tox * Fix: update the crypto tests seperate the crypto tests to the generic crypto tests and crypto accelerator tests * Fix: bypass if failed to set extra seed the extra seed might not be supported for all platform * Fix: fixed bugs fixed bugs * Fix: refactor check_crypto_driver_priority func refactor check_crypto_driver_priority function * Fix: mock systemd-python package and remove packages in tox mock systemd-python package and remove packages in tox * Revamp otg_ports resource job should not fail (#53) * Revamp otg_ports resource job should not fail Add a if condition to handle when the OTG config variable is not set. * Revise the echo string * Add: cupfreq module probe (#51) * Add: cupfreq module probe * Check hwrng test (#57) * Modify hwrng test more flexible * Add README.md * Make ce-oem-info jobs be attachment jobs (#58) ce-oem-info jobs are aimed at collecting the system bootup information from snapd and cloud-init to help developers analyze bugs. Therefore, these jobs are more suitable to be attachment jobs and should not fail even if some information can not be found. For example, some systems may not have cloud-init. In these systems, the job cannot gather the information from cloud-init, but this should not be a failure. * Modify the device tree dump job (#59) The current ce-oem-device-tree/dump job does not match the resource job format and will show warnings when running it. The original plan is to parse this job to match the resource format to let any other jobs able to get the dts info conveniently. As parse the dts to resource job format is a huge work, and will not be done in the near future. This commit excludes the ce-oem-device-tree/dump job from the test plan and add comments to explain the purpose. Also keep the attachment job to make sure we could get the device tree information when checking the report. * Revamp caam_hwrng_test by looping w/ smaller size (#60) Some of the project devices' caam_hwrng have buffer, so previously the dd size is increased to 10M to exceed the buffer size to make the interrupt increase. It will make the device without buffer take a very long time (over 30 mins) to do the dd command. Therefore, this commit separated the size to 512K and looped it 20 times. On the device without buffer it could do 512K and detect the increased interrupt to get the pass result, which is much faster. On the device with buttfer it could also exceed the buffer by dd 20 times and detect the increased interrupt to get the pass result. * Fix the nested crypto test plan name (#61) Previously the caam test plan was revamped to accelerator test plan, but forgot to update the name nested in ce-oem-automated. Fix the name in this commit. * Add: new socketcan stress test (#62) * Add: new socketcan tests add new socketcan stress test (100000 loop echo test) add bus-off test and fixed bug * Add: include new socketcan tests (#63) include new socketecan test plan into ce-oem test plans * Add ubuntu-frame and glmark2 tests (#64) * Add ubuntu-frame and glmark2 tests * Modify the way checking ubuntu-frame active * Modify: Add more details in README * Add: implement the RPMSG-tty tests implement the RPMSG-tty tests for i.MX series * Fix: update tox configuration update tox configuration to perform tests with specific Python version * Add: check VPU device tests (#68) * Add: check VPU device tests adding new test cases about identify VPU devices * Fix: fixed the issues from comments fixed the issues from comments * Fix: bug fixed bug fixed * Fix: fixed issues fixed issues * Fix: refactor the thermal tests (#69) * Fix: refactor the thermal tests refactor the thermal tests * Fix: correct the argument name in socketcan_test (#71) corrected the argument name in socketcan_test.py * Add dbus reboot stress (#70) * Add script for installing test snap and connect interfaces * Add reboot via dbus command This test is for snap strict confinement mode * Add tcp test (#65) * Add TCP test * Move ce-oem README inside the provider directory * Change the ce-oem provider namespace to the contrib namespace * Add README for the contrib area * Remove .gitignore from contrib/ Artefact left from the ce-oem provider migration --------- Co-authored-by: stanley31huang <stanley.huang@canonical.com> Co-authored-by: rickwu666666 <98441647+rickwu666666@users.noreply.github.com> Co-authored-by: Nara Huang <nara.huang@canonical.com> Co-authored-by: rickwu4444 <rick.wu@canonical.com> Co-authored-by: baconyao <patrick.chang@canonical.com> Co-authored-by: liaou3 <vincent.liao@canonical.com> Co-authored-by: patliuu <111331153+patliuu@users.noreply.github.com> Co-authored-by: Patrick Liu <patrick.liu@canonical.com> Co-authored-by: hanhsuan <32028620+hanhsuan@users.noreply.github.com>
LiaoU3
added a commit
to LiaoU3/checkbox
that referenced
this issue
Mar 20, 2024
…al#932) * Initial commit * Migrate ARM generic jobs from launchpad * Initial commit * Clean up this repo, and up it to date. * Change: job id for cpu cases * Change: id of rpmsg * Add: manage.py Since we need this repo as a provider * Change: separate jobs into generic and non-generic * Add: checkbox-provider-arm folder Following the structure of checkbox * Change: the access of tool not created by us To prevent those tools been packaged into checkbox * Add: test plan for ARM generic jobs * Change: test method for NPU TensorFlow Change the test method by using test snap build by Isaac * Change: MTD test method Since we don't have reliable way to indentify the type of MTD. * Change: test method for buzzer Change the test method for buzzer that probe as an input PC speaker * Change: chunkfs test method * Add: dump DTB file * Add: manifest for the jobs * Add: rs485 remote test * Remove: non-generic jobs Since we need to move generic parts to github Co-authored-by: Nara Huang <nara.huang@canonical.com> * Add snapcraft yaml for uc20 (canonical#2) * Add: snapcraft.yaml for uc20 * Add: README.md * Fix: export PYTHONPATH in override-build section * Add: libc6-dev for build-packages of input-pcspkr Since snapcraft7.x will need extra lib from libc6-dev * Fix cpu and device tree cases (canonical#3) * Fix: cpu offlining test * Change: requires of cpu offlineing test * Add: stage-package linuxptp * Fix: test plan for dtb * Add eeprom test and change name of the provider (canonical#6) * Add: eeprom test cases * Change: provider name from arm to ce-oem * Change name space and README.md (canonical#9) * Change: name space * Change: README.md * Fix: wording of checkbox interface snap * Remove: snapcraft.yaml (canonical#8) * Add: led generic test (canonical#7) * Add: support cold reboot tests by pdu (canonical#10) * Add: support cold reboot tests by pdu add cold reboot tests by network PDU for those system which not able to wakeup by rtc * Fix: fix bugs add following changes in this commit - check supported pdu type inside of main function - sync data after dump device information - correct command for init-boot-loop-data job * Fix: fix bug - fix the error reported by shellcheck * Change prefixes (canonical#11) * Change: prefixes for plans and jobs * Change: rename test plan and add PDU related plan * Change: name of category * Fix: missing prefixes of serial test job and plan * Add: informations about used config in readme (canonical#13) * Change: sysfs led method (canonical#12) * Fix: rtc wakealarm checking to right attribute (canonical#14) * Fix: remove ce-oem-power-automated-by-pdu Remove the ce-oem-power-automated-by-pdu from the ce-oem-automated The ce-oem-power-automated-by-pud will always be executed even the user not intend to reboot the DUT by PDU. And the checkbox session will stuck after send out a snmpset command. So remove this test first, and we need to integrate the cold boot related tests to PPC to support cold reboot test for those DUT which not able to wake up by rtc. * Add: add a generic iot test plans Adding a generic IoT test plans, it's a full testing set include tests in certification test plan and ce-oem-generic test plan * Improve the MTD script Before doing the read and write testing of specific mtd partition, we need to know the MTD is read-only or writable, therefore, a new variable called is_mtd_writable to help us to know it. If this mtd is read-only, I only read its content to a file. If this mtd is writable, backup the its original content to a file, then do the compare procedure, finally, recover its content. * Add: CAAM related test (canonical#21) * Add: CAAM related test * Add: generic crypto test * Fix naming with ce-oem * Add: add digital-io tests (canonical#24) * Add: add digital-io tests add digital-io tests * Fix: fixed scripts and test plan - fixed the format string style - fixed the typo in docstring - fixed the variable type for TEST_STATES - remove useless test jobs * Fix: fixed typo fixed typo * Add: add readme (canonical#23) * Fix: updated README.md add a section about how to use checkbox-ce-oem as a checkbox launcher * Fix: updated the installation steps updated the installation steps for Ubuntu classic and Ubuntu core * fix: fixed typo fixed typo * Fix: updated digital-io tests (canonical#25) - corrected digital-io tests in post-suspend stage - import manifest job * Add: cpufreq governors test for 6 governors (canonical#19) * Add: cpufreq governors test for 6 governors Add a resource job to check system supported governors, and test jobs for all 6 governors. These tests are only applicable to the platform using acpi_cpufreq related drivers. Newer intel CPU using intel_pstate driver is NOT applicable. * Fix: typo and apply suggestions * Remove: ce-oem-cpu/offlining_test from the plan Since there is already a fix for cpu/offlining_test in checkbox base provider, this ce-oem-cpu/offlining_test is no longer required. canonical#522 * Modify CAAM and installation time related jobs. (canonical#26) * Modify: modify hwrng and caam algo check to require manifest * Add: requires for installation related jobs Make those jobs run only in ubuntu core * Fix: revert summary section for snapd_installation_time The previous commit accidentally removed the section * Fix script for pep8 check (canonical#27) * Remove: test job/script that not in use. Since we have new script for cpu freq governors. And also cpu_offline test has been updated in base provider. * Fix: python script to fit flake8 * Fix: manifest shouldn't place in resource job Move the manifest to test job. To prevent manifest being executed at bootstrap stage. * Add code check by tox just like checkbox did (canonical#28) * Add code check by tox just like checkbox did This commit following what checkbox did, and add pep8-naming check for python 3.8+. * Change the fork checkbox git repo url to offical one * Fix tox test environment error (canonical#32) * Fix tox test environment error Each tox test environment won't be reseted, therefore removing all providers before test is needed. After checking with checkbox repo, checkbox-ng has to use different install method to install. * Follow checkbox repo to update pyYAML version to 6.0.1 for python 3.10+ to solve tox issue * Fix for shellcheck (canonical#29) * Fix: shell scrip to fit shellcheck * Fix: audio jobs to fit shellcheck * Fix: caam jobs to fit shellcheck * Fix: device jobs to fit shellcheck * Fix: digital-io jobs to fit shellcheck * Change: nested EEPROM test from checkbox base Since EEPROM jobs merged back to the base provider already. * Fix: power-manager jobs to fit shellcheck * Change: nested RTC test from checkbox base Since RTC jobs merged back to the base provider already. * Fix: stress jobs to fit shellcheck * Fix: theraml-sensor jobs to fit shellcheck * Fix: Nested plan to ce-oem Follow the structure as a client-cert plan * Add multiple usb otg test (canonical#33) * Remove: OTG related jobs Those OTG jobs have been migrated back to base provider. * Add: multiple OTG test * Add: readme.md * Refactor cpufreq governor test (canonical#36) * Change: refactor cpufreq governor test CQT-2906 The original tests did not handle the following 2 conditions: 1. CPUfreq scaling driver not found (no files under cpufreq folder) 2. On ARM device, system using multiple policies to handle different CPUs group This commit refactor the tests to make them policies-based tests. Also adding a driver detect job that can be use on all platforms. * Add optee test (canonical#34) * Add: optee test * Add: README.md * Include led test from base (canonical#35) * Add: nested led job from checkbox-base * Change: README.md for led * Change: PTP command (canonical#37) * Change: PTP command lp:2030942, workaround for the current version of ptp4l * Fix: apply musical scales for pwm buzzer test (canonical#40) * Fix: apply musical scales for pwm buzzer test apply musical scales for pwm buzzer test * Fix: added description for test_pwm_buzzer func added description for test_pwm_buzzer function * Fix: add touchscreen tests (canonical#39) * Fix: add touchscreen tests validate the required touch events for touchscreen also add new test to cover maximum fingers support on touchscreen * Change: hardware PTP support keywords For PTP capabilities, ethtool v5.4 (on Ubuntu 20.04) shows `hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE)` But ethtool v5.16 (on Ubuntu 22.04) only shows `hardware-transmit` The test case was using `SOF_TIMESTAMPING_TX_HARDWARE` as keywords, This commit changes it to use `hardware-transmit` for adapting new version of ethtool. * Remove: nested ptp-manual test plans The current manual test job steps are duplicated with the auto test job. Currently we only need to run the auto test job. The manual test steps here is used as a reference. If there are new jobs in th future, the ptp-manual test plan can be nested back. * Add: LD_PATH information in readme.md (canonical#41) * Change: additional patterns to map UDC and USB node (canonical#42) * Change: additional patterns to map UDC and USB node * Fix: the if else logic for map UDC * Fix: include python3-systemd in tox.ini (canonical#48) include the python3-systemd package in tox.ini * Fix: updated the related packages for tox (canonical#50) updated the related packages for tox * Fix: nested plan for server manual (canonical#44) * Fix: nested plan for server manual * Fix: errors when mtd count is 0 (canonical#55) * Fix: errors when mtd count is 0 * Add shellcheck ignore 2126 * Add: rpmsg tests for i.MX and TI processors (canonical#46) * Fix: updated the related packages for tox updated the related packages for tox * Add: rpmsg tests for i.MX and TI processors adding rpmsg tests for i.MX and TI processors * Fix: updated rpmsg_tests.py scripts updated rpmsg_tests.py scripts * Fix: fixed bugs fixed bugs * Modify optee test run out of gadget (canonical#45) * Add: support outsourcing optee-test * 230921 crypto test (canonical#52) * Fix: updated the related packages for tox updated the related packages for tox * Fix: update the crypto tests seperate the crypto tests to the generic crypto tests and crypto accelerator tests * Fix: bypass if failed to set extra seed the extra seed might not be supported for all platform * Fix: fixed bugs fixed bugs * Fix: refactor check_crypto_driver_priority func refactor check_crypto_driver_priority function * Fix: mock systemd-python package and remove packages in tox mock systemd-python package and remove packages in tox * Revamp otg_ports resource job should not fail (canonical#53) * Revamp otg_ports resource job should not fail Add a if condition to handle when the OTG config variable is not set. * Revise the echo string * Add: cupfreq module probe (canonical#51) * Add: cupfreq module probe * Check hwrng test (canonical#57) * Modify hwrng test more flexible * Add README.md * Make ce-oem-info jobs be attachment jobs (canonical#58) ce-oem-info jobs are aimed at collecting the system bootup information from snapd and cloud-init to help developers analyze bugs. Therefore, these jobs are more suitable to be attachment jobs and should not fail even if some information can not be found. For example, some systems may not have cloud-init. In these systems, the job cannot gather the information from cloud-init, but this should not be a failure. * Modify the device tree dump job (canonical#59) The current ce-oem-device-tree/dump job does not match the resource job format and will show warnings when running it. The original plan is to parse this job to match the resource format to let any other jobs able to get the dts info conveniently. As parse the dts to resource job format is a huge work, and will not be done in the near future. This commit excludes the ce-oem-device-tree/dump job from the test plan and add comments to explain the purpose. Also keep the attachment job to make sure we could get the device tree information when checking the report. * Revamp caam_hwrng_test by looping w/ smaller size (canonical#60) Some of the project devices' caam_hwrng have buffer, so previously the dd size is increased to 10M to exceed the buffer size to make the interrupt increase. It will make the device without buffer take a very long time (over 30 mins) to do the dd command. Therefore, this commit separated the size to 512K and looped it 20 times. On the device without buffer it could do 512K and detect the increased interrupt to get the pass result, which is much faster. On the device with buttfer it could also exceed the buffer by dd 20 times and detect the increased interrupt to get the pass result. * Fix the nested crypto test plan name (canonical#61) Previously the caam test plan was revamped to accelerator test plan, but forgot to update the name nested in ce-oem-automated. Fix the name in this commit. * Add: new socketcan stress test (canonical#62) * Add: new socketcan tests add new socketcan stress test (100000 loop echo test) add bus-off test and fixed bug * Add: include new socketcan tests (canonical#63) include new socketecan test plan into ce-oem test plans * Add ubuntu-frame and glmark2 tests (canonical#64) * Add ubuntu-frame and glmark2 tests * Modify the way checking ubuntu-frame active * Modify: Add more details in README * Add: implement the RPMSG-tty tests implement the RPMSG-tty tests for i.MX series * Fix: update tox configuration update tox configuration to perform tests with specific Python version * Add: check VPU device tests (canonical#68) * Add: check VPU device tests adding new test cases about identify VPU devices * Fix: fixed the issues from comments fixed the issues from comments * Fix: bug fixed bug fixed * Fix: fixed issues fixed issues * Fix: refactor the thermal tests (canonical#69) * Fix: refactor the thermal tests refactor the thermal tests * Fix: correct the argument name in socketcan_test (canonical#71) corrected the argument name in socketcan_test.py * Add dbus reboot stress (canonical#70) * Add script for installing test snap and connect interfaces * Add reboot via dbus command This test is for snap strict confinement mode * Add tcp test (canonical#65) * Add TCP test * Move ce-oem README inside the provider directory * Change the ce-oem provider namespace to the contrib namespace * Add README for the contrib area * Remove .gitignore from contrib/ Artefact left from the ce-oem provider migration --------- Co-authored-by: stanley31huang <stanley.huang@canonical.com> Co-authored-by: rickwu666666 <98441647+rickwu666666@users.noreply.github.com> Co-authored-by: Nara Huang <nara.huang@canonical.com> Co-authored-by: rickwu4444 <rick.wu@canonical.com> Co-authored-by: baconyao <patrick.chang@canonical.com> Co-authored-by: liaou3 <vincent.liao@canonical.com> Co-authored-by: patliuu <111331153+patliuu@users.noreply.github.com> Co-authored-by: Patrick Liu <patrick.liu@canonical.com> Co-authored-by: hanhsuan <32028620+hanhsuan@users.noreply.github.com>
binli
pushed a commit
to binli/checkbox
that referenced
this issue
Mar 22, 2024
…al#932) * Initial commit * Migrate ARM generic jobs from launchpad * Initial commit * Clean up this repo, and up it to date. * Change: job id for cpu cases * Change: id of rpmsg * Add: manage.py Since we need this repo as a provider * Change: separate jobs into generic and non-generic * Add: checkbox-provider-arm folder Following the structure of checkbox * Change: the access of tool not created by us To prevent those tools been packaged into checkbox * Add: test plan for ARM generic jobs * Change: test method for NPU TensorFlow Change the test method by using test snap build by Isaac * Change: MTD test method Since we don't have reliable way to indentify the type of MTD. * Change: test method for buzzer Change the test method for buzzer that probe as an input PC speaker * Change: chunkfs test method * Add: dump DTB file * Add: manifest for the jobs * Add: rs485 remote test * Remove: non-generic jobs Since we need to move generic parts to github Co-authored-by: Nara Huang <nara.huang@canonical.com> * Add snapcraft yaml for uc20 (canonical#2) * Add: snapcraft.yaml for uc20 * Add: README.md * Fix: export PYTHONPATH in override-build section * Add: libc6-dev for build-packages of input-pcspkr Since snapcraft7.x will need extra lib from libc6-dev * Fix cpu and device tree cases (canonical#3) * Fix: cpu offlining test * Change: requires of cpu offlineing test * Add: stage-package linuxptp * Fix: test plan for dtb * Add eeprom test and change name of the provider (canonical#6) * Add: eeprom test cases * Change: provider name from arm to ce-oem * Change name space and README.md (canonical#9) * Change: name space * Change: README.md * Fix: wording of checkbox interface snap * Remove: snapcraft.yaml (canonical#8) * Add: led generic test (canonical#7) * Add: support cold reboot tests by pdu (canonical#10) * Add: support cold reboot tests by pdu add cold reboot tests by network PDU for those system which not able to wakeup by rtc * Fix: fix bugs add following changes in this commit - check supported pdu type inside of main function - sync data after dump device information - correct command for init-boot-loop-data job * Fix: fix bug - fix the error reported by shellcheck * Change prefixes (canonical#11) * Change: prefixes for plans and jobs * Change: rename test plan and add PDU related plan * Change: name of category * Fix: missing prefixes of serial test job and plan * Add: informations about used config in readme (canonical#13) * Change: sysfs led method (canonical#12) * Fix: rtc wakealarm checking to right attribute (canonical#14) * Fix: remove ce-oem-power-automated-by-pdu Remove the ce-oem-power-automated-by-pdu from the ce-oem-automated The ce-oem-power-automated-by-pud will always be executed even the user not intend to reboot the DUT by PDU. And the checkbox session will stuck after send out a snmpset command. So remove this test first, and we need to integrate the cold boot related tests to PPC to support cold reboot test for those DUT which not able to wake up by rtc. * Add: add a generic iot test plans Adding a generic IoT test plans, it's a full testing set include tests in certification test plan and ce-oem-generic test plan * Improve the MTD script Before doing the read and write testing of specific mtd partition, we need to know the MTD is read-only or writable, therefore, a new variable called is_mtd_writable to help us to know it. If this mtd is read-only, I only read its content to a file. If this mtd is writable, backup the its original content to a file, then do the compare procedure, finally, recover its content. * Add: CAAM related test (canonical#21) * Add: CAAM related test * Add: generic crypto test * Fix naming with ce-oem * Add: add digital-io tests (canonical#24) * Add: add digital-io tests add digital-io tests * Fix: fixed scripts and test plan - fixed the format string style - fixed the typo in docstring - fixed the variable type for TEST_STATES - remove useless test jobs * Fix: fixed typo fixed typo * Add: add readme (canonical#23) * Fix: updated README.md add a section about how to use checkbox-ce-oem as a checkbox launcher * Fix: updated the installation steps updated the installation steps for Ubuntu classic and Ubuntu core * fix: fixed typo fixed typo * Fix: updated digital-io tests (canonical#25) - corrected digital-io tests in post-suspend stage - import manifest job * Add: cpufreq governors test for 6 governors (canonical#19) * Add: cpufreq governors test for 6 governors Add a resource job to check system supported governors, and test jobs for all 6 governors. These tests are only applicable to the platform using acpi_cpufreq related drivers. Newer intel CPU using intel_pstate driver is NOT applicable. * Fix: typo and apply suggestions * Remove: ce-oem-cpu/offlining_test from the plan Since there is already a fix for cpu/offlining_test in checkbox base provider, this ce-oem-cpu/offlining_test is no longer required. canonical#522 * Modify CAAM and installation time related jobs. (canonical#26) * Modify: modify hwrng and caam algo check to require manifest * Add: requires for installation related jobs Make those jobs run only in ubuntu core * Fix: revert summary section for snapd_installation_time The previous commit accidentally removed the section * Fix script for pep8 check (canonical#27) * Remove: test job/script that not in use. Since we have new script for cpu freq governors. And also cpu_offline test has been updated in base provider. * Fix: python script to fit flake8 * Fix: manifest shouldn't place in resource job Move the manifest to test job. To prevent manifest being executed at bootstrap stage. * Add code check by tox just like checkbox did (canonical#28) * Add code check by tox just like checkbox did This commit following what checkbox did, and add pep8-naming check for python 3.8+. * Change the fork checkbox git repo url to offical one * Fix tox test environment error (canonical#32) * Fix tox test environment error Each tox test environment won't be reseted, therefore removing all providers before test is needed. After checking with checkbox repo, checkbox-ng has to use different install method to install. * Follow checkbox repo to update pyYAML version to 6.0.1 for python 3.10+ to solve tox issue * Fix for shellcheck (canonical#29) * Fix: shell scrip to fit shellcheck * Fix: audio jobs to fit shellcheck * Fix: caam jobs to fit shellcheck * Fix: device jobs to fit shellcheck * Fix: digital-io jobs to fit shellcheck * Change: nested EEPROM test from checkbox base Since EEPROM jobs merged back to the base provider already. * Fix: power-manager jobs to fit shellcheck * Change: nested RTC test from checkbox base Since RTC jobs merged back to the base provider already. * Fix: stress jobs to fit shellcheck * Fix: theraml-sensor jobs to fit shellcheck * Fix: Nested plan to ce-oem Follow the structure as a client-cert plan * Add multiple usb otg test (canonical#33) * Remove: OTG related jobs Those OTG jobs have been migrated back to base provider. * Add: multiple OTG test * Add: readme.md * Refactor cpufreq governor test (canonical#36) * Change: refactor cpufreq governor test CQT-2906 The original tests did not handle the following 2 conditions: 1. CPUfreq scaling driver not found (no files under cpufreq folder) 2. On ARM device, system using multiple policies to handle different CPUs group This commit refactor the tests to make them policies-based tests. Also adding a driver detect job that can be use on all platforms. * Add optee test (canonical#34) * Add: optee test * Add: README.md * Include led test from base (canonical#35) * Add: nested led job from checkbox-base * Change: README.md for led * Change: PTP command (canonical#37) * Change: PTP command lp:2030942, workaround for the current version of ptp4l * Fix: apply musical scales for pwm buzzer test (canonical#40) * Fix: apply musical scales for pwm buzzer test apply musical scales for pwm buzzer test * Fix: added description for test_pwm_buzzer func added description for test_pwm_buzzer function * Fix: add touchscreen tests (canonical#39) * Fix: add touchscreen tests validate the required touch events for touchscreen also add new test to cover maximum fingers support on touchscreen * Change: hardware PTP support keywords For PTP capabilities, ethtool v5.4 (on Ubuntu 20.04) shows `hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE)` But ethtool v5.16 (on Ubuntu 22.04) only shows `hardware-transmit` The test case was using `SOF_TIMESTAMPING_TX_HARDWARE` as keywords, This commit changes it to use `hardware-transmit` for adapting new version of ethtool. * Remove: nested ptp-manual test plans The current manual test job steps are duplicated with the auto test job. Currently we only need to run the auto test job. The manual test steps here is used as a reference. If there are new jobs in th future, the ptp-manual test plan can be nested back. * Add: LD_PATH information in readme.md (canonical#41) * Change: additional patterns to map UDC and USB node (canonical#42) * Change: additional patterns to map UDC and USB node * Fix: the if else logic for map UDC * Fix: include python3-systemd in tox.ini (canonical#48) include the python3-systemd package in tox.ini * Fix: updated the related packages for tox (canonical#50) updated the related packages for tox * Fix: nested plan for server manual (canonical#44) * Fix: nested plan for server manual * Fix: errors when mtd count is 0 (canonical#55) * Fix: errors when mtd count is 0 * Add shellcheck ignore 2126 * Add: rpmsg tests for i.MX and TI processors (canonical#46) * Fix: updated the related packages for tox updated the related packages for tox * Add: rpmsg tests for i.MX and TI processors adding rpmsg tests for i.MX and TI processors * Fix: updated rpmsg_tests.py scripts updated rpmsg_tests.py scripts * Fix: fixed bugs fixed bugs * Modify optee test run out of gadget (canonical#45) * Add: support outsourcing optee-test * 230921 crypto test (canonical#52) * Fix: updated the related packages for tox updated the related packages for tox * Fix: update the crypto tests seperate the crypto tests to the generic crypto tests and crypto accelerator tests * Fix: bypass if failed to set extra seed the extra seed might not be supported for all platform * Fix: fixed bugs fixed bugs * Fix: refactor check_crypto_driver_priority func refactor check_crypto_driver_priority function * Fix: mock systemd-python package and remove packages in tox mock systemd-python package and remove packages in tox * Revamp otg_ports resource job should not fail (canonical#53) * Revamp otg_ports resource job should not fail Add a if condition to handle when the OTG config variable is not set. * Revise the echo string * Add: cupfreq module probe (canonical#51) * Add: cupfreq module probe * Check hwrng test (canonical#57) * Modify hwrng test more flexible * Add README.md * Make ce-oem-info jobs be attachment jobs (canonical#58) ce-oem-info jobs are aimed at collecting the system bootup information from snapd and cloud-init to help developers analyze bugs. Therefore, these jobs are more suitable to be attachment jobs and should not fail even if some information can not be found. For example, some systems may not have cloud-init. In these systems, the job cannot gather the information from cloud-init, but this should not be a failure. * Modify the device tree dump job (canonical#59) The current ce-oem-device-tree/dump job does not match the resource job format and will show warnings when running it. The original plan is to parse this job to match the resource format to let any other jobs able to get the dts info conveniently. As parse the dts to resource job format is a huge work, and will not be done in the near future. This commit excludes the ce-oem-device-tree/dump job from the test plan and add comments to explain the purpose. Also keep the attachment job to make sure we could get the device tree information when checking the report. * Revamp caam_hwrng_test by looping w/ smaller size (canonical#60) Some of the project devices' caam_hwrng have buffer, so previously the dd size is increased to 10M to exceed the buffer size to make the interrupt increase. It will make the device without buffer take a very long time (over 30 mins) to do the dd command. Therefore, this commit separated the size to 512K and looped it 20 times. On the device without buffer it could do 512K and detect the increased interrupt to get the pass result, which is much faster. On the device with buttfer it could also exceed the buffer by dd 20 times and detect the increased interrupt to get the pass result. * Fix the nested crypto test plan name (canonical#61) Previously the caam test plan was revamped to accelerator test plan, but forgot to update the name nested in ce-oem-automated. Fix the name in this commit. * Add: new socketcan stress test (canonical#62) * Add: new socketcan tests add new socketcan stress test (100000 loop echo test) add bus-off test and fixed bug * Add: include new socketcan tests (canonical#63) include new socketecan test plan into ce-oem test plans * Add ubuntu-frame and glmark2 tests (canonical#64) * Add ubuntu-frame and glmark2 tests * Modify the way checking ubuntu-frame active * Modify: Add more details in README * Add: implement the RPMSG-tty tests implement the RPMSG-tty tests for i.MX series * Fix: update tox configuration update tox configuration to perform tests with specific Python version * Add: check VPU device tests (canonical#68) * Add: check VPU device tests adding new test cases about identify VPU devices * Fix: fixed the issues from comments fixed the issues from comments * Fix: bug fixed bug fixed * Fix: fixed issues fixed issues * Fix: refactor the thermal tests (canonical#69) * Fix: refactor the thermal tests refactor the thermal tests * Fix: correct the argument name in socketcan_test (canonical#71) corrected the argument name in socketcan_test.py * Add dbus reboot stress (canonical#70) * Add script for installing test snap and connect interfaces * Add reboot via dbus command This test is for snap strict confinement mode * Add tcp test (canonical#65) * Add TCP test * Move ce-oem README inside the provider directory * Change the ce-oem provider namespace to the contrib namespace * Add README for the contrib area * Remove .gitignore from contrib/ Artefact left from the ce-oem provider migration --------- Co-authored-by: stanley31huang <stanley.huang@canonical.com> Co-authored-by: rickwu666666 <98441647+rickwu666666@users.noreply.github.com> Co-authored-by: Nara Huang <nara.huang@canonical.com> Co-authored-by: rickwu4444 <rick.wu@canonical.com> Co-authored-by: baconyao <patrick.chang@canonical.com> Co-authored-by: liaou3 <vincent.liao@canonical.com> Co-authored-by: patliuu <111331153+patliuu@users.noreply.github.com> Co-authored-by: Patrick Liu <patrick.liu@canonical.com> Co-authored-by: hanhsuan <32028620+hanhsuan@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This issue was migrated from https://bugs.launchpad.net/plainbox-provider-checkbox/+bug/1712301
Summary
Description
We do have S3 single test for discrete graphic (Nvidia graphic) but no stress test. It would be better to have S3 stress for Nvidia and AMD dGPUs as it is cert blocker.
How to switch graphic
To switch to Nvidia card:
To switch to Intel card:
sudo prime-select intel
For amdgpu-pro stack, it is also switchable (amdgpu is not). We need S3 stress test for amdgpu-pro stack as well. Checkbox already has a way to identify amdgpu and amdgpu-pro.
To switch graphic mode on amdgpu-pro stack
sudo /opt/amdgpu-pro/bin/amdgpu-pro-px --mode performance/powersaving
The idea is to have S3 stress test for discrete graphic in stress test plan and can be completed without interaction.
The critical part is system needs reboot after switching graphic mode (performance-discrete and power saving-UMA).
Attachments
/usr/share/plainbox-provider-checkbox/units/graphics/test-plan.pxu
Tags:
['ce-qa-concern']
The text was updated successfully, but these errors were encountered: