Add checkbox snaps code #3
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Release_2019_Week28
The core checkbox snap is also used by snaps for classic image testing. Some extra variables are required and importantly there is no content inteface available so the PROVIDERPATH must be used instead.
checkbox-ng has a dependency on sudo being available when there is need to execute a test as a target user. This makes sure sudo is available and a version that matches other libs in the snap.
Release_2019_Week32
Another extra package need for sudo functionality
Provides dbus-send used by warm-reboot test
The jobs previously contained in plainbox-provider-snappy have moved to plainbox-provider-checkbox. This moves all the build and runtime dependencies over the checkbox provider following the tests.
As of bionic, all i2c tools depends on https://packages.ubuntu.com/bionic/libi2c0 Only priming i2cdetect resulys in: i2cdetect: error while loading shared libraries: libi2c.so.0: cannot open shared object file: No such file or directory
Release_2019_Week37
Release_2019_Week41
Run in read mode it will dump the content of the environment. Taken from https://git.launchpad.net/~ondrak/+git/snap-bootsel-env/tree/snap/snapcraft.yaml
Release_2019_Week45
Release_2022_Week23
Fixes a couple of "bad yaml" errors issued by snapcraft 7.x although at this point user-defined plugins are not supported
These previously resided in the CDTS snap, however there was little value in maintaining two top level snaps.
Release_2022_Week33
|
checkbox-ng-packaging and checkbox-support-packaging are not needed. The debian/ folders for those two are already included (see checkbox-ng/debian and checkbox-support/debian) |
|
Yeah, too many parents 🥲 |
This was referenced Nov 28, 2022
hector-cao
pushed a commit
to hector-cao/checkbox
that referenced
this pull request
Mar 2, 2023
Add checkbox snaps code
kissiel
pushed a commit
that referenced
this pull request
May 1, 2023
* Replace orchestration delays with context-aware timers - dial relevant ports via `snap_wait_port_status()` - check http status codes via `snap_wait_all_services_online()` - enable test that was skipped due to enabling/disabling bug: edgexfoundry/edgex-go#3820 Signed-off-by: Mengyi Wang <mengyi.wang@canonical.com>
pieqq
pushed a commit
that referenced
this pull request
Jan 12, 2024
* Fix: cpu offlining test * Change: requires of cpu offlineing test
pieqq
pushed a commit
that referenced
this pull request
Jan 12, 2024
* Fix: cpu offlining test * Change: requires of cpu offlineing test
kissiel
pushed a commit
that referenced
this pull request
Jan 12, 2024
* Fix: cpu offlining test * Change: requires of cpu offlineing test
kissiel
pushed a commit
that referenced
this pull request
Jan 12, 2024
* Initial commit * Migrate ARM generic jobs from launchpad * Initial commit * Clean up this repo, and up it to date. * Change: job id for cpu cases * Change: id of rpmsg * Add: manage.py Since we need this repo as a provider * Change: separate jobs into generic and non-generic * Add: checkbox-provider-arm folder Following the structure of checkbox * Change: the access of tool not created by us To prevent those tools been packaged into checkbox * Add: test plan for ARM generic jobs * Change: test method for NPU TensorFlow Change the test method by using test snap build by Isaac * Change: MTD test method Since we don't have reliable way to indentify the type of MTD. * Change: test method for buzzer Change the test method for buzzer that probe as an input PC speaker * Change: chunkfs test method * Add: dump DTB file * Add: manifest for the jobs * Add: rs485 remote test * Remove: non-generic jobs Since we need to move generic parts to github Co-authored-by: Nara Huang <nara.huang@canonical.com> * Add snapcraft yaml for uc20 (#2) * Add: snapcraft.yaml for uc20 * Add: README.md * Fix: export PYTHONPATH in override-build section * Add: libc6-dev for build-packages of input-pcspkr Since snapcraft7.x will need extra lib from libc6-dev * Fix cpu and device tree cases (#3) * Fix: cpu offlining test * Change: requires of cpu offlineing test * Add: stage-package linuxptp * Fix: test plan for dtb * Add eeprom test and change name of the provider (#6) * Add: eeprom test cases * Change: provider name from arm to ce-oem * Change name space and README.md (#9) * Change: name space * Change: README.md * Fix: wording of checkbox interface snap * Remove: snapcraft.yaml (#8) * Add: led generic test (#7) * Add: support cold reboot tests by pdu (#10) * Add: support cold reboot tests by pdu add cold reboot tests by network PDU for those system which not able to wakeup by rtc * Fix: fix bugs add following changes in this commit - check supported pdu type inside of main function - sync data after dump device information - correct command for init-boot-loop-data job * Fix: fix bug - fix the error reported by shellcheck * Change prefixes (#11) * Change: prefixes for plans and jobs * Change: rename test plan and add PDU related plan * Change: name of category * Fix: missing prefixes of serial test job and plan * Add: informations about used config in readme (#13) * Change: sysfs led method (#12) * Fix: rtc wakealarm checking to right attribute (#14) * Fix: remove ce-oem-power-automated-by-pdu Remove the ce-oem-power-automated-by-pdu from the ce-oem-automated The ce-oem-power-automated-by-pud will always be executed even the user not intend to reboot the DUT by PDU. And the checkbox session will stuck after send out a snmpset command. So remove this test first, and we need to integrate the cold boot related tests to PPC to support cold reboot test for those DUT which not able to wake up by rtc. * Add: add a generic iot test plans Adding a generic IoT test plans, it's a full testing set include tests in certification test plan and ce-oem-generic test plan * Improve the MTD script Before doing the read and write testing of specific mtd partition, we need to know the MTD is read-only or writable, therefore, a new variable called is_mtd_writable to help us to know it. If this mtd is read-only, I only read its content to a file. If this mtd is writable, backup the its original content to a file, then do the compare procedure, finally, recover its content. * Add: CAAM related test (#21) * Add: CAAM related test * Add: generic crypto test * Fix naming with ce-oem * Add: add digital-io tests (#24) * Add: add digital-io tests add digital-io tests * Fix: fixed scripts and test plan - fixed the format string style - fixed the typo in docstring - fixed the variable type for TEST_STATES - remove useless test jobs * Fix: fixed typo fixed typo * Add: add readme (#23) * Fix: updated README.md add a section about how to use checkbox-ce-oem as a checkbox launcher * Fix: updated the installation steps updated the installation steps for Ubuntu classic and Ubuntu core * fix: fixed typo fixed typo * Fix: updated digital-io tests (#25) - corrected digital-io tests in post-suspend stage - import manifest job * Add: cpufreq governors test for 6 governors (#19) * Add: cpufreq governors test for 6 governors Add a resource job to check system supported governors, and test jobs for all 6 governors. These tests are only applicable to the platform using acpi_cpufreq related drivers. Newer intel CPU using intel_pstate driver is NOT applicable. * Fix: typo and apply suggestions * Remove: ce-oem-cpu/offlining_test from the plan Since there is already a fix for cpu/offlining_test in checkbox base provider, this ce-oem-cpu/offlining_test is no longer required. #522 * Modify CAAM and installation time related jobs. (#26) * Modify: modify hwrng and caam algo check to require manifest * Add: requires for installation related jobs Make those jobs run only in ubuntu core * Fix: revert summary section for snapd_installation_time The previous commit accidentally removed the section * Fix script for pep8 check (#27) * Remove: test job/script that not in use. Since we have new script for cpu freq governors. And also cpu_offline test has been updated in base provider. * Fix: python script to fit flake8 * Fix: manifest shouldn't place in resource job Move the manifest to test job. To prevent manifest being executed at bootstrap stage. * Add code check by tox just like checkbox did (#28) * Add code check by tox just like checkbox did This commit following what checkbox did, and add pep8-naming check for python 3.8+. * Change the fork checkbox git repo url to offical one * Fix tox test environment error (#32) * Fix tox test environment error Each tox test environment won't be reseted, therefore removing all providers before test is needed. After checking with checkbox repo, checkbox-ng has to use different install method to install. * Follow checkbox repo to update pyYAML version to 6.0.1 for python 3.10+ to solve tox issue * Fix for shellcheck (#29) * Fix: shell scrip to fit shellcheck * Fix: audio jobs to fit shellcheck * Fix: caam jobs to fit shellcheck * Fix: device jobs to fit shellcheck * Fix: digital-io jobs to fit shellcheck * Change: nested EEPROM test from checkbox base Since EEPROM jobs merged back to the base provider already. * Fix: power-manager jobs to fit shellcheck * Change: nested RTC test from checkbox base Since RTC jobs merged back to the base provider already. * Fix: stress jobs to fit shellcheck * Fix: theraml-sensor jobs to fit shellcheck * Fix: Nested plan to ce-oem Follow the structure as a client-cert plan * Add multiple usb otg test (#33) * Remove: OTG related jobs Those OTG jobs have been migrated back to base provider. * Add: multiple OTG test * Add: readme.md * Refactor cpufreq governor test (#36) * Change: refactor cpufreq governor test CQT-2906 The original tests did not handle the following 2 conditions: 1. CPUfreq scaling driver not found (no files under cpufreq folder) 2. On ARM device, system using multiple policies to handle different CPUs group This commit refactor the tests to make them policies-based tests. Also adding a driver detect job that can be use on all platforms. * Add optee test (#34) * Add: optee test * Add: README.md * Include led test from base (#35) * Add: nested led job from checkbox-base * Change: README.md for led * Change: PTP command (#37) * Change: PTP command lp:2030942, workaround for the current version of ptp4l * Fix: apply musical scales for pwm buzzer test (#40) * Fix: apply musical scales for pwm buzzer test apply musical scales for pwm buzzer test * Fix: added description for test_pwm_buzzer func added description for test_pwm_buzzer function * Fix: add touchscreen tests (#39) * Fix: add touchscreen tests validate the required touch events for touchscreen also add new test to cover maximum fingers support on touchscreen * Change: hardware PTP support keywords For PTP capabilities, ethtool v5.4 (on Ubuntu 20.04) shows `hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE)` But ethtool v5.16 (on Ubuntu 22.04) only shows `hardware-transmit` The test case was using `SOF_TIMESTAMPING_TX_HARDWARE` as keywords, This commit changes it to use `hardware-transmit` for adapting new version of ethtool. * Remove: nested ptp-manual test plans The current manual test job steps are duplicated with the auto test job. Currently we only need to run the auto test job. The manual test steps here is used as a reference. If there are new jobs in th future, the ptp-manual test plan can be nested back. * Add: LD_PATH information in readme.md (#41) * Change: additional patterns to map UDC and USB node (#42) * Change: additional patterns to map UDC and USB node * Fix: the if else logic for map UDC * Fix: include python3-systemd in tox.ini (#48) include the python3-systemd package in tox.ini * Fix: updated the related packages for tox (#50) updated the related packages for tox * Fix: nested plan for server manual (#44) * Fix: nested plan for server manual * Fix: errors when mtd count is 0 (#55) * Fix: errors when mtd count is 0 * Add shellcheck ignore 2126 * Add: rpmsg tests for i.MX and TI processors (#46) * Fix: updated the related packages for tox updated the related packages for tox * Add: rpmsg tests for i.MX and TI processors adding rpmsg tests for i.MX and TI processors * Fix: updated rpmsg_tests.py scripts updated rpmsg_tests.py scripts * Fix: fixed bugs fixed bugs * Modify optee test run out of gadget (#45) * Add: support outsourcing optee-test * 230921 crypto test (#52) * Fix: updated the related packages for tox updated the related packages for tox * Fix: update the crypto tests seperate the crypto tests to the generic crypto tests and crypto accelerator tests * Fix: bypass if failed to set extra seed the extra seed might not be supported for all platform * Fix: fixed bugs fixed bugs * Fix: refactor check_crypto_driver_priority func refactor check_crypto_driver_priority function * Fix: mock systemd-python package and remove packages in tox mock systemd-python package and remove packages in tox * Revamp otg_ports resource job should not fail (#53) * Revamp otg_ports resource job should not fail Add a if condition to handle when the OTG config variable is not set. * Revise the echo string * Add: cupfreq module probe (#51) * Add: cupfreq module probe * Check hwrng test (#57) * Modify hwrng test more flexible * Add README.md * Make ce-oem-info jobs be attachment jobs (#58) ce-oem-info jobs are aimed at collecting the system bootup information from snapd and cloud-init to help developers analyze bugs. Therefore, these jobs are more suitable to be attachment jobs and should not fail even if some information can not be found. For example, some systems may not have cloud-init. In these systems, the job cannot gather the information from cloud-init, but this should not be a failure. * Modify the device tree dump job (#59) The current ce-oem-device-tree/dump job does not match the resource job format and will show warnings when running it. The original plan is to parse this job to match the resource format to let any other jobs able to get the dts info conveniently. As parse the dts to resource job format is a huge work, and will not be done in the near future. This commit excludes the ce-oem-device-tree/dump job from the test plan and add comments to explain the purpose. Also keep the attachment job to make sure we could get the device tree information when checking the report. * Revamp caam_hwrng_test by looping w/ smaller size (#60) Some of the project devices' caam_hwrng have buffer, so previously the dd size is increased to 10M to exceed the buffer size to make the interrupt increase. It will make the device without buffer take a very long time (over 30 mins) to do the dd command. Therefore, this commit separated the size to 512K and looped it 20 times. On the device without buffer it could do 512K and detect the increased interrupt to get the pass result, which is much faster. On the device with buttfer it could also exceed the buffer by dd 20 times and detect the increased interrupt to get the pass result. * Fix the nested crypto test plan name (#61) Previously the caam test plan was revamped to accelerator test plan, but forgot to update the name nested in ce-oem-automated. Fix the name in this commit. * Add: new socketcan stress test (#62) * Add: new socketcan tests add new socketcan stress test (100000 loop echo test) add bus-off test and fixed bug * Add: include new socketcan tests (#63) include new socketecan test plan into ce-oem test plans * Add ubuntu-frame and glmark2 tests (#64) * Add ubuntu-frame and glmark2 tests * Modify the way checking ubuntu-frame active * Modify: Add more details in README * Add: implement the RPMSG-tty tests implement the RPMSG-tty tests for i.MX series * Fix: update tox configuration update tox configuration to perform tests with specific Python version * Add: check VPU device tests (#68) * Add: check VPU device tests adding new test cases about identify VPU devices * Fix: fixed the issues from comments fixed the issues from comments * Fix: bug fixed bug fixed * Fix: fixed issues fixed issues * Fix: refactor the thermal tests (#69) * Fix: refactor the thermal tests refactor the thermal tests * Fix: correct the argument name in socketcan_test (#71) corrected the argument name in socketcan_test.py * Add dbus reboot stress (#70) * Add script for installing test snap and connect interfaces * Add reboot via dbus command This test is for snap strict confinement mode * Add tcp test (#65) * Add TCP test * Move ce-oem README inside the provider directory * Change the ce-oem provider namespace to the contrib namespace * Add README for the contrib area * Remove .gitignore from contrib/ Artefact left from the ce-oem provider migration --------- Co-authored-by: stanley31huang <stanley.huang@canonical.com> Co-authored-by: rickwu666666 <98441647+rickwu666666@users.noreply.github.com> Co-authored-by: Nara Huang <nara.huang@canonical.com> Co-authored-by: rickwu4444 <rick.wu@canonical.com> Co-authored-by: baconyao <patrick.chang@canonical.com> Co-authored-by: liaou3 <vincent.liao@canonical.com> Co-authored-by: patliuu <111331153+patliuu@users.noreply.github.com> Co-authored-by: Patrick Liu <patrick.liu@canonical.com> Co-authored-by: hanhsuan <32028620+hanhsuan@users.noreply.github.com>
LiaoU3
added a commit
to LiaoU3/checkbox
that referenced
this pull request
Mar 20, 2024
…al#932) * Initial commit * Migrate ARM generic jobs from launchpad * Initial commit * Clean up this repo, and up it to date. * Change: job id for cpu cases * Change: id of rpmsg * Add: manage.py Since we need this repo as a provider * Change: separate jobs into generic and non-generic * Add: checkbox-provider-arm folder Following the structure of checkbox * Change: the access of tool not created by us To prevent those tools been packaged into checkbox * Add: test plan for ARM generic jobs * Change: test method for NPU TensorFlow Change the test method by using test snap build by Isaac * Change: MTD test method Since we don't have reliable way to indentify the type of MTD. * Change: test method for buzzer Change the test method for buzzer that probe as an input PC speaker * Change: chunkfs test method * Add: dump DTB file * Add: manifest for the jobs * Add: rs485 remote test * Remove: non-generic jobs Since we need to move generic parts to github Co-authored-by: Nara Huang <nara.huang@canonical.com> * Add snapcraft yaml for uc20 (canonical#2) * Add: snapcraft.yaml for uc20 * Add: README.md * Fix: export PYTHONPATH in override-build section * Add: libc6-dev for build-packages of input-pcspkr Since snapcraft7.x will need extra lib from libc6-dev * Fix cpu and device tree cases (canonical#3) * Fix: cpu offlining test * Change: requires of cpu offlineing test * Add: stage-package linuxptp * Fix: test plan for dtb * Add eeprom test and change name of the provider (canonical#6) * Add: eeprom test cases * Change: provider name from arm to ce-oem * Change name space and README.md (canonical#9) * Change: name space * Change: README.md * Fix: wording of checkbox interface snap * Remove: snapcraft.yaml (canonical#8) * Add: led generic test (canonical#7) * Add: support cold reboot tests by pdu (canonical#10) * Add: support cold reboot tests by pdu add cold reboot tests by network PDU for those system which not able to wakeup by rtc * Fix: fix bugs add following changes in this commit - check supported pdu type inside of main function - sync data after dump device information - correct command for init-boot-loop-data job * Fix: fix bug - fix the error reported by shellcheck * Change prefixes (canonical#11) * Change: prefixes for plans and jobs * Change: rename test plan and add PDU related plan * Change: name of category * Fix: missing prefixes of serial test job and plan * Add: informations about used config in readme (canonical#13) * Change: sysfs led method (canonical#12) * Fix: rtc wakealarm checking to right attribute (canonical#14) * Fix: remove ce-oem-power-automated-by-pdu Remove the ce-oem-power-automated-by-pdu from the ce-oem-automated The ce-oem-power-automated-by-pud will always be executed even the user not intend to reboot the DUT by PDU. And the checkbox session will stuck after send out a snmpset command. So remove this test first, and we need to integrate the cold boot related tests to PPC to support cold reboot test for those DUT which not able to wake up by rtc. * Add: add a generic iot test plans Adding a generic IoT test plans, it's a full testing set include tests in certification test plan and ce-oem-generic test plan * Improve the MTD script Before doing the read and write testing of specific mtd partition, we need to know the MTD is read-only or writable, therefore, a new variable called is_mtd_writable to help us to know it. If this mtd is read-only, I only read its content to a file. If this mtd is writable, backup the its original content to a file, then do the compare procedure, finally, recover its content. * Add: CAAM related test (canonical#21) * Add: CAAM related test * Add: generic crypto test * Fix naming with ce-oem * Add: add digital-io tests (canonical#24) * Add: add digital-io tests add digital-io tests * Fix: fixed scripts and test plan - fixed the format string style - fixed the typo in docstring - fixed the variable type for TEST_STATES - remove useless test jobs * Fix: fixed typo fixed typo * Add: add readme (canonical#23) * Fix: updated README.md add a section about how to use checkbox-ce-oem as a checkbox launcher * Fix: updated the installation steps updated the installation steps for Ubuntu classic and Ubuntu core * fix: fixed typo fixed typo * Fix: updated digital-io tests (canonical#25) - corrected digital-io tests in post-suspend stage - import manifest job * Add: cpufreq governors test for 6 governors (canonical#19) * Add: cpufreq governors test for 6 governors Add a resource job to check system supported governors, and test jobs for all 6 governors. These tests are only applicable to the platform using acpi_cpufreq related drivers. Newer intel CPU using intel_pstate driver is NOT applicable. * Fix: typo and apply suggestions * Remove: ce-oem-cpu/offlining_test from the plan Since there is already a fix for cpu/offlining_test in checkbox base provider, this ce-oem-cpu/offlining_test is no longer required. canonical#522 * Modify CAAM and installation time related jobs. (canonical#26) * Modify: modify hwrng and caam algo check to require manifest * Add: requires for installation related jobs Make those jobs run only in ubuntu core * Fix: revert summary section for snapd_installation_time The previous commit accidentally removed the section * Fix script for pep8 check (canonical#27) * Remove: test job/script that not in use. Since we have new script for cpu freq governors. And also cpu_offline test has been updated in base provider. * Fix: python script to fit flake8 * Fix: manifest shouldn't place in resource job Move the manifest to test job. To prevent manifest being executed at bootstrap stage. * Add code check by tox just like checkbox did (canonical#28) * Add code check by tox just like checkbox did This commit following what checkbox did, and add pep8-naming check for python 3.8+. * Change the fork checkbox git repo url to offical one * Fix tox test environment error (canonical#32) * Fix tox test environment error Each tox test environment won't be reseted, therefore removing all providers before test is needed. After checking with checkbox repo, checkbox-ng has to use different install method to install. * Follow checkbox repo to update pyYAML version to 6.0.1 for python 3.10+ to solve tox issue * Fix for shellcheck (canonical#29) * Fix: shell scrip to fit shellcheck * Fix: audio jobs to fit shellcheck * Fix: caam jobs to fit shellcheck * Fix: device jobs to fit shellcheck * Fix: digital-io jobs to fit shellcheck * Change: nested EEPROM test from checkbox base Since EEPROM jobs merged back to the base provider already. * Fix: power-manager jobs to fit shellcheck * Change: nested RTC test from checkbox base Since RTC jobs merged back to the base provider already. * Fix: stress jobs to fit shellcheck * Fix: theraml-sensor jobs to fit shellcheck * Fix: Nested plan to ce-oem Follow the structure as a client-cert plan * Add multiple usb otg test (canonical#33) * Remove: OTG related jobs Those OTG jobs have been migrated back to base provider. * Add: multiple OTG test * Add: readme.md * Refactor cpufreq governor test (canonical#36) * Change: refactor cpufreq governor test CQT-2906 The original tests did not handle the following 2 conditions: 1. CPUfreq scaling driver not found (no files under cpufreq folder) 2. On ARM device, system using multiple policies to handle different CPUs group This commit refactor the tests to make them policies-based tests. Also adding a driver detect job that can be use on all platforms. * Add optee test (canonical#34) * Add: optee test * Add: README.md * Include led test from base (canonical#35) * Add: nested led job from checkbox-base * Change: README.md for led * Change: PTP command (canonical#37) * Change: PTP command lp:2030942, workaround for the current version of ptp4l * Fix: apply musical scales for pwm buzzer test (canonical#40) * Fix: apply musical scales for pwm buzzer test apply musical scales for pwm buzzer test * Fix: added description for test_pwm_buzzer func added description for test_pwm_buzzer function * Fix: add touchscreen tests (canonical#39) * Fix: add touchscreen tests validate the required touch events for touchscreen also add new test to cover maximum fingers support on touchscreen * Change: hardware PTP support keywords For PTP capabilities, ethtool v5.4 (on Ubuntu 20.04) shows `hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE)` But ethtool v5.16 (on Ubuntu 22.04) only shows `hardware-transmit` The test case was using `SOF_TIMESTAMPING_TX_HARDWARE` as keywords, This commit changes it to use `hardware-transmit` for adapting new version of ethtool. * Remove: nested ptp-manual test plans The current manual test job steps are duplicated with the auto test job. Currently we only need to run the auto test job. The manual test steps here is used as a reference. If there are new jobs in th future, the ptp-manual test plan can be nested back. * Add: LD_PATH information in readme.md (canonical#41) * Change: additional patterns to map UDC and USB node (canonical#42) * Change: additional patterns to map UDC and USB node * Fix: the if else logic for map UDC * Fix: include python3-systemd in tox.ini (canonical#48) include the python3-systemd package in tox.ini * Fix: updated the related packages for tox (canonical#50) updated the related packages for tox * Fix: nested plan for server manual (canonical#44) * Fix: nested plan for server manual * Fix: errors when mtd count is 0 (canonical#55) * Fix: errors when mtd count is 0 * Add shellcheck ignore 2126 * Add: rpmsg tests for i.MX and TI processors (canonical#46) * Fix: updated the related packages for tox updated the related packages for tox * Add: rpmsg tests for i.MX and TI processors adding rpmsg tests for i.MX and TI processors * Fix: updated rpmsg_tests.py scripts updated rpmsg_tests.py scripts * Fix: fixed bugs fixed bugs * Modify optee test run out of gadget (canonical#45) * Add: support outsourcing optee-test * 230921 crypto test (canonical#52) * Fix: updated the related packages for tox updated the related packages for tox * Fix: update the crypto tests seperate the crypto tests to the generic crypto tests and crypto accelerator tests * Fix: bypass if failed to set extra seed the extra seed might not be supported for all platform * Fix: fixed bugs fixed bugs * Fix: refactor check_crypto_driver_priority func refactor check_crypto_driver_priority function * Fix: mock systemd-python package and remove packages in tox mock systemd-python package and remove packages in tox * Revamp otg_ports resource job should not fail (canonical#53) * Revamp otg_ports resource job should not fail Add a if condition to handle when the OTG config variable is not set. * Revise the echo string * Add: cupfreq module probe (canonical#51) * Add: cupfreq module probe * Check hwrng test (canonical#57) * Modify hwrng test more flexible * Add README.md * Make ce-oem-info jobs be attachment jobs (canonical#58) ce-oem-info jobs are aimed at collecting the system bootup information from snapd and cloud-init to help developers analyze bugs. Therefore, these jobs are more suitable to be attachment jobs and should not fail even if some information can not be found. For example, some systems may not have cloud-init. In these systems, the job cannot gather the information from cloud-init, but this should not be a failure. * Modify the device tree dump job (canonical#59) The current ce-oem-device-tree/dump job does not match the resource job format and will show warnings when running it. The original plan is to parse this job to match the resource format to let any other jobs able to get the dts info conveniently. As parse the dts to resource job format is a huge work, and will not be done in the near future. This commit excludes the ce-oem-device-tree/dump job from the test plan and add comments to explain the purpose. Also keep the attachment job to make sure we could get the device tree information when checking the report. * Revamp caam_hwrng_test by looping w/ smaller size (canonical#60) Some of the project devices' caam_hwrng have buffer, so previously the dd size is increased to 10M to exceed the buffer size to make the interrupt increase. It will make the device without buffer take a very long time (over 30 mins) to do the dd command. Therefore, this commit separated the size to 512K and looped it 20 times. On the device without buffer it could do 512K and detect the increased interrupt to get the pass result, which is much faster. On the device with buttfer it could also exceed the buffer by dd 20 times and detect the increased interrupt to get the pass result. * Fix the nested crypto test plan name (canonical#61) Previously the caam test plan was revamped to accelerator test plan, but forgot to update the name nested in ce-oem-automated. Fix the name in this commit. * Add: new socketcan stress test (canonical#62) * Add: new socketcan tests add new socketcan stress test (100000 loop echo test) add bus-off test and fixed bug * Add: include new socketcan tests (canonical#63) include new socketecan test plan into ce-oem test plans * Add ubuntu-frame and glmark2 tests (canonical#64) * Add ubuntu-frame and glmark2 tests * Modify the way checking ubuntu-frame active * Modify: Add more details in README * Add: implement the RPMSG-tty tests implement the RPMSG-tty tests for i.MX series * Fix: update tox configuration update tox configuration to perform tests with specific Python version * Add: check VPU device tests (canonical#68) * Add: check VPU device tests adding new test cases about identify VPU devices * Fix: fixed the issues from comments fixed the issues from comments * Fix: bug fixed bug fixed * Fix: fixed issues fixed issues * Fix: refactor the thermal tests (canonical#69) * Fix: refactor the thermal tests refactor the thermal tests * Fix: correct the argument name in socketcan_test (canonical#71) corrected the argument name in socketcan_test.py * Add dbus reboot stress (canonical#70) * Add script for installing test snap and connect interfaces * Add reboot via dbus command This test is for snap strict confinement mode * Add tcp test (canonical#65) * Add TCP test * Move ce-oem README inside the provider directory * Change the ce-oem provider namespace to the contrib namespace * Add README for the contrib area * Remove .gitignore from contrib/ Artefact left from the ce-oem provider migration --------- Co-authored-by: stanley31huang <stanley.huang@canonical.com> Co-authored-by: rickwu666666 <98441647+rickwu666666@users.noreply.github.com> Co-authored-by: Nara Huang <nara.huang@canonical.com> Co-authored-by: rickwu4444 <rick.wu@canonical.com> Co-authored-by: baconyao <patrick.chang@canonical.com> Co-authored-by: liaou3 <vincent.liao@canonical.com> Co-authored-by: patliuu <111331153+patliuu@users.noreply.github.com> Co-authored-by: Patrick Liu <patrick.liu@canonical.com> Co-authored-by: hanhsuan <32028620+hanhsuan@users.noreply.github.com>
binli
pushed a commit
to binli/checkbox
that referenced
this pull request
Mar 22, 2024
…al#932) * Initial commit * Migrate ARM generic jobs from launchpad * Initial commit * Clean up this repo, and up it to date. * Change: job id for cpu cases * Change: id of rpmsg * Add: manage.py Since we need this repo as a provider * Change: separate jobs into generic and non-generic * Add: checkbox-provider-arm folder Following the structure of checkbox * Change: the access of tool not created by us To prevent those tools been packaged into checkbox * Add: test plan for ARM generic jobs * Change: test method for NPU TensorFlow Change the test method by using test snap build by Isaac * Change: MTD test method Since we don't have reliable way to indentify the type of MTD. * Change: test method for buzzer Change the test method for buzzer that probe as an input PC speaker * Change: chunkfs test method * Add: dump DTB file * Add: manifest for the jobs * Add: rs485 remote test * Remove: non-generic jobs Since we need to move generic parts to github Co-authored-by: Nara Huang <nara.huang@canonical.com> * Add snapcraft yaml for uc20 (canonical#2) * Add: snapcraft.yaml for uc20 * Add: README.md * Fix: export PYTHONPATH in override-build section * Add: libc6-dev for build-packages of input-pcspkr Since snapcraft7.x will need extra lib from libc6-dev * Fix cpu and device tree cases (canonical#3) * Fix: cpu offlining test * Change: requires of cpu offlineing test * Add: stage-package linuxptp * Fix: test plan for dtb * Add eeprom test and change name of the provider (canonical#6) * Add: eeprom test cases * Change: provider name from arm to ce-oem * Change name space and README.md (canonical#9) * Change: name space * Change: README.md * Fix: wording of checkbox interface snap * Remove: snapcraft.yaml (canonical#8) * Add: led generic test (canonical#7) * Add: support cold reboot tests by pdu (canonical#10) * Add: support cold reboot tests by pdu add cold reboot tests by network PDU for those system which not able to wakeup by rtc * Fix: fix bugs add following changes in this commit - check supported pdu type inside of main function - sync data after dump device information - correct command for init-boot-loop-data job * Fix: fix bug - fix the error reported by shellcheck * Change prefixes (canonical#11) * Change: prefixes for plans and jobs * Change: rename test plan and add PDU related plan * Change: name of category * Fix: missing prefixes of serial test job and plan * Add: informations about used config in readme (canonical#13) * Change: sysfs led method (canonical#12) * Fix: rtc wakealarm checking to right attribute (canonical#14) * Fix: remove ce-oem-power-automated-by-pdu Remove the ce-oem-power-automated-by-pdu from the ce-oem-automated The ce-oem-power-automated-by-pud will always be executed even the user not intend to reboot the DUT by PDU. And the checkbox session will stuck after send out a snmpset command. So remove this test first, and we need to integrate the cold boot related tests to PPC to support cold reboot test for those DUT which not able to wake up by rtc. * Add: add a generic iot test plans Adding a generic IoT test plans, it's a full testing set include tests in certification test plan and ce-oem-generic test plan * Improve the MTD script Before doing the read and write testing of specific mtd partition, we need to know the MTD is read-only or writable, therefore, a new variable called is_mtd_writable to help us to know it. If this mtd is read-only, I only read its content to a file. If this mtd is writable, backup the its original content to a file, then do the compare procedure, finally, recover its content. * Add: CAAM related test (canonical#21) * Add: CAAM related test * Add: generic crypto test * Fix naming with ce-oem * Add: add digital-io tests (canonical#24) * Add: add digital-io tests add digital-io tests * Fix: fixed scripts and test plan - fixed the format string style - fixed the typo in docstring - fixed the variable type for TEST_STATES - remove useless test jobs * Fix: fixed typo fixed typo * Add: add readme (canonical#23) * Fix: updated README.md add a section about how to use checkbox-ce-oem as a checkbox launcher * Fix: updated the installation steps updated the installation steps for Ubuntu classic and Ubuntu core * fix: fixed typo fixed typo * Fix: updated digital-io tests (canonical#25) - corrected digital-io tests in post-suspend stage - import manifest job * Add: cpufreq governors test for 6 governors (canonical#19) * Add: cpufreq governors test for 6 governors Add a resource job to check system supported governors, and test jobs for all 6 governors. These tests are only applicable to the platform using acpi_cpufreq related drivers. Newer intel CPU using intel_pstate driver is NOT applicable. * Fix: typo and apply suggestions * Remove: ce-oem-cpu/offlining_test from the plan Since there is already a fix for cpu/offlining_test in checkbox base provider, this ce-oem-cpu/offlining_test is no longer required. canonical#522 * Modify CAAM and installation time related jobs. (canonical#26) * Modify: modify hwrng and caam algo check to require manifest * Add: requires for installation related jobs Make those jobs run only in ubuntu core * Fix: revert summary section for snapd_installation_time The previous commit accidentally removed the section * Fix script for pep8 check (canonical#27) * Remove: test job/script that not in use. Since we have new script for cpu freq governors. And also cpu_offline test has been updated in base provider. * Fix: python script to fit flake8 * Fix: manifest shouldn't place in resource job Move the manifest to test job. To prevent manifest being executed at bootstrap stage. * Add code check by tox just like checkbox did (canonical#28) * Add code check by tox just like checkbox did This commit following what checkbox did, and add pep8-naming check for python 3.8+. * Change the fork checkbox git repo url to offical one * Fix tox test environment error (canonical#32) * Fix tox test environment error Each tox test environment won't be reseted, therefore removing all providers before test is needed. After checking with checkbox repo, checkbox-ng has to use different install method to install. * Follow checkbox repo to update pyYAML version to 6.0.1 for python 3.10+ to solve tox issue * Fix for shellcheck (canonical#29) * Fix: shell scrip to fit shellcheck * Fix: audio jobs to fit shellcheck * Fix: caam jobs to fit shellcheck * Fix: device jobs to fit shellcheck * Fix: digital-io jobs to fit shellcheck * Change: nested EEPROM test from checkbox base Since EEPROM jobs merged back to the base provider already. * Fix: power-manager jobs to fit shellcheck * Change: nested RTC test from checkbox base Since RTC jobs merged back to the base provider already. * Fix: stress jobs to fit shellcheck * Fix: theraml-sensor jobs to fit shellcheck * Fix: Nested plan to ce-oem Follow the structure as a client-cert plan * Add multiple usb otg test (canonical#33) * Remove: OTG related jobs Those OTG jobs have been migrated back to base provider. * Add: multiple OTG test * Add: readme.md * Refactor cpufreq governor test (canonical#36) * Change: refactor cpufreq governor test CQT-2906 The original tests did not handle the following 2 conditions: 1. CPUfreq scaling driver not found (no files under cpufreq folder) 2. On ARM device, system using multiple policies to handle different CPUs group This commit refactor the tests to make them policies-based tests. Also adding a driver detect job that can be use on all platforms. * Add optee test (canonical#34) * Add: optee test * Add: README.md * Include led test from base (canonical#35) * Add: nested led job from checkbox-base * Change: README.md for led * Change: PTP command (canonical#37) * Change: PTP command lp:2030942, workaround for the current version of ptp4l * Fix: apply musical scales for pwm buzzer test (canonical#40) * Fix: apply musical scales for pwm buzzer test apply musical scales for pwm buzzer test * Fix: added description for test_pwm_buzzer func added description for test_pwm_buzzer function * Fix: add touchscreen tests (canonical#39) * Fix: add touchscreen tests validate the required touch events for touchscreen also add new test to cover maximum fingers support on touchscreen * Change: hardware PTP support keywords For PTP capabilities, ethtool v5.4 (on Ubuntu 20.04) shows `hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE)` But ethtool v5.16 (on Ubuntu 22.04) only shows `hardware-transmit` The test case was using `SOF_TIMESTAMPING_TX_HARDWARE` as keywords, This commit changes it to use `hardware-transmit` for adapting new version of ethtool. * Remove: nested ptp-manual test plans The current manual test job steps are duplicated with the auto test job. Currently we only need to run the auto test job. The manual test steps here is used as a reference. If there are new jobs in th future, the ptp-manual test plan can be nested back. * Add: LD_PATH information in readme.md (canonical#41) * Change: additional patterns to map UDC and USB node (canonical#42) * Change: additional patterns to map UDC and USB node * Fix: the if else logic for map UDC * Fix: include python3-systemd in tox.ini (canonical#48) include the python3-systemd package in tox.ini * Fix: updated the related packages for tox (canonical#50) updated the related packages for tox * Fix: nested plan for server manual (canonical#44) * Fix: nested plan for server manual * Fix: errors when mtd count is 0 (canonical#55) * Fix: errors when mtd count is 0 * Add shellcheck ignore 2126 * Add: rpmsg tests for i.MX and TI processors (canonical#46) * Fix: updated the related packages for tox updated the related packages for tox * Add: rpmsg tests for i.MX and TI processors adding rpmsg tests for i.MX and TI processors * Fix: updated rpmsg_tests.py scripts updated rpmsg_tests.py scripts * Fix: fixed bugs fixed bugs * Modify optee test run out of gadget (canonical#45) * Add: support outsourcing optee-test * 230921 crypto test (canonical#52) * Fix: updated the related packages for tox updated the related packages for tox * Fix: update the crypto tests seperate the crypto tests to the generic crypto tests and crypto accelerator tests * Fix: bypass if failed to set extra seed the extra seed might not be supported for all platform * Fix: fixed bugs fixed bugs * Fix: refactor check_crypto_driver_priority func refactor check_crypto_driver_priority function * Fix: mock systemd-python package and remove packages in tox mock systemd-python package and remove packages in tox * Revamp otg_ports resource job should not fail (canonical#53) * Revamp otg_ports resource job should not fail Add a if condition to handle when the OTG config variable is not set. * Revise the echo string * Add: cupfreq module probe (canonical#51) * Add: cupfreq module probe * Check hwrng test (canonical#57) * Modify hwrng test more flexible * Add README.md * Make ce-oem-info jobs be attachment jobs (canonical#58) ce-oem-info jobs are aimed at collecting the system bootup information from snapd and cloud-init to help developers analyze bugs. Therefore, these jobs are more suitable to be attachment jobs and should not fail even if some information can not be found. For example, some systems may not have cloud-init. In these systems, the job cannot gather the information from cloud-init, but this should not be a failure. * Modify the device tree dump job (canonical#59) The current ce-oem-device-tree/dump job does not match the resource job format and will show warnings when running it. The original plan is to parse this job to match the resource format to let any other jobs able to get the dts info conveniently. As parse the dts to resource job format is a huge work, and will not be done in the near future. This commit excludes the ce-oem-device-tree/dump job from the test plan and add comments to explain the purpose. Also keep the attachment job to make sure we could get the device tree information when checking the report. * Revamp caam_hwrng_test by looping w/ smaller size (canonical#60) Some of the project devices' caam_hwrng have buffer, so previously the dd size is increased to 10M to exceed the buffer size to make the interrupt increase. It will make the device without buffer take a very long time (over 30 mins) to do the dd command. Therefore, this commit separated the size to 512K and looped it 20 times. On the device without buffer it could do 512K and detect the increased interrupt to get the pass result, which is much faster. On the device with buttfer it could also exceed the buffer by dd 20 times and detect the increased interrupt to get the pass result. * Fix the nested crypto test plan name (canonical#61) Previously the caam test plan was revamped to accelerator test plan, but forgot to update the name nested in ce-oem-automated. Fix the name in this commit. * Add: new socketcan stress test (canonical#62) * Add: new socketcan tests add new socketcan stress test (100000 loop echo test) add bus-off test and fixed bug * Add: include new socketcan tests (canonical#63) include new socketecan test plan into ce-oem test plans * Add ubuntu-frame and glmark2 tests (canonical#64) * Add ubuntu-frame and glmark2 tests * Modify the way checking ubuntu-frame active * Modify: Add more details in README * Add: implement the RPMSG-tty tests implement the RPMSG-tty tests for i.MX series * Fix: update tox configuration update tox configuration to perform tests with specific Python version * Add: check VPU device tests (canonical#68) * Add: check VPU device tests adding new test cases about identify VPU devices * Fix: fixed the issues from comments fixed the issues from comments * Fix: bug fixed bug fixed * Fix: fixed issues fixed issues * Fix: refactor the thermal tests (canonical#69) * Fix: refactor the thermal tests refactor the thermal tests * Fix: correct the argument name in socketcan_test (canonical#71) corrected the argument name in socketcan_test.py * Add dbus reboot stress (canonical#70) * Add script for installing test snap and connect interfaces * Add reboot via dbus command This test is for snap strict confinement mode * Add tcp test (canonical#65) * Add TCP test * Move ce-oem README inside the provider directory * Change the ce-oem provider namespace to the contrib namespace * Add README for the contrib area * Remove .gitignore from contrib/ Artefact left from the ce-oem provider migration --------- Co-authored-by: stanley31huang <stanley.huang@canonical.com> Co-authored-by: rickwu666666 <98441647+rickwu666666@users.noreply.github.com> Co-authored-by: Nara Huang <nara.huang@canonical.com> Co-authored-by: rickwu4444 <rick.wu@canonical.com> Co-authored-by: baconyao <patrick.chang@canonical.com> Co-authored-by: liaou3 <vincent.liao@canonical.com> Co-authored-by: patliuu <111331153+patliuu@users.noreply.github.com> Co-authored-by: Patrick Liu <patrick.liu@canonical.com> Co-authored-by: hanhsuan <32028620+hanhsuan@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR import checkbox snaps repos without any modification for better tracking of the changes.