qemu: Update to QEMU 6.2 release #421

stephanosio · 2021-12-16T03:54:51Z

This commit updates the QEMU version to 6.2.

QEMU 6.2 is required to support ARM M-Profile Vector Extension (MVE)
emulation.

Signed-off-by: Stephanos Ioannidis root@stephanos.io

Closes #420.
Closes #423.

galak

feels like the patch list needs modification.

meta-zephyr-sdk/recipes-devtools/qemu/zephyr-qemu_git.bb

stephanosio · 2021-12-27T03:23:17Z

DO NOT MERGE: Validating the commit that switches to using the Zephyr fork first. The commit that actually updates QEMU to v6.2.0 will be added here soon.

stephanosio · 2021-12-27T06:44:21Z

Patch Status

0001-hw-misc-mps2-scc-Add-QEMU-interface-comment.patch
  Merged upstream (c52c266d24b10f1482602e6d22938d9e21f874f5)
0002-hw-misc-mps2-scc-Support-using-CFG0-bit-0-for-remapp.patch
  Merged upstream (5bddf92e689c0a3da57f4fd17b83d4eb1e436b80)
0003-hw-arm-mps2-tz-Implement-AN524-memory-remapping-via-.patch
  Merged upstream (f1dfab0d9b77b8649d60ded3d96870f88d0baccd)
0004-qemu-nios2-Add-Altera-MAX-10-board-support-for-Zephy.patch
  Applied as-is (Zephyr-specific)
0005-hw-sparc-Add-leon-at697-machine.patch
  Applied as-is
0006-hw-sparc-leon-Fix-compilation-errors.patch
  Applied as-is
0007-hw-sparc-leon-timer-Call-leon_timer_io_read-for-TIME.patch
  Applied as-is
0008-hw-sparc-leon-Switch-to-transaction-based-ptimer-API.patch
  Applied as-is
0009-hw-sparc-leon-Fix-compilation-errors-with-qemu-6-cha.patch
  Applied as-is
0010-Add-ARC-support.patch
  Not applied (Multiple conflicts, patch too big to be reworked)
0011-hw-arm-mps2-tz-Don-t-duplicate-modelling-of-SRAM-in-.patch
  Merged upstream (b6889c5ae3895cf5a4322adb32b2133e9b91d158)
0012-hw-arm-mps2-tz-Make-SRAM_ADDR_WIDTH-board-specific.patch
  Merged upstream (902b28ae4eba6df303cba57016945426865a6d59)
0013-hw-arm-armsse.c-Correct-modelling-of-SSE-300-interna.patch
  Merged upstream (4eb1770988397354ed95dec2bd63c09348ebf707)
0014-hw-arm-armsse-Convert-armsse_realize-to-use-ERRP_GUA.patch
  Merged upstream (32962103523838faf15c070bff0fc75dfd2f42c8)
0015-hw-arm-mps2-tz-Allow-board-to-specify-a-boot-RAM-siz.patch
  Merged upstream (2f12dca05928078ecc5c7d209a2bc5af61bff966)
0016-hw-arm-Model-TCMs-in-the-SSE-300-not-the-AN547.patch
  Merged upstream (cbb563887781d813e67c59b68dd76891cb78c3d4)
0017-target-arm-Use-correct-SP-in-M-profile-exception-ret.patch
  Merged upstream (659f042ba8ef80ca1053042a781a4488f9a587dc)
0018-arc-virt-Make-target-memory-size-configurable.patch
  Not applied (Depends on '0010-Add-ARC-support.patch')

stephanosio · 2021-12-27T06:49:34Z

@abrodkin It looks like the QEMU ARC patch (0010-Add-ARC-support.patch) has not been upstreamed yet. Are you able to provide an updated patch based on top of the QEMU 6.2 release?

abrodkin · 2021-12-27T10:18:18Z

@stephanosio well my patch could be easily applied on top of QEMU v6.2 in theory. But the reality is ARC port is still based on upstream v6.0.0 and my quick attempt to rebase on v6.2.0 was not successful as there're not only build-system changes but also some data structures etc. That said I'd like our QEMU guru @cupertinomiranda to conduct that re-base and since he's on Christmas holidays it might only happen sometime early next year.

And given ARC pieces were not yet accepted into upstream QEMU I'd like to ask to wait a bit for QEMU update in Zephyr SDK, as otherwise all ARC support will be effectively dropped from QEMU in the SDK/perf-PR CI etc. And I'd like to not add qemu-arc to the meta-zephyr-sdk by all means (at least now).

stephanosio · 2021-12-28T03:50:29Z

@abrodkin Thanks. I will put this PR on hold until the rebased ARC patch is provided.

abrodkin · 2021-12-28T07:14:43Z

@stephanosio just in case when do you think you'll really need to switch to v6.2? To make sure because of other things we don't slip way too much our rebase.

stephanosio · 2021-12-28T07:56:34Z

just in case when do you think you'll really need to switch to v6.2?

@abrodkin Ideally, as soon as possible. The motivation for switching to v6.2 is to support the ARM MVE instruction emulation and testing in the CI using the QEMU (see #420). There are a few Zephyr PRs that are waiting for this as well.

stephanosio · 2021-12-30T11:25:34Z

TODO: Install/update meson version in the SDK CI image (pip3 install meson).

../git/meson.build:1:0: ERROR: Meson version is 0.55.1 but project requires >=0.58.2

stephanosio · 2022-01-19T00:26:49Z

@abrodkin @cupertinomiranda any updates?

abrodkin · 2022-01-21T08:09:12Z

@stephanosio thanks to the work done by @cupertinomiranda we've got our port re-based on top of v6.2.0.
Now I'm going to test it against Zephyr and once I see it works as good as before and will send a PR for the SDK-ng.

stephanosio · 2022-01-21T08:33:10Z

@stephanosio thanks to the work done by @cupertinomiranda we've got our port re-based on top of v6.2.0. Now I'm going to test it against Zephyr and once I see it works as good as before and will send a PR for the SDK-ng.

@abrodkin Thanks! When you are done testing, you can open a pull request to https://github.com/zephyrproject-rtos/qemu/commits/zephyr-qemu-v6.2.0, or you can provide a patch file, whichever is more convenient for you.

microbuilder · 2022-02-02T12:16:24Z

Just a heads up that there is a bug with systick in QEMU 6.2.0 that breaks mps2_an521 and mps3_an547 support, perhaps others. I tested this fix this morning, but we will either need to update to the next release to fix MPSx support, or apply this patch: https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg00333.html

stephanosio · 2022-02-03T05:55:25Z

@microbuilder Thanks for the heads up. I will make sure to apply that patch and test all supported ARM platforms before merging this.

microbuilder · 2022-02-16T16:57:27Z

Thanks for the heads up. I will make sure to apply that patch and test all supported ARM platforms before merging this.

An LSM303DLHC magnetometer driver I wrote was merged in, as well as a change to make a subset of I2C drivers available to most ARM QEMU targets that enable I2C (mps2_an521, mps3_an547, etc.). This is after the 6.2.0 tag, but I think these changes are valuable to Zephyr to enable testing I2C in CI with QEMU.

You can enable this optional driver with something like:

if(BOARD MATCHES "mps2_an521")
  list(APPEND QEMU_EXTRA_FLAGS -device lsm303dlhc_mag,id=lsm303,address=0x1E)
endif()

abrodkin · 2022-03-11T07:02:46Z

@stephanosio oh, sorry for the delay. Indeed, we have now usable QEMU port for ARC based on 6.2.
The reason I was a bit hesitant on posting an update - there's one reliable regression now: https://github.com/zephyrproject-rtos/zephyr/tree/main/tests/kernel/timer/timer_monotonic fails on all ARC QEMU targets.

And even though we did and still plan to solve it, to not delay SDK update with newer QEMU, I guess it should be fine to just ignore that test for ARC QEMU platforms for a time being.

stephanosio · 2022-03-11T11:20:27Z

@abrodkin Thanks for the update. Since it looks like it will take some time to upstream the ARC support for QEMU and it will be hard to keep the Zephyr SDK QEMU version and the ARC QEMU version in sync until that happens, I think it would be best to create a separate ARC QEMU component, which builds directly from https://github.com/foss-for-synopsys-dwc-arc-processors/qemu, as we do with the Xilinx QEMU.

What do you think?

stephanosio · 2022-03-11T11:22:19Z

Postponing this to 0.14.1, which will be a quick follow-up to 0.14.0 (multi-platform SDK) in a month or so.

This commit upgrades the meson package version from 0.55.1, which is provided by the current Poky distribution used, to 0.59.1. Meson 0.58.2 or above is required for building QEMU 6.2. Signed-off-by: Stephanos Ioannidis <root@stephanos.io>

Use the Zephyr QEMU fork (zephyrproject-rtos/qemu) to build the QEMU for the Zephyr SDK. This uses the `zephyr-qemu-v6.0.0` branch in the Zephyr QEMU fork, which is based on the QEMU v6.0.0 release and contains all the Zephyr- related patches previously used by the OE build system. Note that the patch file `cross.patch` was not integrated into the Zephyr fork branch because it is a Yocto/OE build environment-specific workaround. Signed-off-by: Stephanos Ioannidis <root@stephanos.io>

This commit updates the QEMU version to 6.2. QEMU 6.2 is required to support ARM M-Profile Vector Extension (MVE) emulation. Signed-off-by: Stephanos Ioannidis <root@stephanos.io>

This commit makes the following changes to add a separate QEMU recipe for ARC. 1. Re-generate "cross.patch" as there're quite some changes in the newer "configure" script. 2. Remove "--disable-sheepdog" as otherwise configurations fails with --------------->8------------ ERROR: unknown option --disable-sheepdog --------------->8------------ 3. Add symlinks to the ARC QEMU emulator executables in `/usr/bin`. Co-authored-by: Stephanos Ioannidis <root@stephanos.io> Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com> Signed-off-by: Stephanos Ioannidis <root@stephanos.io>

addressed

stephanosio · 2022-03-18T00:00:13Z

Just a heads up that there is a bug with systick in QEMU 6.2.0 that breaks mps2_an521 and mps3_an547 support, perhaps others. I tested this fix this morning, but we will either need to update to the next release to fix MPSx support, or apply this patch: https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg00333.html

@microbuilder I have integrated this patch here and verified that the Zephyr kernel tests pass on the MPS boards.

stephanosio · 2022-03-18T00:01:05Z

An LSM303DLHC magnetometer driver I wrote was merged in, as well as a change to make a subset of I2C drivers available to most ARM QEMU targets that enable I2C (mps2_an521, mps3_an547, etc.). This is after the 6.2.0 tag, but I think these changes are valuable to Zephyr to enable testing I2C in CI with QEMU.

https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg01482.html

https://lists.gnu.org/archive/html/qemu-devel/2022-02/msg01818.html

@microbuilder I will look into integrating these patches in a separate PR.

stephanosio · 2022-03-18T00:02:01Z

@abrodkin I have verified that all ARC tests pass using the separate ARC QEMU build here.

stephanosio · 2022-03-18T00:02:11Z

@microbuilder @abrodkin please review

abrodkin

Awesome! Thanks for all the work here. LGTM!
The only thing which is IMHO missing inability to download built artifacts, i.e. it's not possible to test-drive built entire SDK. Maybe it's due to this repo settings as back in the days of Buildkite I was able to do so (thanks to @galak who granted me extended permissions).

abrodkin · 2022-03-18T13:30:15Z

meta-zephyr-sdk/recipes-devtools/arc_qemu/files/cross.patch

+===================================================================
+--- a/configure
+++ b/configure
+@@ -3934,7 +3934,6 @@ if test "$skip_meson" = no; then


Offsets differ from the same patch for generic QEMU, so maybe you copy over one version over another so that they match exactly? Otherwise it might look a bit suspicious.

stephanosio · 2022-03-18T13:59:43Z

The only thing which is IMHO missing inability to download built artifacts, i.e. it's not possible to test-drive built entire SDK. Maybe it's due to this repo settings as back in the days of Buildkite I was able to do so (thanks to @galak who granted me extended permissions).

@abrodkin You should find the artifacts under the checks "Summary" tab:
https://github.com/zephyrproject-rtos/sdk-ng/actions/runs/2000487547

abrodkin · 2022-03-18T14:39:19Z

@stephanosio ooops, I didn't notice that section as it was just below my screen edge, sorry for that!

microbuilder · 2022-03-18T18:36:40Z

@stephanosio It looks like we only generate x86 builds? I'm on an AArch64 machine, but I'll setup an x86-64 machine to test these and make sure QEMU behaves as expected with the MPS2 and MPS3 targets, etc.

stephanosio · 2022-03-18T19:02:35Z

@stephanosio It looks like we only generate x86 builds? I'm on an AArch64 machine, but I'll setup an x86-64 machine to test these and make sure QEMU behaves as expected with the MPS2 and MPS3 targets, etc.

PR builds are, by default, x86-64 only (need to label the PR ci-linux-aarch64 to build for AArch64).

microbuilder · 2022-03-21T11:09:26Z

@stephanosio ~~We'll need this patch in the Zephyr SDK or the systick timer is broken on a number of platforms we use to test Arm in CI: https://patchew.org/QEMU/20220201192650.289584-1-git@rpls.de/~~

EDIT: Sorry, looks like this was included.

stephanosio · 2022-03-21T11:12:13Z

@stephanosio We'll need this patch in the Zephyr SDK or the systick timer is broken on a number of platforms we use to test Arm in CI: https://patchew.org/QEMU/20220201192650.289584-1-git@rpls.de/

The issue was introduced between 6.1.0 and 6.2.0.

Not sure if you saw #421 (comment). I have already applied that patch and tested that all tests for the emulated platforms (including MPS) pass.

abrodkin · 2022-03-21T12:03:42Z

@stephanosio for the record with the SDK build from this PR's CI I have very clean results while executing full twister on all 3 ARC QEMU platforms: qemu_arc_em, qemu_arc_hs & qemu_arc_hs6x with just 1 failure, already captured in foss-for-synopsys-dwc-arc-processors/qemu#67:

$ ./scripts/twister --platform qemu_arc_em --platform qemu_arc_hs --platform qemu_arc_hs6x
INFO    - Zephyr version: zephyr-v3.0.0-1286-g38938b4e594a
INFO    - JOBS: 12
INFO    - Using 'zephyr' toolchain.
INFO    - Building initial testcase list...
INFO    - 1651 test scenarios (4686 configurations) selected, 3483 configurations discarded due to filters.
INFO    - Adding tasks to the queue...
INFO    - Added initial list of jobs to queue
INFO    - Total complete:  553/1203  45%  skipped: 3601, failed:    0
ERROR   - qemu_arc_em               tests/kernel/mem_protect/syscalls/kernel.memory_protection.syscalls FAILED: Exited with 2
ERROR   - see: .../twister-out/qemu_arc_em/tests/kernel/mem_protect/syscalls/kernel.memory_protection.syscalls/handler.log
INFO    - Total complete: 1203/1203  100%  skipped: 3851, failed:    1
INFO    - 834 of 835 test configurations passed (99.88%), 1 failed, 3851 skipped with 0 warnings in 3971.16 seconds
INFO    - In total 5425 test cases were executed, 55082 skipped on 3 out of total 442 platforms (0.68%)
INFO    - 706 test configurations executed on platforms, 129 test configurations were only built.

Again, thanks for all the efforts!
And shouldn't we close #445 now?

stephanosio requested a review from galak December 16, 2021 03:54

galak previously requested changes Dec 16, 2021

View reviewed changes

meta-zephyr-sdk/recipes-devtools/qemu/zephyr-qemu_git.bb Show resolved Hide resolved

stephanosio force-pushed the qemu_6_2 branch from 3e35f71 to 1f605d4 Compare December 27, 2021 03:17

stephanosio added the DNM DO NOT MERGE label Dec 27, 2021

stephanosio force-pushed the qemu_6_2 branch from 1f605d4 to 9a05d41 Compare December 27, 2021 04:25

stephanosio added this to the 0.13.3 milestone Dec 27, 2021

stephanosio force-pushed the qemu_6_2 branch from d7cad80 to 3543bf4 Compare December 28, 2021 03:45

stephanosio force-pushed the qemu_6_2 branch from 3543bf4 to f1b2520 Compare December 28, 2021 12:36

stephanosio mentioned this pull request Dec 30, 2021

Upgrade qemu to fix breakage in mps3-an547 zephyrproject-rtos/zephyr#40970

Closed

microbuilder mentioned this pull request Feb 2, 2022

Update Qemu to get MVE support for the MPS3 AN547 #420

Closed

stephanosio added the area: QEMU Issues related to QEMU label Feb 22, 2022

stephanosio modified the milestones: 0.13.3, 0.14.1 Mar 11, 2022

stephanosio force-pushed the qemu_6_2 branch 2 times, most recently from 0acf2dc to d99c61f Compare March 17, 2022 07:36

stephanosio added 2 commits March 17, 2022 16:45

meta-zephyr-sdk: Upgrade meson to 0.59.1

93e8560

This commit upgrades the meson package version from 0.55.1, which is provided by the current Poky distribution used, to 0.59.1. Meson 0.58.2 or above is required for building QEMU 6.2. Signed-off-by: Stephanos Ioannidis <root@stephanos.io>

stephanosio force-pushed the qemu_6_2 branch 2 times, most recently from 144611c to 818dcd6 Compare March 17, 2022 11:45

qemu: Update to QEMU 6.2 release

d676b65

This commit updates the QEMU version to 6.2. QEMU 6.2 is required to support ARM M-Profile Vector Extension (MVE) emulation. Signed-off-by: Stephanos Ioannidis <root@stephanos.io>

stephanosio force-pushed the qemu_6_2 branch from 818dcd6 to 2403370 Compare March 17, 2022 15:50

stephanosio force-pushed the qemu_6_2 branch from 2403370 to 9b16518 Compare March 17, 2022 19:35

stephanosio requested a review from microbuilder March 17, 2022 23:59

stephanosio requested a review from abrodkin March 18, 2022 00:38

stephanosio removed the DNM DO NOT MERGE label Mar 18, 2022

abrodkin approved these changes Mar 18, 2022

View reviewed changes

microbuilder approved these changes Mar 21, 2022

View reviewed changes

stephanosio merged commit 5631596 into zephyrproject-rtos:main Mar 21, 2022

stephanosio linked an issue Mar 21, 2022 that may be closed by this pull request

Add separate QEMU build for ARC #445

Closed

stephanosio mentioned this pull request Mar 24, 2022

qemu: Pull in I2C device emulation support for ARM targets #449

Merged

qemu: Update to QEMU 6.2 release #421

qemu: Update to QEMU 6.2 release #421

Uh oh!

Conversation

stephanosio commented Dec 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

galak left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

stephanosio commented Dec 27, 2021

Uh oh!

stephanosio commented Dec 27, 2021

Uh oh!

stephanosio commented Dec 27, 2021

Uh oh!

abrodkin commented Dec 27, 2021

Uh oh!

stephanosio commented Dec 28, 2021

Uh oh!

abrodkin commented Dec 28, 2021

Uh oh!

stephanosio commented Dec 28, 2021

Uh oh!

stephanosio commented Dec 30, 2021

Uh oh!

stephanosio commented Jan 19, 2022

Uh oh!

abrodkin commented Jan 21, 2022

Uh oh!

stephanosio commented Jan 21, 2022

Uh oh!

microbuilder commented Feb 2, 2022

Uh oh!

stephanosio commented Feb 3, 2022

Uh oh!

microbuilder commented Feb 16, 2022

Uh oh!

abrodkin commented Mar 11, 2022

Uh oh!

stephanosio commented Mar 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stephanosio commented Mar 11, 2022

Uh oh!

stephanosio commented Mar 18, 2022

Uh oh!

stephanosio commented Mar 18, 2022

Uh oh!

stephanosio commented Mar 18, 2022

Uh oh!

stephanosio commented Mar 18, 2022

Uh oh!

abrodkin left a comment

Choose a reason for hiding this comment

Uh oh!

abrodkin Mar 18, 2022

Choose a reason for hiding this comment

Uh oh!

stephanosio commented Mar 18, 2022

Uh oh!

abrodkin commented Mar 18, 2022

Uh oh!

microbuilder commented Mar 18, 2022

Uh oh!

stephanosio commented Mar 18, 2022

Uh oh!

microbuilder commented Mar 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stephanosio commented Mar 21, 2022

Uh oh!

abrodkin commented Mar 21, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

stephanosio commented Dec 16, 2021 •

edited

Loading

stephanosio commented Mar 11, 2022 •

edited

Loading

microbuilder commented Mar 21, 2022 •

edited

Loading