Skip to content

initial SVE intrinsics#2071

Open
davidtwco wants to merge 13 commits intorust-lang:mainfrom
davidtwco:sve-intrinsics
Open

initial SVE intrinsics#2071
davidtwco wants to merge 13 commits intorust-lang:mainfrom
davidtwco:sve-intrinsics

Conversation

@davidtwco
Copy link
Copy Markdown
Member

@davidtwco davidtwco commented Apr 5, 2026

This is a very large patch but the vast majority of it is generated code, the rest should be relatively straightforward to review.

This passes tests for me locally with the latest nightly after rust-lang/rust#153286 has landed.

I've bolded the commits that do the most interesting things - the rest are largely updating the support that has been partially upstreamed in the past to work with the current implementation in rustc.

  1. a6a76cb re-adds the "missing run-time test" check in stdarch-verify for Arm - it was accidentally removed in 713444d.
  2. 099249b removes some special-casing in the generator for files being generated from sve.spec.yml and sve2.spec.yml and instead adds a key that can be set in those files to achieve the same behaviour.
  3. 8d352c0 removes SvUndef expressions from the generator, these were used in svundef intrinsics and some others. svundef now zeroes and so doesn't need a special generator expression, and the other uses call svundef.
  4. b97d6fb changes uses of simd_reinterpret in the generator to transmute_unchecked - simd_reinterpret was expected to be added by rustc_scalable_vector(N) rust#143924 but was changed during reviews.
  5. 0934280 changes the behaviour of auto-llvm-sign-conversion in the generator for return types to match what it does for arguments. auto-llvm-sign-conversion isn't used by any existing intrinsic specification files.
  6. 4832b15 adds a static_assert_range helper that will be used in later commits.
  7. cddc125 defines the scalable vector types and adds the modules that the generated intrinsics will eventually live in. This defines some additional non-public scalable vector types necessary to match the signatures of the LLVM intrinsics, and some traits for conversions between those and the public scalable vector types.
  8. d754502 changes the generator to generate calls to SveInto::sve_into (a trait introduced in the previous commit). This trait is necessary because target_feature annotations are necessary on the trait method and that requires the method be unsafe, which is incompatible with the signature of Into::into.
  9. bc28f52 adds the intrinsic definitions - there's a lot of definitions and it likely isn't too practical to look over all of them. We later check that everything that should be being generated is being generated using stdarch-verify. Load/store tests will also be generated from these definitions.
  10. b8feb57 runs the generator after the above commit to generate the Rust code for all the intrinsic definitions.
  11. ebe8860 updates the intrinsics_data/arm_intrinsics.json file to add all the SVE intrinsics so that stdarch-verify can check all of the intrinsics are present.
  12. 5bdaf54 updates stdarch-verify with the necessary support and special-cases for SVE intrinsics.
  13. 7203b17 updates the intrinsic-test tool so that it can still parse arm_intrinsics.json and skips the SVE intrinsics.

The intrinsic test tool needs a lot of changes to work with SVE and this patch is getting big as it is, so we're going to do this as a follow-up (agreed in advance with @Amanieu).

This patch is based on #1509 and I've tried to preserve the co-authorship of everyone involved in that patch.

r? @Amanieu

davidtwco and others added 13 commits April 5, 2026 12:24
This was accidentally removed in 713444d.
Instead of generating load/store tests based on the input filename -
which no longer works given the expected input file structure of
`stdarch-gen-arm` - add a simple global context option that SVE specs can
set.
The `SvUndef` expression is no longer necessary as a
`core::intrinsics::scalable::sve_undef` intrinsic has been introduced
to produce an undefined SVE vector, used by `svundef*` intrinsics. Other
intrinsics that used `SvUndef` now use the `svundef*` intrinsics.
`simd_reinterpret` was expected to be used when it was added as
`transmute_unchecked` requires `Sized`, but scalable vectors are now
`Sized` so `transmute_unchecked` can be used and `simd_reinterpret` was
not added in rust-lang/rust#143924.
Matching the current behaviour for arguments, `auto_llvm_sign_conversion`
should only be required for `as_unsigned` conversions, not `into`
conversions.
This is a convenience macro used by the generated SVE intrinsics.

Co-authored-by: Jamie Cunliffe <Jamie.Cunliffe@arm.com>
Co-authored-by: Luca Vizzarro <Luca.Vizzarro@arm.com>
Co-authored-by: Adam Gemmell <adam.gemmell@arm.com>
Co-authored-by: Jacob Bramley <jacob.bramley@arm.com>
Add the SVE types (without any of the generated intrinsics) and empty
modules where the generated intrinsics will be. Enables the
`adt_const_params` crate feature that the generated intrinsics will use.

Co-authored-by: Jamie Cunliffe <Jamie.Cunliffe@arm.com>
Co-authored-by: Luca Vizzarro <Luca.Vizzarro@arm.com>
Co-authored-by: Adam Gemmell <adam.gemmell@arm.com>
Co-authored-by: Jacob Bramley <jacob.bramley@arm.com>
`Into::into` can't be used here because the implementations can't have
the required target feature, so `SveInto` needs to be introduced and
written by the generator
Thousands of lines of SVE intrinsic definitions..

Co-authored-by: Jamie Cunliffe <Jamie.Cunliffe@arm.com>
Co-authored-by: Luca Vizzarro <Luca.Vizzarro@arm.com>
Co-authored-by: Adam Gemmell <adam.gemmell@arm.com>
Co-authored-by: Jacob Bramley <jacob.bramley@arm.com>
Following from previous commit, this commit only contains generated code
from the SVE intrinsic specifications

Co-authored-by: Jamie Cunliffe <Jamie.Cunliffe@arm.com>
Co-authored-by: Luca Vizzarro <Luca.Vizzarro@arm.com>
Co-authored-by: Adam Gemmell <adam.gemmell@arm.com>
Co-authored-by: Jacob Bramley <jacob.bramley@arm.com>
Co-authored-by: Adam Gemmell <Adam.Gemmell@arm.com>
Co-authored-by: Jamie Cunliffe <Jamie.Cunliffe@arm.com>
Co-authored-by: Jacob Bramley <jacob.bramley@arm.com>
Co-authored-by: Luca Vizzarro <Luca.Vizzarro@arm.com>
Co-authored-by: Adam Gemmell <Adam.Gemmell@arm.com>
Co-authored-by: Jamie Cunliffe <Jamie.Cunliffe@arm.com>
Co-authored-by: Jacob Bramley <jacob.bramley@arm.com>
Co-authored-by: Luca Vizzarro <Luca.Vizzarro@arm.com>
With SVE intrinsics in the `arm_intrinsics.json`, the parsing needs to be
updated to know to expect any new fields.
@davidtwco
Copy link
Copy Markdown
Member Author

davidtwco commented Apr 5, 2026

Looks like this'll need another rustc patch to skip the "scalable vectors aren't supported on this architecture" error for doc builds

@davidtwco
Copy link
Copy Markdown
Member Author

rust-lang/rust#154850 will fix this

JonathanBrouwer added a commit to JonathanBrouwer/rust that referenced this pull request Apr 5, 2026
…, r=JonathanBrouwer

ast_validation: scalable vectors okay for rustdoc

Scalable vector types in `core_arch` are cfg'd for aarch64 and for rustdoc, which can successfully document these types given any `--target` (`core_arch` CI uses `i686-unknown-linux-gnu`) - this shouldn't trigger the "scalable vectors not supported on arch" error.

This fixes the CI failure in rust-lang/stdarch#2071.
JonathanBrouwer added a commit to JonathanBrouwer/rust that referenced this pull request Apr 5, 2026
…, r=JonathanBrouwer

ast_validation: scalable vectors okay for rustdoc

Scalable vector types in `core_arch` are cfg'd for aarch64 and for rustdoc, which can successfully document these types given any `--target` (`core_arch` CI uses `i686-unknown-linux-gnu`) - this shouldn't trigger the "scalable vectors not supported on arch" error.

This fixes the CI failure in rust-lang/stdarch#2071.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants