Skip to content

Commit

Permalink
[AArch64] Always allow the __bf16 type
Browse files Browse the repository at this point in the history
We would like to make the ACLE NEON and SVE intrinsics more useable by
gating them on the target, not by ifdef preprocessor macros. In order to
do this the types they use need to be available. This patches makes
__bf16 always available under AArch64 not just when the bf16
architecture feature is present. This bringing it in-line with GCC. In
subsequent patches the NEON bfloat16x8_t and SVE svbfloat16_t types
(along with bfloat16_t used in arm_sve.h) will be made unconditional
too.

The operations valid on the types are still very limited. They can be
used as a storage type, but the intrinsics used for convertions are
still behind an ifdef guard in arm_neon.h/arm_bf16.h.

Differential Revision: https://reviews.llvm.org/D130973
  • Loading branch information
davemgreen committed Aug 4, 2022
1 parent 88501dc commit 8c30f4a
Show file tree
Hide file tree
Showing 8 changed files with 59 additions and 39 deletions.
2 changes: 0 additions & 2 deletions clang/docs/LanguageExtensions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -757,8 +757,6 @@ performing the operation, and then truncating to ``_Float16``.
* 32-bit ARM
* 64-bit ARM (AArch64)

The ``__bf16`` type is only available when supported in hardware.

``__fp16`` is a storage and interchange format only. This means that values of
``__fp16`` are immediately promoted to (at least) ``float`` when used in arithmetic
operations, so that e.g. the result of adding two ``__fp16`` values has type ``float``.
Expand Down
4 changes: 4 additions & 0 deletions clang/lib/Basic/Targets/AArch64.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -670,6 +670,10 @@ bool AArch64TargetInfo::handleTargetFeatures(std::vector<std::string> &Features,
return true;
}

bool AArch64TargetInfo::hasBFloat16Type() const {
return true;
}

TargetInfo::CallingConvCheckResult
AArch64TargetInfo::checkCallingConvention(CallingConv CC) const {
switch (CC) {
Expand Down
2 changes: 2 additions & 0 deletions clang/lib/Basic/Targets/AArch64.h
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,8 @@ class LLVM_LIBRARY_VISIBILITY AArch64TargetInfo : public TargetInfo {
bool handleTargetFeatures(std::vector<std::string> &Features,
DiagnosticsEngine &Diags) override;

bool hasBFloat16Type() const override;

CallingConvCheckResult checkCallingConvention(CallingConv CC) const override;

bool isCLZForZeroUndef() const override;
Expand Down
11 changes: 8 additions & 3 deletions clang/test/CodeGen/arm-bf16-params-returns.c
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// RUN: %clang_cc1 -triple armv8.6a-arm-none-eabi -target-abi aapcs -mfloat-abi hard -target-feature +bf16 -target-feature +neon -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefix=CHECK32-HARD
// RUN: %clang_cc1 -triple armv8.6a-arm-none-eabi -target-abi aapcs -mfloat-abi softfp -target-feature +bf16 -target-feature +neon -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefix=CHECK32-SOFTFP
// RUN: %clang_cc1 -triple aarch64-arm-none-eabi -target-abi aapcs -target-feature +bf16 -target-feature +neon -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefix=CHECK64
// RUN: %clang_cc1 -triple aarch64-arm-none-eabi -target-abi aapcs -target-feature +bf16 -target-feature +neon -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefixes=CHECK64,CHECK64NEON
// RUN: %clang_cc1 -triple aarch64-arm-none-eabi -target-abi aapcs -target-feature -bf16 -target-feature +neon -DNONEON -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefix=CHECK64

// REQUIRES: aarch64-registered-target || arm-registered-target

Expand All @@ -17,12 +18,16 @@ __bf16 test_ret_bf16(__bf16 v) {
// CHECK64: define{{.*}} bfloat @test_ret_bf16(bfloat noundef returned %v) {{.*}} {
// CHECK64: ret bfloat %v

#ifndef NONEON

bfloat16x4_t test_ret_bf16x4_t(bfloat16x4_t v) {
return v;
}
// CHECK32-HARD: define{{.*}} arm_aapcs_vfpcc <4 x bfloat> @test_ret_bf16x4_t(<4 x bfloat> noundef returned %v) {{.*}} {
// CHECK32-HARD: ret <4 x bfloat> %v
// CHECK32-SOFTFP: define{{.*}} <2 x i32> @test_ret_bf16x4_t(<2 x i32> [[V0:.*]]) {{.*}} {
// CHECK32-SOFTFP: ret <2 x i32> %v
// CHECK64: define{{.*}} <4 x bfloat> @test_ret_bf16x4_t(<4 x bfloat> noundef returned %v) {{.*}} {
// CHECK64: ret <4 x bfloat> %v
// CHECK64NEON: define{{.*}} <4 x bfloat> @test_ret_bf16x4_t(<4 x bfloat> noundef returned %v) {{.*}} {
// CHECK64NEON: ret <4 x bfloat> %v

#endif
1 change: 1 addition & 0 deletions clang/test/CodeGen/arm-mangle-bf16.cpp
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// RUN: %clang_cc1 -triple aarch64-arm-none-eabi -target-feature +bf16 -emit-llvm -o - %s | FileCheck %s
// RUN: %clang_cc1 -triple aarch64-arm-none-eabi -target-feature -bf16 -emit-llvm -o - %s | FileCheck %s
// RUN: %clang_cc1 -triple arm-arm-none-eabi -target-feature +bf16 -mfloat-abi hard -emit-llvm -o - %s | FileCheck %s
// RUN: %clang_cc1 -triple arm-arm-none-eabi -target-feature +bf16 -mfloat-abi softfp -emit-llvm -o - %s | FileCheck %s

Expand Down
1 change: 1 addition & 0 deletions clang/test/Sema/arm-bf16-forbidden-ops.c
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// RUN: %clang_cc1 -fsyntax-only -verify -triple aarch64 -target-feature +bf16 %s
// RUN: %clang_cc1 -fsyntax-only -verify -triple aarch64 -target-feature -bf16 %s

__bf16 test_cast_from_float(float in) {
return (__bf16)in; // expected-error {{cannot type-cast to __bf16}}
Expand Down
1 change: 1 addition & 0 deletions clang/test/Sema/arm-bf16-forbidden-ops.cpp
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// RUN: %clang_cc1 -fsyntax-only -verify -triple aarch64 -target-feature +bf16 %s
// RUN: %clang_cc1 -fsyntax-only -verify -triple aarch64 -target-feature -bf16 %s

__bf16 test_static_cast_from_float(float in) {
return static_cast<__bf16>(in); // expected-error {{static_cast from 'float' to '__bf16' is not allowed}}
Expand Down
76 changes: 42 additions & 34 deletions clang/test/Sema/arm-bfloat.cpp
Original file line number Diff line number Diff line change
@@ -1,49 +1,57 @@
// RUN: %clang_cc1 -fsyntax-only -verify -std=c++11 \
// RUN: -triple aarch64-arm-none-eabi -target-cpu cortex-a75 \
// RUN: -target-feature +bf16 -target-feature +neon %s
// RUN: %clang_cc1 -fsyntax-only -verify -std=c++11 \
// RUN: -triple arm-arm-none-eabi -target-cpu cortex-a53 \
// RUN: -target-feature +bf16 -target-feature +neon %s
// RUN: %clang_cc1 -fsyntax-only -verify=scalar,neon -std=c++11 \
// RUN: -triple aarch64-arm-none-eabi -target-cpu cortex-a75 \
// RUN: -target-feature +bf16 -target-feature +neon %s
// RUN: %clang_cc1 -fsyntax-only -verify=scalar,neon -std=c++11 \
// RUN: -triple arm-arm-none-eabi -target-cpu cortex-a53 \
// RUN: -target-feature +bf16 -target-feature +neon %s

// The types should be available under AArch64 even without the bf16 feature
// RUN: %clang_cc1 -fsyntax-only -verify=scalar -DNONEON -std=c++11 \
// RUN: -triple aarch64-arm-none-eabi -target-cpu cortex-a75 \
// RUN: -target-feature -bf16 -target-feature +neon %s

// REQUIRES: aarch64-registered-target || arm-registered-target

void test(bool b) {
__bf16 bf16;

bf16 + bf16; // expected-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
bf16 - bf16; // expected-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
bf16 * bf16; // expected-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
bf16 / bf16; // expected-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
bf16 + bf16; // scalar-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
bf16 - bf16; // scalar-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
bf16 * bf16; // scalar-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
bf16 / bf16; // scalar-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}

__fp16 fp16;

bf16 + fp16; // expected-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
fp16 + bf16; // expected-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
bf16 - fp16; // expected-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
fp16 - bf16; // expected-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
bf16 * fp16; // expected-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
fp16 * bf16; // expected-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
bf16 / fp16; // expected-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
fp16 / bf16; // expected-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
bf16 = fp16; // expected-error {{assigning to '__bf16' from incompatible type '__fp16'}}
fp16 = bf16; // expected-error {{assigning to '__fp16' from incompatible type '__bf16'}}
bf16 + (b ? fp16 : bf16); // expected-error {{incompatible operand types ('__fp16' and '__bf16')}}
bf16 + fp16; // scalar-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
fp16 + bf16; // scalar-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
bf16 - fp16; // scalar-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
fp16 - bf16; // scalar-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
bf16 * fp16; // scalar-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
fp16 * bf16; // scalar-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
bf16 / fp16; // scalar-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
fp16 / bf16; // scalar-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
bf16 = fp16; // scalar-error {{assigning to '__bf16' from incompatible type '__fp16'}}
fp16 = bf16; // scalar-error {{assigning to '__fp16' from incompatible type '__bf16'}}
bf16 + (b ? fp16 : bf16); // scalar-error {{incompatible operand types ('__fp16' and '__bf16')}}
}

#ifndef NONEON

#include <arm_neon.h>

void test_vector(bfloat16x4_t a, bfloat16x4_t b, float16x4_t c) {
a + b; // expected-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'bfloat16x4_t')}}
a - b; // expected-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'bfloat16x4_t')}}
a * b; // expected-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'bfloat16x4_t')}}
a / b; // expected-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'bfloat16x4_t')}}

a + c; // expected-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'float16x4_t' (vector of 4 'float16_t' values))}}
a - c; // expected-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'float16x4_t' (vector of 4 'float16_t' values))}}
a * c; // expected-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'float16x4_t' (vector of 4 'float16_t' values))}}
a / c; // expected-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'float16x4_t' (vector of 4 'float16_t' values))}}
c + b; // expected-error {{invalid operands to binary expression ('float16x4_t' (vector of 4 'float16_t' values) and 'bfloat16x4_t' (vector of 4 'bfloat16_t' values))}}
c - b; // expected-error {{invalid operands to binary expression ('float16x4_t' (vector of 4 'float16_t' values) and 'bfloat16x4_t' (vector of 4 'bfloat16_t' values))}}
c * b; // expected-error {{invalid operands to binary expression ('float16x4_t' (vector of 4 'float16_t' values) and 'bfloat16x4_t' (vector of 4 'bfloat16_t' values))}}
c / b; // expected-error {{invalid operands to binary expression ('float16x4_t' (vector of 4 'float16_t' values) and 'bfloat16x4_t' (vector of 4 'bfloat16_t' values))}}
a + b; // neon-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'bfloat16x4_t')}}
a - b; // neon-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'bfloat16x4_t')}}
a * b; // neon-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'bfloat16x4_t')}}
a / b; // neon-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'bfloat16x4_t')}}

a + c; // neon-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'float16x4_t' (vector of 4 'float16_t' values))}}
a - c; // neon-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'float16x4_t' (vector of 4 'float16_t' values))}}
a * c; // neon-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'float16x4_t' (vector of 4 'float16_t' values))}}
a / c; // neon-error {{invalid operands to binary expression ('bfloat16x4_t' (vector of 4 'bfloat16_t' values) and 'float16x4_t' (vector of 4 'float16_t' values))}}
c + b; // neon-error {{invalid operands to binary expression ('float16x4_t' (vector of 4 'float16_t' values) and 'bfloat16x4_t' (vector of 4 'bfloat16_t' values))}}
c - b; // neon-error {{invalid operands to binary expression ('float16x4_t' (vector of 4 'float16_t' values) and 'bfloat16x4_t' (vector of 4 'bfloat16_t' values))}}
c * b; // neon-error {{invalid operands to binary expression ('float16x4_t' (vector of 4 'float16_t' values) and 'bfloat16x4_t' (vector of 4 'bfloat16_t' values))}}
c / b; // neon-error {{invalid operands to binary expression ('float16x4_t' (vector of 4 'float16_t' values) and 'bfloat16x4_t' (vector of 4 'bfloat16_t' values))}}
}
#endif

0 comments on commit 8c30f4a

Please sign in to comment.