[BoundsSafety] Initial documentation for -fbounds-safety #70749

rapidsna · 2023-10-30T23:56:41Z

The document is mostly the exact copy of RFC: Enforcing Bounds Safety in C, except some minor adjustments in the tone to make it more suitable for documentation:
https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-c-fbounds-safety/70854

Further changes and clarifications for the programming model will be done as separate patches to make it easier to track history of changes.

delcypher

I have a few minor nits but other than that LGTM.

Please make sure this builds without errors/warnings.

I think you can configure CMake with -DLLVM_BUILD_DOCS=ON -D LLVM_ENABLE_SPHINX=ON -DSPHINX_WARNINGS_AS_ERROR=ON to enable building the documentation. The target to build the docs is docs-clang-html

clang/docs/BoundsSafety.rst

delcypher · 2023-11-03T21:34:48Z

clang/docs/BoundsSafety.rst

+
+Using this bounds information, the compiler inserts bounds checks on every pointer dereference, ensuring that the program does not access memory outside the specified bounds. The compiler requires programmers to provide enough bounds information so that the accesses can be checked at either run time or compile time — and it rejects code if it cannot.
+
+The most important contribution of “-fbounds-safety” is how it reduces the programmer’s annotation burden by reconciling bounds annotations at ABI boundaries with the use of implicit wide pointers (a.k.a. “fat” pointers) that carry bounds information on local variables without the need for annotations. We designed this model so that it preserves ABI compatibility with C while minimizing adoption effort.


Did you mean to use open and close quotes (“ and ”) rather than symmetrical quote "? I would prefer to use the symmetrical quote because it makes edits to the document easier.

clang/docs/BoundsSafety.rst

delcypher · 2023-11-07T22:01:20Z

@rapidsna The CI is complaining about trailing whitespace

https://buildkite.com/llvm-project/clang-ci/builds/5817#018b830d-fb91-40df-8956-9e4dd325d27f

AaronBallman

Thank you for the documentation! In general, it's looking great. I did have some specific questions or ideas on the more user-facing documentation. I've not yet thoroughly reviewed the implementation plans docs.

One thing that's not clear from this is how bounds safety annotations interact with variable-length arrays or variably-modified types. e.g.,

void func(int n, int vla[n]) {
 // Within the function, is vla treated as-if it was __counted_by(n)?
}

or

void func(int n) {
  int vla[n];
  int *ptr = vla; // Does this calculate the correct upper bounds based on n?
}

We should probably have explicit mention given that VLAs are a source of security issues related to bounds. (In fact, we might even want to add specific bounds checks for VLAs such as "will the VLA fit comfortably within the stack frame?" or "these VLA bounds are user-controllable which is a Very Bad Idea™".)

clang/docs/BoundsSafety.rst

AaronBallman · 2023-11-29T14:32:31Z

clang/docs/BoundsSafety.rst

+
+In C, when an array is referenced, it is automatically promoted (or “decayed”) to a pointer to its first element (e.g., ``&arr[0]``). Similarly, in ``-fbounds-safety``, arrays are also promoted to pointers, but with the addition of an implicit bounds annotation. Arrays on function parameters are promoted to corresponding ``__counted_by`` pointers. Consequently, incomplete arrays (or arrays without size) will cause a compiler error unless it has ``__counted_by`` annotation in its bracket. All other arrays are promoted to ``__bidi_indexable`` pointers, with the equivalent of ``&arr[0]`` serving as the lower bound and ``&arr[array_size]`` (or one past the last element) serving as the upper bound. This way, all array accesses are subject to bounds checking, just as their corresponding pointers are.
+
+Maintaining correctness of bounds annotations


One thing that's not clear is how this interacts with optimizations around undefined behavior; can the optimizer remove these checks? Can it do so if it thinks the situation can't happen because it would be UB if it did and the optimizer assumes no UB?

The optimizer may be able to remove checks if the pointer is derived from __counted_by and the size is known at compile time. I added some details to explain at the end of "Maintaining correctness of bounds annotations".

Thanks! The root of my concern was time travel optimizations, but I'm not certain we can run into such a problem with the way you've designed things. e.g., (using null pointers instead of bounds)

void func(int *ip) { int i = *ip; if (ip) { foo(i); } }

where the optimizer sees the dereference and decides it can remove the if statement as a result. I think you can't run into a similar problem though because of the instrumented checks you add. e.g.,

void func(int n, int *ip __counted_by(n)) { int *end = ip + 100; // Oops, should be + n instead! ... }

I was concerned the optimizer may decide this means ip + 100 must be a valid range because otherwise it would be UB. But I think the instrumentation code will add actual bounds checking there and so the optimizer won't make that assumption. Does that sound about right to you?

Right. -fbounds-safety determines pointer arithmetic overflow to be equivalent of two's complement integer computation and it means ip + 100 doesn't get inbounds. Only the known upper bound calculation, e.g., ip + n provided by __counted_by(n), gets inbounds.

Interestingly, this means, if the compiler knows n > 100 as in the following example, the bounds check may still be able to be optimized.

int func(int n, int *ip __counted_by(n)) { int *ptr = ip + 100; if (n > 100) return *ptr; // The bounds check for `ptr` may be optimized. return 0; }

Thanks, that's great!

clang/docs/BoundsSafety.rst

adrian-prantl

Debug info part looks good!

clang/docs/BoundsSafetyImplPlans.rst

clang/docs/BoundsSafety.rst

AaronBallman · 2023-12-06T19:10:09Z

clang/docs/BoundsSafety.rst

+
+In C, when an array is referenced, it is automatically promoted (or “decayed”) to a pointer to its first element (e.g., ``&arr[0]``). Similarly, in ``-fbounds-safety``, arrays are also promoted to pointers, but with the addition of an implicit bounds annotation. Arrays on function parameters are promoted to corresponding ``__counted_by`` pointers. Consequently, incomplete arrays (or arrays without size) will cause a compiler error unless it has ``__counted_by`` annotation in its bracket. All other arrays are promoted to ``__bidi_indexable`` pointers, with the equivalent of ``&arr[0]`` serving as the lower bound and ``&arr[array_size]`` (or one past the last element) serving as the upper bound. This way, all array accesses are subject to bounds checking, just as their corresponding pointers are.
+
+Maintaining correctness of bounds annotations


Thanks! The root of my concern was time travel optimizations, but I'm not certain we can run into such a problem with the way you've designed things. e.g., (using null pointers instead of bounds)

void func(int *ip) { int i = *ip; if (ip) { foo(i); } }

where the optimizer sees the dereference and decides it can remove the if statement as a result. I think you can't run into a similar problem though because of the instrumented checks you add. e.g.,

void func(int n, int *ip __counted_by(n)) { int *end = ip + 100; // Oops, should be + n instead! ... }

I was concerned the optimizer may decide this means ip + 100 must be a valid range because otherwise it would be UB. But I think the instrumentation code will add actual bounds checking there and so the optimizer won't make that assumption. Does that sound about right to you?

AaronBallman

FWIW, there were other comments from a previous round of review that didn't get addressed (GitHub may have helpfully hidden them from view for you -- I had to click on a "More" button to get them to show for me.)

AaronBallman · 2023-12-06T19:30:05Z

clang/docs/BoundsSafety.rst

+subject to be actively updated with a more detailed specification. The
+implementation plan can be found in Implementation plans for -fbounds-safety.
+
+.. Cross reference doesn't currently work


Sphinx references need to start with an underscore and end with a colon e.g., .. _some-ref: and then get referenced via :ref:`_some-ref` Perhaps that will fix it for you?

I wanted to add ref to the other file BoundsSafetyImplPlans.rst. I guess it should wait until we land this to have the url link?

No, you should be able to land both at the same time with the correct references I believe.

clang/docs/BoundsSafety.rst

rapidsna

Thanks @AaronBallman for your feedback! I left inlined comments to answer these and updated documents.

rapidsna · 2023-12-11T00:53:54Z

clang/docs/BoundsSafety.rst

+
+The requirement to annotate all pointers with explicit bounds information could present a significant adoption burden. To tackle this issue, the model incorporates the concept of a “wide pointer” (a.k.a. fat pointer) – a larger pointer that carries bounds information alongside the pointer value. Utilizing wide pointers can potentially reduce the adoption burden, as it contains bounds information internally and eliminates the need for explicit bounds annotations. However, wide pointers differ from standard C pointers in their data layout, which may result in incompatibilities with the application binary interface (ABI). Breaking the ABI complicates interoperability with external code that has not adopted the same programming model.
+
+``-fbounds-safety`` harmonizes the wide pointer and the bounds annotation approaches to reduce the adoption burden while maintaining the ABI. In this model, local variables of pointer type are implicitly treated as wide pointers, allowing them to carry bounds information without requiring explicit bounds annotations. This approach does not impact the ABI, as local variables are hidden from the ABI. Pointers associated with any other variables are treated as single object pointers (i.e., ``__single``), ensuring that they always have the tightest bounds by default and offering a strong bounds safety guarantee.


There are multiple things going on here.

Firstly, function parameters are on the ABI surface, so they are __single by default (not __bidi_indexable) unless otherwise annotated. I'll clarify local variables are implicitly wide pointers, but function parameters are not.

Secondly, the type of p is int *__counted_by(count) which is dependent on count, another parameter of func. Applying this exact type on another variable will create another dependency with the parameter count, which is undesirable in most cases. Function hmm in this example, doesn't have count to use for typeof and even if it did, it's not the same count on typeof(p). Thus, the compiler will report an error for using typeof on type with any dependent variable. On the other hand, typeof(q) where q is of type int *__counted_by(4) will be allowed as the type doesn't depend on another variable.

Therefore, it behaves as the following.

void func(int *__counted_by(count) p, size_t count) { extern void hmm(typeof(p)); // error: `typeof` on a type with a dependent variable } void func(int *p, size_t count) { // `p` is a parameter so it's `__single` by default extern void hmm(typeof(p)); // creates an interface of type `extern void hmm(int *__single);` } void func(int *__counted_by(4) p) { // `p` is annotated with `__counted_by(4) so it's not a wide pointer extern void hmm(typeof(p)); // creates an interface of type `extern void hmm(int *__counted_by(4));` }

As you pointed out, however, a local variable isn't totally hidden from the ABI when we consider a non-parameter local variable here:

void func(void) { int *p; // implicitly `int *__bidi_indexable p` extern void hmm(typeof(p)); // creates an interface of type `void hmm(int *__bidi_indexable)` }

And this is problematic because at the definition of function hmm, the parameter will be __single by default as any other parameters.

I think we should report a warning when typeof is used on an interface with an implicitly wide pointer to avoid silently breaking the ABI.

Section "ABI implications of default bounds annotations" is trying to discuss how -fbounds-safety tries to avoid similar issues. I'll update the section to discuss this case.

rapidsna · 2023-12-11T00:59:28Z

clang/docs/BoundsSafety.rst

+
+External bounds annotations include ``__counted_by``, ``__sized_by``, and ``__ended_by``. These annotations do not change the pointer representation, meaning they do not have ABI implications.
+
+* ``__counted_by(N)`` : The pointer points to memory that contains ``N`` elements of pointee type. ``N`` is an expression of integer type which can be a simple reference to declaration, a constant including calls to constant functions, or an arithmetic expression that does not have side effect. The annotation cannot apply to pointers to incomplete types or types without size such as ``void *``.


a constant including calls to constant functions -- only in C++, right?

And this document doesn't discuss the C++ specific model.

This includes a call to a function with __attribute__((const)) when the call arguments are all constants. If the function definition doesn't respect __attribute__((const)), how the count is evaluated will be an undefined behavior. I'll clarify the text.

rapidsna · 2023-12-11T01:12:43Z

clang/docs/BoundsSafety.rst

+
+External bounds annotations include ``__counted_by``, ``__sized_by``, and ``__ended_by``. These annotations do not change the pointer representation, meaning they do not have ABI implications.
+
+* ``__counted_by(N)`` : The pointer points to memory that contains ``N`` elements of pointee type. ``N`` is an expression of integer type which can be a simple reference to declaration, a constant including calls to constant functions, or an arithmetic expression that does not have side effect. The annotation cannot apply to pointers to incomplete types or types without size such as ``void *``.


The annotation cannot apply to pointers to incomplete types or types without size such as void *.
We support a GCC extension that treats void * as being iterable as if it were a char *: > https://godbolt.org/z/q9fqG5nrY so should we allow bounds info on void * or should we not > allow it and explain that we want people to avoid using void * that way in this model?

void * should instead use __sized_by. As described in the next bullet, the annotation indicates the byte size. I'll clarify the __counted_by bullet to avoid confusion.

rapidsna · 2023-12-11T01:22:16Z

clang/docs/BoundsSafety.rst

+
+   void null_with_count_10(int *__counted_by(count) buf, unsigned count) {
+   buf = 0;
+   count = 10; // This is not allowed as it creates a null pointer with non-zero length


You can still change the value of count when buf is also updated together, and the compiler will insert a run-time check to make sure the updated count and buf are still valid.

rapidsna · 2023-12-11T01:27:14Z

clang/docs/BoundsSafety.rst

+* ``__indexable`` : A pointer with this annotation becomes a wide pointer carrying the upper bound (but no explicit lower bound), the layout of which is equivalent to ``struct { T *ptr; T *upper_bound; };``. Since ``__indexable`` pointers do not have a separate lower bound, the pointer value itself acts as the lower bound. An ``__indexable`` pointer can only be incremented or indexed in the positive direction. Decrementing it with a known negative index triggers a compile-time error. Otherwise, the compiler inserts a run-time check to ensure pointer arithmetic doesn’t make the pointer smaller than the original ``__indexable`` pointer (Note that ``__indexable`` doesn’t have a lower bound so the pointer value is effectively the lower bound). As pointer arithmetic overflow will make the pointer smaller than the original pointer, it will cause a trap at runtime. Similar to ``__bidi_indexable``, an ``__indexable`` pointer is allowed to have a pointer value above the upper bound and creating such a pointer is well-defined behavior. Dereferencing such a pointer, however, will cause a run-time trap.
+* ``__bidi_indexable`` offers the best flexibility out of all the pointer annotations in this model, as ``__bidi_indexable`` pointers can be used for any pointer operation. However, this comes with the largest code size and memory cost out of the available pointer annotations in this model. In some cases, use of the ``__bidi_indexable`` annotation may be duplicating bounds information that exists elsewhere in the program. In such cases, using external bounds annotations may be a better choice.
+
+``__bidi_indexable`` is the default annotation for non-ABI visible pointers, such as local pointer variables — that is, if the programmer does not specify another bounds annotation, a local pointer variable is implicitly ``__bidi_indexable``. Since ``__bidi_indexable`` pointers automatically carry bounds information and have no restrictions on kinds of pointer operations that can be used with these pointers, most code inside a function works as is without modification. In the example below, ``int *buf`` doesn’t require manual annotation as it’s implicitly ``int *__bidi_indexable buf``, carrying the bounds information passed from the return value of malloc, which is necessary to insert bounds checking for ``buf[i]``.


Responded in the earlier question.

rapidsna · 2023-12-12T04:22:27Z

clang/docs/BoundsSafety.rst

+* ``__unsafe_indexable`` : A pointer with this annotation behaves the same as a plain C pointer. That is, the pointer does not have any bounds information and pointer operations are not checked.
+* ``__unsafe_indexable`` can be used to mark pointers from system headers or pointers from code that has not adopted -fbounds safety. This enables interoperation between code using ``-fbounds-safety`` and code that does not.


rapidsna · 2023-12-12T04:22:34Z

clang/docs/BoundsSafety.rst

+Requiring ``-fbounds-safety`` adopters to add bounds annotations to all pointers in the codebase would be a significant adoption burden. To avoid this and to secure all pointers by default, ``-fbounds-safety`` applies default bounds annotations to pointer types.
+
+Default annotations apply to pointer types of declarations


rapidsna · 2023-12-12T04:26:33Z

clang/docs/BoundsSafety.rst

+
+``-fbounds-safety`` applies default bounds annotations to pointer types used in declarations. The default annotations are determined by the ABI visibility of the pointer. A pointer type is ABI-visible if changing its size or representation affects the ABI. For instance, changing the size of a type used in a function parameter will affect the ABI and thus pointers used in function parameters are ABI-visible pointers. On the other hand, changing the types of local variables won’t have such ABI implications. Hence, ``-fbounds-safety`` considers the outermost pointer types of local variables as non-ABI visible. The rest of the pointers such as nested pointer types, pointer types of global variables, struct fields, and function prototypes are considered ABI-visible.
+
+All ABI-visible pointers are treated as ``__single`` by default unless annotated otherwise. This default both preserves ABI and makes these pointers safe by default. This behavior can be controlled with pragma to set the default annotation for ABI-visible pointers to be either ``__single``, ``__bidi_indexable``, ``__indexable``, or ``__unsafe_indexable``. For instance, ``__ptrcheck_abi_assume_unsafe_indexable()`` will make all ABI-visible pointers be ``__unsafe_indexable``.


I converted the text from pragma .. to macros, i.e., __ptrcheck_abi_assume_*ATTR*().

And added this line at the end of the paragraph __ptrcheck_abi_assume_*ATTR*() macros are defined as pragmas in the toolchain header with a code block how the macros are defined.

rapidsna · 2023-12-12T04:27:58Z

clang/docs/BoundsSafety.rst

+
+All ABI-visible pointers are treated as ``__single`` by default unless annotated otherwise. This default both preserves ABI and makes these pointers safe by default. This behavior can be controlled with pragma to set the default annotation for ABI-visible pointers to be either ``__single``, ``__bidi_indexable``, ``__indexable``, or ``__unsafe_indexable``. For instance, ``__ptrcheck_abi_assume_unsafe_indexable()`` will make all ABI-visible pointers be ``__unsafe_indexable``.
+Non-ABI visible pointers — the outermost pointer types of local variables — are ``__bidi_indexable`` by default, so that these pointers have the bounds information necessary to perform bounds checks without the need for a manual annotation.
+All ``const char`` pointers are ``__null_terminated`` by default.


I extended the paragraph to document how it applies to wchar_t, char8_t, or other general typedefs of characters.

rapidsna · 2023-12-12T04:42:54Z

clang/docs/BoundsSafety.rst

+Default pointer types in ``sizeof()``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A pointer type in ``sizeof()`` does not have an implicit bounds annotation. When a bounds attribute is not specified, the evaluated pointer type is treated identically to a plain C pointer type. Therefore, ``sizeof(int*)`` remains the same with or without ``-fbounds-safety``. That said, programmers can explicitly add attribute to the types, e.g., ``sizeof(int *__bidi_indexable)``, in which case the sizeof evaluates to the size of type ``int *__bidi_indexable`` (the value equivalent to ``3 * sizeof(int*)``).


I added section Default pointer types in alignof() next to the sizeof section to explain this.

AaronBallman

Thank you for the updates; the changes LGTM!

AaronBallman · 2024-01-12T14:48:09Z

clang/docs/BoundsSafety.rst

+subject to be actively updated with a more detailed specification. The
+implementation plan can be found in Implementation plans for -fbounds-safety.
+
+.. Cross reference doesn't currently work


No, you should be able to land both at the same time with the correct references I believe.

AaronBallman · 2024-01-12T14:55:18Z

clang/docs/BoundsSafety.rst

+
+In C, when an array is referenced, it is automatically promoted (or “decayed”) to a pointer to its first element (e.g., ``&arr[0]``). Similarly, in ``-fbounds-safety``, arrays are also promoted to pointers, but with the addition of an implicit bounds annotation. Arrays on function parameters are promoted to corresponding ``__counted_by`` pointers. Consequently, incomplete arrays (or arrays without size) will cause a compiler error unless it has ``__counted_by`` annotation in its bracket. All other arrays are promoted to ``__bidi_indexable`` pointers, with the equivalent of ``&arr[0]`` serving as the lower bound and ``&arr[array_size]`` (or one past the last element) serving as the upper bound. This way, all array accesses are subject to bounds checking, just as their corresponding pointers are.
+
+Maintaining correctness of bounds annotations


Thanks, that's great!

rapidsna · 2024-01-12T17:46:14Z

Thank you for the updates; the changes LGTM!

@AaronBallman Thanks for your review and insightful feedback!

The document is mostly the exact copy of RFC: Enforcing Bounds Safety in C, except some minor adjustments in the tone to make it more suitable for documentation: https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-c-fbounds-safety/70854 Further changes and clarifications for the programming model will be done as separate patches to make it easier to track history of changes.

grammar Co-authored-by: Adrian Prantl <adrian-prantl@users.noreply.github.com>

rapidsna · 2024-01-12T18:16:59Z

Rebased

The document is mostly the exact copy of RFC: Enforcing Bounds Safety in C, except some clarifications made over the PR review: https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-c-fbounds-safety/70854 Further changes and clarifications for the programming model will be done as separate patches to make it easier to track history of changes.

llvmbot added the clang Clang issues not falling into any other category label Oct 30, 2023

rapidsna mentioned this pull request Oct 31, 2023

[Driver][BoundsSafety] Add -fexperimental-bounds-safety flag #70480

Open

rapidsna marked this pull request as draft October 31, 2023 00:04

delcypher added documentation clang:bounds-safety Issue/PR relating to the experimental -fbounds-safety feature in Clang labels Nov 3, 2023

delcypher requested review from delcypher and hnrklssn November 3, 2023 21:37

delcypher approved these changes Nov 3, 2023

View reviewed changes

rapidsna self-assigned this Nov 29, 2023

rapidsna requested review from kees, nickdesaulniers, devincoughlin, AaronBallman and bwendling November 29, 2023 09:52

AaronBallman reviewed Nov 29, 2023

View reviewed changes

rapidsna requested review from apple-fcloutier, fhahn and adrian-prantl November 29, 2023 19:17

rapidsna commented Nov 30, 2023

View reviewed changes

clang/docs/BoundsSafety.rst Show resolved Hide resolved

rapidsna marked this pull request as ready for review November 30, 2023 07:26

adrian-prantl approved these changes Nov 30, 2023

View reviewed changes

clang/docs/BoundsSafetyImplPlans.rst Outdated Show resolved Hide resolved

AaronBallman reviewed Dec 6, 2023

View reviewed changes

rapidsna commented Dec 12, 2023

View reviewed changes

rapidsna mentioned this pull request Dec 18, 2023

[Clang] Generate the GEP instead of adding AST nodes #73730

Closed

rapidsna requested a review from AaronBallman December 18, 2023 23:48

AaronBallman approved these changes Jan 12, 2024

View reviewed changes

rapidsna force-pushed the bounds-safety-doc branch from e4cd70c to 6fe79d2 Compare January 12, 2024 18:14

rapidsna and others added 9 commits January 12, 2024 10:15

Separate design and implementation plan documents; add monospacing

dbd5fbb

Apply word wrap for column 80

443a3ae

Address Aaron's feedback

ad35d05

Update clang/docs/BoundsSafetyImplPlans.rst

98a268c

grammar Co-authored-by: Adrian Prantl <adrian-prantl@users.noreply.github.com>

replace smart quotes with default quotes

add356e

Address quick comments from Aaron

1aee64e

Update document for clarifications

fbf5360

Fix reference to BoundsSafetyImplPlans.rst

659fa47

rapidsna force-pushed the bounds-safety-doc branch from 6fe79d2 to 659fa47 Compare January 12, 2024 18:16

rapidsna merged commit dda2ce8 into llvm:main Jan 16, 2024
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BoundsSafety] Initial documentation for -fbounds-safety #70749

[BoundsSafety] Initial documentation for -fbounds-safety #70749

rapidsna commented Oct 30, 2023

delcypher left a comment

delcypher Nov 3, 2023

delcypher commented Nov 7, 2023

AaronBallman left a comment

AaronBallman Nov 29, 2023

rapidsna Nov 30, 2023

AaronBallman Dec 6, 2023

rapidsna Dec 11, 2023

AaronBallman Jan 12, 2024

adrian-prantl left a comment

AaronBallman Dec 6, 2023

AaronBallman left a comment

AaronBallman Dec 6, 2023

rapidsna Dec 12, 2023

AaronBallman Jan 12, 2024

rapidsna left a comment

rapidsna Dec 11, 2023

rapidsna Dec 11, 2023

rapidsna Dec 11, 2023

rapidsna Dec 11, 2023

rapidsna Dec 11, 2023

rapidsna Dec 12, 2023

rapidsna Dec 12, 2023

rapidsna Dec 12, 2023

rapidsna Dec 12, 2023

rapidsna Dec 12, 2023

AaronBallman left a comment

AaronBallman Jan 12, 2024

AaronBallman Jan 12, 2024

rapidsna commented Jan 12, 2024

rapidsna commented Jan 12, 2024


		Using this bounds information, the compiler inserts bounds checks on every pointer dereference, ensuring that the program does not access memory outside the specified bounds. The compiler requires programmers to provide enough bounds information so that the accesses can be checked at either run time or compile time — and it rejects code if it cannot.

		The most important contribution of “-fbounds-safety” is how it reduces the programmer’s annotation burden by reconciling bounds annotations at ABI boundaries with the use of implicit wide pointers (a.k.a. “fat” pointers) that carry bounds information on local variables without the need for annotations. We designed this model so that it preserves ABI compatibility with C while minimizing adoption effort.


		In C, when an array is referenced, it is automatically promoted (or “decayed”) to a pointer to its first element (e.g., ``&arr[0]``). Similarly, in ``-fbounds-safety``, arrays are also promoted to pointers, but with the addition of an implicit bounds annotation. Arrays on function parameters are promoted to corresponding ``__counted_by`` pointers. Consequently, incomplete arrays (or arrays without size) will cause a compiler error unless it has ``__counted_by`` annotation in its bracket. All other arrays are promoted to ``__bidi_indexable`` pointers, with the equivalent of ``&arr[0]`` serving as the lower bound and ``&arr[array_size]`` (or one past the last element) serving as the upper bound. This way, all array accesses are subject to bounds checking, just as their corresponding pointers are.

		Maintaining correctness of bounds annotations


		The requirement to annotate all pointers with explicit bounds information could present a significant adoption burden. To tackle this issue, the model incorporates the concept of a “wide pointer” (a.k.a. fat pointer) – a larger pointer that carries bounds information alongside the pointer value. Utilizing wide pointers can potentially reduce the adoption burden, as it contains bounds information internally and eliminates the need for explicit bounds annotations. However, wide pointers differ from standard C pointers in their data layout, which may result in incompatibilities with the application binary interface (ABI). Breaking the ABI complicates interoperability with external code that has not adopted the same programming model.

		``-fbounds-safety`` harmonizes the wide pointer and the bounds annotation approaches to reduce the adoption burden while maintaining the ABI. In this model, local variables of pointer type are implicitly treated as wide pointers, allowing them to carry bounds information without requiring explicit bounds annotations. This approach does not impact the ABI, as local variables are hidden from the ABI. Pointers associated with any other variables are treated as single object pointers (i.e., ``__single``), ensuring that they always have the tightest bounds by default and offering a strong bounds safety guarantee.


		External bounds annotations include ``__counted_by``, ``__sized_by``, and ``__ended_by``. These annotations do not change the pointer representation, meaning they do not have ABI implications.

		* ``__counted_by(N)`` : The pointer points to memory that contains ``N`` elements of pointee type. ``N`` is an expression of integer type which can be a simple reference to declaration, a constant including calls to constant functions, or an arithmetic expression that does not have side effect. The annotation cannot apply to pointers to incomplete types or types without size such as ``void *``.

		* ``__unsafe_indexable`` : A pointer with this annotation behaves the same as a plain C pointer. That is, the pointer does not have any bounds information and pointer operations are not checked.
		* ``__unsafe_indexable`` can be used to mark pointers from system headers or pointers from code that has not adopted -fbounds safety. This enables interoperation between code using ``-fbounds-safety`` and code that does not.

		Requiring ``-fbounds-safety`` adopters to add bounds annotations to all pointers in the codebase would be a significant adoption burden. To avoid this and to secure all pointers by default, ``-fbounds-safety`` applies default bounds annotations to pointer types.

		Default annotations apply to pointer types of declarations


		``-fbounds-safety`` applies default bounds annotations to pointer types used in declarations. The default annotations are determined by the ABI visibility of the pointer. A pointer type is ABI-visible if changing its size or representation affects the ABI. For instance, changing the size of a type used in a function parameter will affect the ABI and thus pointers used in function parameters are ABI-visible pointers. On the other hand, changing the types of local variables won’t have such ABI implications. Hence, ``-fbounds-safety`` considers the outermost pointer types of local variables as non-ABI visible. The rest of the pointers such as nested pointer types, pointer types of global variables, struct fields, and function prototypes are considered ABI-visible.

		All ABI-visible pointers are treated as ``__single`` by default unless annotated otherwise. This default both preserves ABI and makes these pointers safe by default. This behavior can be controlled with pragma to set the default annotation for ABI-visible pointers to be either ``__single``, ``__bidi_indexable``, ``__indexable``, or ``__unsafe_indexable``. For instance, ``__ptrcheck_abi_assume_unsafe_indexable()`` will make all ABI-visible pointers be ``__unsafe_indexable``.

[BoundsSafety] Initial documentation for -fbounds-safety #70749

[BoundsSafety] Initial documentation for -fbounds-safety #70749

Conversation

rapidsna commented Oct 30, 2023

delcypher left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

delcypher commented Nov 7, 2023

AaronBallman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adrian-prantl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AaronBallman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rapidsna left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AaronBallman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rapidsna commented Jan 12, 2024

rapidsna commented Jan 12, 2024