From 1ed33ef32424bee2c7e4b2a2106a4a7b41f99944 Mon Sep 17 00:00:00 2001 From: Alistair Francis Date: Thu, 3 Nov 2022 10:42:55 +1000 Subject: [PATCH 1/9] doc: Add ExternalDependencies.md Signed-off-by: Alistair Francis --- doc/Design.md | 35 ++++++++-------------- doc/ExternalDependencies.md | 60 +++++++++++++++++++++++++++++++++++++ 2 files changed, 73 insertions(+), 22 deletions(-) create mode 100644 doc/ExternalDependencies.md diff --git a/doc/Design.md b/doc/Design.md index 0dc13c2b0c..483cccdad9 100644 --- a/doc/Design.md +++ b/doc/Design.md @@ -13,7 +13,7 @@ Tock Design - [In-Kernel Design Principles](#in-kernel-design-principles) * [Role of HILs](#role-of-hils) * [Split-phase Operation](#split-phase-operation) - * [No External Dependencies](#no-external-dependencies) + * [External Dependencies](#external-dependencies) * [Using `unsafe` and Capabilities](#using-unsafe-and-capabilities) * [Ease of Use and Understanding](#ease-of-use-and-understanding) * [Demonstrated Features](#demonstrated-features) @@ -289,27 +289,18 @@ that the operation is synchronous. These cases are rare, though: the operation has to be so fast that it's not worth allowing other code to run during the delay. -### No External Dependencies - -Tock chooses to not use any external libraries for any of the crates in the -kernel. This is done to promote safety, as auditing the Tock code only requires -inspecting the code in the Tock repository. Tock tries to be very specific with -its use of `unsafe`, and tries to ensure that when it is used it is clear as to -why. With external dependencies it would be significantly more challenging to -ensure that uses of `unsafe` are valid, particularly as external libraries -evolve. - -We also realize, however, that external libraries can be very useful. Tock's -compromise has been to pull in specific portions of libraries into the -`libraries` folder. This puts the library's source in the same repository, while -keeping the library as a clearly separate crate. We do try to limit how often -this happens. - -In the future, we hope that `cargo` and other Rust tools make it significantly -easier to audit and manage dependencies. For example, cargo currently has no -mechanism to emit an error if a dependency uses `unsafe`. If new tools emerge -that help ensure that dependent code is safe, Tock would likely be able to -leverage external dependencies. +### External Dependencies + +Generally it's best to avoid adding any external crates to the Tock project. + +In certain situations Tock does allow external depenencies. This is decided on +a case by case basis. For more details on this see +[External Dependencies](ExternalDependencies.md). + +The other option to add an external library is to put specific portions of +libraries into the `libraries` folder. This puts the library's source in the +same repository, while keeping the library as a clearly separate crate. We do +try to limit how often this happens. ### Using `unsafe` and Capabilities diff --git a/doc/ExternalDependencies.md b/doc/ExternalDependencies.md new file mode 100644 index 0000000000..b5186d5518 --- /dev/null +++ b/doc/ExternalDependencies.md @@ -0,0 +1,60 @@ +External Dependencies +===================== + + + + + +- [Adding a dependency](#adding-a-dependency) + * [Provide Important Functionality](#provide-important-functionality) + * [Project maturity](#project-maturity) + * [Limited dependencies](#limited-dependencies) +- [Dependency Tree](#dependency-tree) + + + +## Adding a dependency + +External dependencies can be added to Tock on a case by case basis. Each +dependency will be reviewed for inclusion. Below is a general list of +requirements for adding a new external crate. + +### Provide Important Functionality + +The external crate must provide important functionality that couldn't +easily or realistically be provided by the Tock developers. + +Examples of this include: + + * Crypto libraries + * No one should write their own crypto library + * Wireless protocols + * Wireless implementations are difficult to get the correct timing + * Wireless protocols are also very expensive to certify + * Rust embedded HALs + * For some platforms there are mature Rust Embedded HALs. This would take + a long time for Tock developers to re-implement themselves. + +### Project maturity + +The external crate being added must be a mature project, with a high quality +of code. The project must be well regarded in the Rust community. + +### Limited dependencies + +The external crate should include only a few dependencies. The fewer +dependencies the crate introduces the more likely it is to be accepted. + +## Dependency Tree + +The Tock dependency tree can be generated by running the below command in the +top level of the Tock repo. + +```shell +cargo tree | grep -v /tock +``` + +The below tree should be updated whenever a dependency change is made. + +``` +``` From 2e77bb6885468e2081faf2d6777faa61c872da97 Mon Sep 17 00:00:00 2001 From: Brad Campbell Date: Tue, 15 Nov 2022 11:40:35 -0500 Subject: [PATCH 2/9] doc: expand ext. dep. instructions --- doc/ExternalDependencies.md | 155 ++++++++++++++++++++++++++++-------- 1 file changed, 123 insertions(+), 32 deletions(-) diff --git a/doc/ExternalDependencies.md b/doc/ExternalDependencies.md index b5186d5518..477db44d62 100644 --- a/doc/ExternalDependencies.md +++ b/doc/ExternalDependencies.md @@ -5,56 +5,147 @@ External Dependencies -- [Adding a dependency](#adding-a-dependency) - * [Provide Important Functionality](#provide-important-functionality) - * [Project maturity](#project-maturity) - * [Limited dependencies](#limited-dependencies) -- [Dependency Tree](#dependency-tree) +- [Limited External Dependency Rationale](#limited-external-dependency-rationale) +- [External Dependency Selection](#external-dependency-selection) + * [Core External Dependencies](#core-external-dependencies) + + [Provide Important Functionality](#provide-important-functionality) + + [Project Maturity](#project-maturity) + + [Limited Sub-dependencies](#limited-sub-dependencies) + * [Board-Specific External Dependencies](#board-specific-external-dependencies) +- [Including the Dependency](#including-the-dependency) + * [Including Core External Dependencies](#including-core-external-dependencies) + * [Including Board-Specific External Dependencies](#including-board-specific-external-dependencies) + * [Documenting the Dependency and its Tree](#documenting-the-dependency-and-its-tree) -## Adding a dependency +Tock's general policy is the kernel does not include external dependencies (i.e. +rust crates outside of the `tock/tock` repository) that are not part of the Rust +standard library. However, on a limited, case-by-case basis with appropriate +safeguards, external dependencies can be used in the Tock kernel. The rationale +and policy for this is described in this document. -External dependencies can be added to Tock on a case by case basis. Each -dependency will be reviewed for inclusion. Below is a general list of -requirements for adding a new external crate. -### Provide Important Functionality + +## Limited External Dependency Rationale + +Tock limits its use of external libraries for all crates in the kernel. This is +done to promote safety, as auditing the Tock code only requires inspecting the +code in the Tock repository. Tock tries to be very specific with its use of +`unsafe`, and tries to ensure that when it is used it is clear as to why. With +external dependencies, verifying uses of `unsafe` are valid is more challenging +to, particularly as external libraries evolve. + +External dependencies also typically themselves rely on dependencies, so +including one external crate likely pulls in several external crate. As of Nov +2022, cargo provides no mechanism for auditing and prohibiting `unsafe` in a +dependency hierarchy. Also, the dependency chain for an external crate is +largely hidden from developers using the external crate. Lacking automated +tools, managing dependencies is a manual process, and to limit overhead Tock +generally avoids external dependencies. + + +## External Dependency Selection + +External dependencies can be added to Tock on a case-by-case basis. Each +dependency will be reviewed for inclusion, according to the criteria in this +section. The requirements are intentionally strict. + +There are two general methods to for including an external dependency in the +Tock kernel: core external dependencies, and board-specific external +dependencies. Core external dependencies may be used in "core" Tock crates, such +as the `kernel`, `chips`, and `capsules` crates. Board-specific external +dependencies may _only_ be used by crates in the `board/` folder. The processes +for inclusion between these two methods are different. + +### Core External Dependencies + +There are well-specified requirements for including a core external dependency. + +#### Provide Important Functionality The external crate must provide important functionality that couldn't easily or realistically be provided by the Tock developers. -Examples of this include: +The list of currently accepted important functionality: - * Crypto libraries - * No one should write their own crypto library - * Wireless protocols - * Wireless implementations are difficult to get the correct timing - * Wireless protocols are also very expensive to certify - * Rust embedded HALs - * For some platforms there are mature Rust Embedded HALs. This would take - a long time for Tock developers to re-implement themselves. +* Cryptography libraries. Writing cryptographically secure code that is both + correct and resistant to attacks is challenging. Leveraging validated, + high-quality cryptographic libraries instead of Tock-specific cryptographic + code increases the security of the Tock kernel. -### Project maturity +#### Project Maturity The external crate being added must be a mature project, with a high quality of code. The project must be well regarded in the Rust community. -### Limited dependencies +The top-level external crate must belong to one of the following set of +repository organizations: + +* [RustCrypto](https://github.com/RustCrypto) + +#### Limited Sub-dependencies + +The external crate should have a limited sub-dependency tree. The fewer +dependencies the crate introduces the more likely it is to be accepted. There is +no set threshold, instead this is evaluated on a case-by-case basis. + + +### Board-Specific External Dependencies + +As board crates (i.e. crates in the `boards/` directory) are generally regarded +as use-case specific, managed by specific board maintainers, and audited by the +specific board maintainers, Tock is more flexible with including external +dependencies in board crates. + +Examples of when a board may want to use an external library: + +* Wireless protocols. + * Wireless implementations are difficult to get the correct timing. + * Wireless protocols are also very expensive to certify. + +Note, however, that _only_ the crate in `boards/` may include an external +dependency in its `Cargo.toml` file. Other crates in the kernel must not include +the dependency, specifically a chip crate or the capsules crate. Therefore, a +wrapper must be provided to interface Tock kernel code with the external +dependency. This prevents external APIs from leaking directly into the Tock +kernel. + + +## Including the Dependency + +To help ensure maintainability and to promote transparency with including +external dependencies, Tock follows a specific policy for their inclusion. + +### Including Core External Dependencies + +The only crate that may specify an external dependency in its `Cargo.toml` file +is the `tockextern` crate. This crate solely exists to house external +dependencies, and only re-exports the dependencies within the `tockextern` +namespace. + +Other crates in the Tock kernel then include the `tockextern` crate, and use the +dependencies through that namespace. -The external crate should include only a few dependencies. The fewer -dependencies the crate introduces the more likely it is to be accepted. +This removes any ambiguity about when an external dependency is used throughout +the Tock kernel, and makes such usages easier to search for. As Rust allows +external dependencies to be used in a source file with the same syntax as other +modules in the same crate, determining which code is external and which is part +of the Tock kernel is difficult without deeper inspection. By including our own +namespace we force the source file to include `use tockextern::` when using an +external dependency. -## Dependency Tree +### Including Board-Specific External Dependencies -The Tock dependency tree can be generated by running the below command in the -top level of the Tock repo. +Boards may include external dependencies directly in their board's `Cargo.toml` +file and use them directly. -```shell -cargo tree | grep -v /tock -``` +### Documenting the Dependency and its Tree -The below tree should be updated whenever a dependency change is made. +Each crate that includes an external dependency in its `Cargo.toml` file must +include a section titled "External Dependencies" in its README. Each external +dependency must be listed along with its dependency tree. This documentation +must be included in the PR that adds the external dependency. -``` -``` +The Tock dependency tree can be generated by running `cargo tree`. The tree +should be updated whenever a dependency change is made. From 89303d7aeaf79bc7cc6630ec138faf1ab5e7c328 Mon Sep 17 00:00:00 2001 From: Brad Campbell Date: Tue, 15 Nov 2022 11:42:58 -0500 Subject: [PATCH 3/9] doc: add ext. dep. link to top level readme --- doc/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/README.md b/doc/README.md index 0c54f6c1ac..d30fdc9a87 100644 --- a/doc/README.md +++ b/doc/README.md @@ -27,6 +27,7 @@ Tock Guides - **[Userland](Userland.md)** - Description of userland applications. - **[Networking Stack](Networking_Stack.md)** - Design of the networking stack in Tock. - **[Configuration](Configuration.md)** - Configuration options for the kernel. +- **[External Dependencies](ExternalDependencies.md)** - Policy for including external dependencies. ### Interface Details - **[Syscall Interfaces](syscalls)** - API between userland and the kernel. From 40c33b89b615203ff7702f196caed5b231715701 Mon Sep 17 00:00:00 2001 From: Brad Campbell Date: Wed, 26 Apr 2023 10:42:32 -0400 Subject: [PATCH 4/9] doc: external: add goals/motivation --- doc/ExternalDependencies.md | 47 +++++++++++++++++++++++++++++++++++-- 1 file changed, 45 insertions(+), 2 deletions(-) diff --git a/doc/ExternalDependencies.md b/doc/ExternalDependencies.md index 477db44d62..d62be6ab2b 100644 --- a/doc/ExternalDependencies.md +++ b/doc/ExternalDependencies.md @@ -5,7 +5,9 @@ External Dependencies -- [Limited External Dependency Rationale](#limited-external-dependency-rationale) +- [External Dependency Design](#external-dependency-design) + * [Rationale](#rationale) + * [Specific Approach Motivation](#specific-approach-motivation) - [External Dependency Selection](#external-dependency-selection) * [Core External Dependencies](#core-external-dependencies) + [Provide Important Functionality](#provide-important-functionality) @@ -27,7 +29,13 @@ and policy for this is described in this document. -## Limited External Dependency Rationale +## External Dependency Design + +This document describes both Tock's external dependency policy and mechanism, as +well as the rationale behind the approach. + + +### Rationale Tock limits its use of external libraries for all crates in the kernel. This is done to promote safety, as auditing the Tock code only requires inspecting the @@ -44,6 +52,41 @@ largely hidden from developers using the external crate. Lacking automated tools, managing dependencies is a manual process, and to limit overhead Tock generally avoids external dependencies. +### Specific Approach Motivation + +The mechanism for including external dependencies is designed to satisfy the +following goals. These goals were converged upon over multiple discussions of +the Tock developers. + +Goals: + +- Boards which do not need or want the functionality provided by the external + dependency can ensure the dependency is not included in the kernel build. +- Boards which do not use the dependency do not have to compile the dependency. +- Boards should have discretion on which code to include in their build. +- All uses of the external dependency in the Tock code base are explicit and + obvious. +- The location within the Tock code tree for external dependencies is clear and + consistent, and there is a consistent format to document the dependency. +- There is not undue overhead or boilerplate required to add an external + dependency. + +These goals necessitate a few design decisions. For example, as crates are the +smallest unit of compilation in Rust, external dependencies must be included +through new crates added to the Tock source tree so they can be individually +included or excluded in specific builds. Also, crates provide a namespace to use +to identify when external dependencies are being incorporated. + +Additionally, we avoid using traits or HIL-like interfaces for dependencies +(i.e. core Tock capsules/modules would use a Tock-defined trait much like +capsules use HILs, and a wrapper would use the external dependency to implement +the trait) to avoid the overhead of implementing and maintaining a wrapper to +implement the trait. While architecturally this has advantages, the overhead was +deemed too burdensome for the expected benefit. + +We explicitly document the goals to help motivate the specific design in the +remainder of this document. Also, this policy may change in the future, but +these goals should be considered in any future updates. ## External Dependency Selection From 884caaab3a25938561ba38f57aced43cf4b3de7c Mon Sep 17 00:00:00 2001 From: Brad Campbell Date: Wed, 26 Apr 2023 10:50:31 -0400 Subject: [PATCH 5/9] doc: external dep: updated policy --- doc/ExternalDependencies.md | 22 ++++++++-------------- 1 file changed, 8 insertions(+), 14 deletions(-) diff --git a/doc/ExternalDependencies.md b/doc/ExternalDependencies.md index d62be6ab2b..83512e9d16 100644 --- a/doc/ExternalDependencies.md +++ b/doc/ExternalDependencies.md @@ -162,20 +162,14 @@ external dependencies, Tock follows a specific policy for their inclusion. ### Including Core External Dependencies -The only crate that may specify an external dependency in its `Cargo.toml` file -is the `tockextern` crate. This crate solely exists to house external -dependencies, and only re-exports the dependencies within the `tockextern` -namespace. - -Other crates in the Tock kernel then include the `tockextern` crate, and use the -dependencies through that namespace. - -This removes any ambiguity about when an external dependency is used throughout -the Tock kernel, and makes such usages easier to search for. As Rust allows -external dependencies to be used in a source file with the same syntax as other -modules in the same crate, determining which code is external and which is part -of the Tock kernel is difficult without deeper inspection. By including our own -namespace we force the source file to include `use tockextern::` when using an +The only crates that can contain external dependencies must be inside the +`capsules/` directory. For each new dependency a new crate should be added +within the `capsules/` directory. Only that crate can have the external +dependency. + +The only crates that can use the newly created external dependency crates are +board crates in the `boards/` directory. This ensures that boards which do not +want to include the external dependency can avoid including the crate with the external dependency. ### Including Board-Specific External Dependencies From 65b3674bdad3de1e9426f385a12ac5314550cb6a Mon Sep 17 00:00:00 2001 From: Leon Schuermann Date: Mon, 1 May 2023 11:44:53 -0400 Subject: [PATCH 6/9] doc/ExternalDependencies: update policy to reflect discussion on calls --- doc/ExternalDependencies.md | 120 +++++++++++++++++++++++++++--------- 1 file changed, 92 insertions(+), 28 deletions(-) diff --git a/doc/ExternalDependencies.md b/doc/ExternalDependencies.md index 83512e9d16..747976bde1 100644 --- a/doc/ExternalDependencies.md +++ b/doc/ExternalDependencies.md @@ -88,6 +88,36 @@ We explicitly document the goals to help motivate the specific design in the remainder of this document. Also, this policy may change in the future, but these goals should be considered in any future updates. +### Dependency Structure of Tock-Internal Crates + +Following from the above, an external dependency added to a crate which is +depended on internally within Tock (e.g. the `kernel` crate) will have a higher +impact than a dependency added to a crate with no reverse dependencies (e.g. a +board crate). Thus, this policy is increasingly liberal with crate-types that +have fewer reverse dependencies. + +This document considers Tock's crate structure by referring to the following +types of crates internal to Tock: + +- the kernel crate: `kernel/` +- arch crates: crates in the `arch/` directory +- chip crates: crates in the `chips/` directory +- board crates: crates in the `boards/` directory +- capsule crates: crates in the `capsules/` directory + +Furthermore, this policy assumes the following rules regarding crate +dependencies internal to Tock: + +- a _board crate_ is not a dependency of any other Tock-internal crate +- a _chip crate_ is only a dependency of _board crates_ or other + _chip crates_ +- a _capsule crate_ is only a dependency of other _capsule crates_ or _board + crates_ +- an _arch crate_ may only depend on the _kernel crate_ and other + _arch crates_ +- the _kernel crate_ does not depend on _arch_, _chip_, _board_, or _capsule + crates_ + ## External Dependency Selection External dependencies can be added to Tock on a case-by-case basis. Each @@ -95,11 +125,19 @@ dependency will be reviewed for inclusion, according to the criteria in this section. The requirements are intentionally strict. There are two general methods to for including an external dependency in the -Tock kernel: core external dependencies, and board-specific external -dependencies. Core external dependencies may be used in "core" Tock crates, such -as the `kernel`, `chips`, and `capsules` crates. Board-specific external -dependencies may _only_ be used by crates in the `board/` folder. The processes -for inclusion between these two methods are different. +Tock kernel: core external dependencies, and capsule- or board-specific +external dependencies. + +- Core external dependencies may be used in "core" Tock crates, such as the + _kernel_, _arch_, and _chip crates_. Notably, individual boards generally do + not have a reasonable choice whether to depend on those crates. + +- Capsule-, or board-specific external dependencies may _only_ be in the + respective `boards/`-, or `chips/`-crates, or in a `capsules/`-crate that is + _not_ + + - `capsules/core`, or + - `capsules/extra`. ### Core External Dependencies @@ -107,7 +145,7 @@ There are well-specified requirements for including a core external dependency. #### Provide Important Functionality -The external crate must provide important functionality that couldn't +The external crate must provide important functionality that could not easily or realistically be provided by the Tock developers. The list of currently accepted important functionality: @@ -133,13 +171,11 @@ The external crate should have a limited sub-dependency tree. The fewer dependencies the crate introduces the more likely it is to be accepted. There is no set threshold, instead this is evaluated on a case-by-case basis. - ### Board-Specific External Dependencies -As board crates (i.e. crates in the `boards/` directory) are generally regarded -as use-case specific, managed by specific board maintainers, and audited by the -specific board maintainers, Tock is more flexible with including external -dependencies in board crates. +As board crates are generally regarded as use-case specific, managed by specific +chip and board maintainers, and audited by those maintainers, Tock is more +flexible with including external dependencies in those crates. Examples of when a board may want to use an external library: @@ -147,36 +183,64 @@ Examples of when a board may want to use an external library: * Wireless implementations are difficult to get the correct timing. * Wireless protocols are also very expensive to certify. -Note, however, that _only_ the crate in `boards/` may include an external -dependency in its `Cargo.toml` file. Other crates in the kernel must not include -the dependency, specifically a chip crate or the capsules crate. Therefore, a -wrapper must be provided to interface Tock kernel code with the external -dependency. This prevents external APIs from leaking directly into the Tock -kernel. +Note, however, that _only_ the board crate itself may include such an external +dependency in its `Cargo.toml` file. + +A possible way to have other crates indirectly use such a dependency is through +a wrapper-trait. Such traits abstract the external dependency in a way that +allows other crates to still be built without the dependency included. + +### Capsule Crate-Specific External Dependencies + +Capsules are a mechanism to provide semi-trusted infrastructure to a Tock board, +for instance non chip-specific peripheral drivers (see [Design](./Design.md)). +As such, it can be desirable to utilize external dependencies to implement +complex subsystems. Examples for this are wireless- or networking-protocols such +as Bluetooth LE or TCP. + +To support such use-cases without forcing all boards to include external +dependencies, capsules are split into multiple crates: +- The `capsules/core` crate contains drivers and abstractions deemed essential + to most boards' operation, in addition to commonly used infrastructure and + _virtualizers_. It must not have any external dependencies. + +- The `capsules/extra` crate contains miscellaneous drivers and abstractions + which do not fit into other capsule crates. It must not have any external + dependencies. + +Capsule crates other than `core` and `extra` _may_ include external +dependencies. The granularity of such crates may range from implementing an +entire subsystem (e.g. a TCP/IP stack) to a single module providing some +isolated functionality. Whether an external dependency may be added to a given +crate, and the granularity of said crate, is evaluated on a case-by-case +basis. Concerns to take into account could be the utility, complexity and +quality of the external dependency, and whether the capsule would provide value +without this dependency. + +Newly contributed code or code from `capsules/extra` can be moved to a new +capsule crate when deemed necessary; this is evaluated on a case-by-case basis. ## Including the Dependency To help ensure maintainability and to promote transparency with including external dependencies, Tock follows a specific policy for their inclusion. -### Including Core External Dependencies - -The only crates that can contain external dependencies must be inside the -`capsules/` directory. For each new dependency a new crate should be added -within the `capsules/` directory. Only that crate can have the external -dependency. +### Including Capsule Crate-Specific External Dependencies -The only crates that can use the newly created external dependency crates are -board crates in the `boards/` directory. This ensures that boards which do not -want to include the external dependency can avoid including the crate with the -external dependency. +Capsules other than `capsules/core` and `capsules/extra` may include external +dependencies directly in their `Cargo.toml` file and use them directly. ### Including Board-Specific External Dependencies -Boards may include external dependencies directly in their board's `Cargo.toml` +Board crates may include external dependencies directly in their `Cargo.toml` file and use them directly. +### Including Core External Dependencies + +As of now, no process for including core external dependencies has been +established. + ### Documenting the Dependency and its Tree Each crate that includes an external dependency in its `Cargo.toml` file must From 8f19da6979b7eb28ecd3172583f65c06f9388d92 Mon Sep 17 00:00:00 2001 From: Leon Schuermann Date: Mon, 1 May 2023 12:08:17 -0400 Subject: [PATCH 7/9] doc/ExternalDependencies: update TOC --- doc/ExternalDependencies.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/doc/ExternalDependencies.md b/doc/ExternalDependencies.md index 747976bde1..88014b974f 100644 --- a/doc/ExternalDependencies.md +++ b/doc/ExternalDependencies.md @@ -8,15 +8,18 @@ External Dependencies - [External Dependency Design](#external-dependency-design) * [Rationale](#rationale) * [Specific Approach Motivation](#specific-approach-motivation) + * [Dependency Structure of Tock-Internal Crates](#dependency-structure-of-tock-internal-crates) - [External Dependency Selection](#external-dependency-selection) * [Core External Dependencies](#core-external-dependencies) + [Provide Important Functionality](#provide-important-functionality) + [Project Maturity](#project-maturity) + [Limited Sub-dependencies](#limited-sub-dependencies) * [Board-Specific External Dependencies](#board-specific-external-dependencies) + * [Capsule Crate-Specific External Dependencies](#capsule-crate-specific-external-dependencies) - [Including the Dependency](#including-the-dependency) - * [Including Core External Dependencies](#including-core-external-dependencies) + * [Including Capsule Crate-Specific External Dependencies](#including-capsule-crate-specific-external-dependencies) * [Including Board-Specific External Dependencies](#including-board-specific-external-dependencies) + * [Including Core External Dependencies](#including-core-external-dependencies) * [Documenting the Dependency and its Tree](#documenting-the-dependency-and-its-tree) From 7e197284066bd5efbacebd1d4c78aa4e1d1702e7 Mon Sep 17 00:00:00 2001 From: Leon Schuermann Date: Mon, 1 May 2023 21:57:27 -0400 Subject: [PATCH 8/9] doc/ExternalDependencies: remove "core" deps, change to guidelines In accordance with many discussions on the Tock core team calls, in addition to discussions on this document's PR (#3312, [1]), the notion of "core external dependencies" shall be removed for now. If required in the future, this original text can be restored. [1]: https://github.com/tock/tock/pull/3312/files/8f19da6979b7eb28ecd3172583f65c06f9388d92#r1181726351 --- doc/ExternalDependencies.md | 40 +++++++++---------------------------- 1 file changed, 9 insertions(+), 31 deletions(-) diff --git a/doc/ExternalDependencies.md b/doc/ExternalDependencies.md index 88014b974f..4857f7f728 100644 --- a/doc/ExternalDependencies.md +++ b/doc/ExternalDependencies.md @@ -10,7 +10,7 @@ External Dependencies * [Specific Approach Motivation](#specific-approach-motivation) * [Dependency Structure of Tock-Internal Crates](#dependency-structure-of-tock-internal-crates) - [External Dependency Selection](#external-dependency-selection) - * [Core External Dependencies](#core-external-dependencies) + * [General Guidelines for Dependency Selection](#general-guidelines-for-dependency-selection) + [Provide Important Functionality](#provide-important-functionality) + [Project Maturity](#project-maturity) + [Limited Sub-dependencies](#limited-sub-dependencies) @@ -19,7 +19,6 @@ External Dependencies - [Including the Dependency](#including-the-dependency) * [Including Capsule Crate-Specific External Dependencies](#including-capsule-crate-specific-external-dependencies) * [Including Board-Specific External Dependencies](#including-board-specific-external-dependencies) - * [Including Core External Dependencies](#including-core-external-dependencies) * [Documenting the Dependency and its Tree](#documenting-the-dependency-and-its-tree) @@ -128,30 +127,19 @@ dependency will be reviewed for inclusion, according to the criteria in this section. The requirements are intentionally strict. There are two general methods to for including an external dependency in the -Tock kernel: core external dependencies, and capsule- or board-specific -external dependencies. +Tock kernel: capsule-specific or board-specific external dependencies. -- Core external dependencies may be used in "core" Tock crates, such as the - _kernel_, _arch_, and _chip crates_. Notably, individual boards generally do - not have a reasonable choice whether to depend on those crates. +### General Guidelines for Dependency Selection -- Capsule-, or board-specific external dependencies may _only_ be in the - respective `boards/`-, or `chips/`-crates, or in a `capsules/`-crate that is - _not_ - - - `capsules/core`, or - - `capsules/extra`. - -### Core External Dependencies - -There are well-specified requirements for including a core external dependency. +In general, the following guidelines can provide an indication whether an +external dependency is suitable for inclusion in Tock. #### Provide Important Functionality -The external crate must provide important functionality that could not -easily or realistically be provided by the Tock developers. +The external crate provides important functionality that could not easily or +realistically be provided by the Tock developers. -The list of currently accepted important functionality: +Such functionality includes: * Cryptography libraries. Writing cryptographically secure code that is both correct and resistant to attacks is challenging. Leveraging validated, @@ -160,14 +148,9 @@ The list of currently accepted important functionality: #### Project Maturity -The external crate being added must be a mature project, with a high quality +The external crate being added should be a mature project, with a high quality of code. The project must be well regarded in the Rust community. -The top-level external crate must belong to one of the following set of -repository organizations: - -* [RustCrypto](https://github.com/RustCrypto) - #### Limited Sub-dependencies The external crate should have a limited sub-dependency tree. The fewer @@ -239,11 +222,6 @@ dependencies directly in their `Cargo.toml` file and use them directly. Board crates may include external dependencies directly in their `Cargo.toml` file and use them directly. -### Including Core External Dependencies - -As of now, no process for including core external dependencies has been -established. - ### Documenting the Dependency and its Tree Each crate that includes an external dependency in its `Cargo.toml` file must From 3e8bdfc808fb204b3567b16ac47819a7041a98a2 Mon Sep 17 00:00:00 2001 From: Brad Campbell Date: Wed, 10 May 2023 18:29:25 -0400 Subject: [PATCH 9/9] doc: external dep: update text after discussion --- doc/Design.md | 19 ++++--- doc/ExternalDependencies.md | 100 +++++++++++++++++++----------------- 2 files changed, 61 insertions(+), 58 deletions(-) diff --git a/doc/Design.md b/doc/Design.md index 483cccdad9..791bf3c515 100644 --- a/doc/Design.md +++ b/doc/Design.md @@ -291,16 +291,15 @@ delay. ### External Dependencies -Generally it's best to avoid adding any external crates to the Tock project. - -In certain situations Tock does allow external depenencies. This is decided on -a case by case basis. For more details on this see -[External Dependencies](ExternalDependencies.md). - -The other option to add an external library is to put specific portions of -libraries into the `libraries` folder. This puts the library's source in the -same repository, while keeping the library as a clearly separate crate. We do -try to limit how often this happens. +Tock generally prohibits any external crates within the Tock kernel to avoid +including external unsafe code. However, in certain situations Tock does allow +external dependencies. This is decided on a case by case basis. For more details +on this see [External Dependencies](ExternalDependencies.md). + +Tock uses some external libraries by vendoring them within the `libraries` +folder. This puts the library's source in the same repository, while keeping the +library as a clearly separate crate. This adds a maintenance requirement and +complicates updates, so this is also used on a limited basis. ### Using `unsafe` and Capabilities diff --git a/doc/ExternalDependencies.md b/doc/ExternalDependencies.md index 4857f7f728..7be0d6c387 100644 --- a/doc/ExternalDependencies.md +++ b/doc/ExternalDependencies.md @@ -7,7 +7,6 @@ External Dependencies - [External Dependency Design](#external-dependency-design) * [Rationale](#rationale) - * [Specific Approach Motivation](#specific-approach-motivation) * [Dependency Structure of Tock-Internal Crates](#dependency-structure-of-tock-internal-crates) - [External Dependency Selection](#external-dependency-selection) * [General Guidelines for Dependency Selection](#general-guidelines-for-dependency-selection) @@ -20,6 +19,7 @@ External Dependencies * [Including Capsule Crate-Specific External Dependencies](#including-capsule-crate-specific-external-dependencies) * [Including Board-Specific External Dependencies](#including-board-specific-external-dependencies) * [Documenting the Dependency and its Tree](#documenting-the-dependency-and-its-tree) +- [Design Goals and Alternative Approaches](#design-goals-and-alternative-approaches) @@ -27,8 +27,9 @@ Tock's general policy is the kernel does not include external dependencies (i.e. rust crates outside of the `tock/tock` repository) that are not part of the Rust standard library. However, on a limited, case-by-case basis with appropriate safeguards, external dependencies can be used in the Tock kernel. The rationale -and policy for this is described in this document. - +and policy for this is described in this document. This document only applies to +the Tock kernel binary itself, not userspace or other tools or binaries within +the Tock project. ## External Dependency Design @@ -47,48 +48,14 @@ external dependencies, verifying uses of `unsafe` are valid is more challenging to, particularly as external libraries evolve. External dependencies also typically themselves rely on dependencies, so -including one external crate likely pulls in several external crate. As of Nov -2022, cargo provides no mechanism for auditing and prohibiting `unsafe` in a -dependency hierarchy. Also, the dependency chain for an external crate is +including one external crate likely pulls in several external crates. As of May +2023, cargo does not provide a robust way to audit and prohibit `unsafe` within +a dependency hierarchy. Also, the dependency chain for an external crate is largely hidden from developers using the external crate. Lacking automated tools, managing dependencies is a manual process, and to limit overhead Tock generally avoids external dependencies. -### Specific Approach Motivation - -The mechanism for including external dependencies is designed to satisfy the -following goals. These goals were converged upon over multiple discussions of -the Tock developers. - -Goals: - -- Boards which do not need or want the functionality provided by the external - dependency can ensure the dependency is not included in the kernel build. -- Boards which do not use the dependency do not have to compile the dependency. -- Boards should have discretion on which code to include in their build. -- All uses of the external dependency in the Tock code base are explicit and - obvious. -- The location within the Tock code tree for external dependencies is clear and - consistent, and there is a consistent format to document the dependency. -- There is not undue overhead or boilerplate required to add an external - dependency. - -These goals necessitate a few design decisions. For example, as crates are the -smallest unit of compilation in Rust, external dependencies must be included -through new crates added to the Tock source tree so they can be individually -included or excluded in specific builds. Also, crates provide a namespace to use -to identify when external dependencies are being incorporated. - -Additionally, we avoid using traits or HIL-like interfaces for dependencies -(i.e. core Tock capsules/modules would use a Tock-defined trait much like -capsules use HILs, and a wrapper would use the external dependency to implement -the trait) to avoid the overhead of implementing and maintaining a wrapper to -implement the trait. While architecturally this has advantages, the overhead was -deemed too burdensome for the expected benefit. -We explicitly document the goals to help motivate the specific design in the -remainder of this document. Also, this policy may change in the future, but -these goals should be considered in any future updates. ### Dependency Structure of Tock-Internal Crates @@ -111,12 +78,10 @@ Furthermore, this policy assumes the following rules regarding crate dependencies internal to Tock: - a _board crate_ is not a dependency of any other Tock-internal crate -- a _chip crate_ is only a dependency of _board crates_ or other - _chip crates_ +- a _chip crate_ is only a dependency of _board crates_ or other _chip crates_ - a _capsule crate_ is only a dependency of other _capsule crates_ or _board crates_ -- an _arch crate_ may only depend on the _kernel crate_ and other - _arch crates_ +- an _arch crate_ may only depend on the _kernel crate_ and other _arch crates_ - the _kernel crate_ does not depend on _arch_, _chip_, _board_, or _capsule crates_ @@ -174,15 +139,17 @@ dependency in its `Cargo.toml` file. A possible way to have other crates indirectly use such a dependency is through a wrapper-trait. Such traits abstract the external dependency in a way that -allows other crates to still be built without the dependency included. +allows other crates to still be built without the dependency included. While +using a wrapper-trait is not required, in certain scenarios wrapper-traits may +be useful or desirable. ### Capsule Crate-Specific External Dependencies Capsules are a mechanism to provide semi-trusted infrastructure to a Tock board, for instance non chip-specific peripheral drivers (see [Design](./Design.md)). -As such, it can be desirable to utilize external dependencies to implement -complex subsystems. Examples for this are wireless- or networking-protocols such -as Bluetooth LE or TCP. +As such, external dependencies may be useful to implement complex subsystems. +Examples for this are wireless or networking protocols such as Bluetooth Low +Energy or TCP. To support such use-cases without forcing all boards to include external dependencies, capsules are split into multiple crates: @@ -231,3 +198,40 @@ must be included in the PR that adds the external dependency. The Tock dependency tree can be generated by running `cargo tree`. The tree should be updated whenever a dependency change is made. + +## Design Goals and Alternative Approaches + +While exploring a policy for including external dependencies, the Tock project +considered many options. This resulted in establishing a list of goals for an +external dependency approach. These goals were converged upon over multiple +discussions of the Tock developers. + +Goals: + +- Boards which do not need or want the functionality provided by the external + dependency can ensure the dependency is not included in the kernel build. +- Boards which do not use the dependency do not have to compile the dependency. +- Boards should have discretion on which code to include in their build. +- All uses of the external dependency in the Tock code base are explicit and + obvious. +- The location within the Tock code tree for external dependencies is clear and + consistent, and there is a consistent format to document the dependency. +- There is not undue overhead or boilerplate required to add an external + dependency. + +These goals necessitate a few design decisions. For example, as crates are the +smallest unit of compilation in Rust, external dependencies must be included +through new crates added to the Tock source tree so they can be individually +included or excluded in specific builds. Also, crates provide a namespace to use +to identify when external dependencies are being incorporated. + +Additionally, we avoid using traits or HIL-like interfaces for dependencies +(i.e. core Tock capsules/modules would use a Tock-defined trait much like +capsules use HILs, and a wrapper would use the external dependency to implement +the trait) to avoid the overhead of implementing and maintaining a wrapper to +implement the trait. While architecturally this has advantages, the overhead was +deemed too burdensome for the expected benefit. + +We explicitly document the goals to help motivate the specific design in the +remainder of this document. Also, this policy may change in the future, but +these goals should be considered in any future updates.