Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Mbed OS for PSA Crypto API 1.0b3 #11315

Merged
merged 17 commits into from Sep 3, 2019

Conversation

@Patater
Copy link
Contributor

commented Aug 23, 2019

Description

Update Mbed OS for the PSA Crypto API 1.0 beta 3. This involves refactoring code that uses the PSA Crypto API, updating the PSA Crypto client/server implementation, and updating the versions of Mbed TLS and Mbed Crypto that come pre-integrated with Mbed OS.

Pull request type

[ ] Fix
[X] Refactor
[ ] Target update
[ ] Functionality change
[ ] Docs update
[ ] Test update
[ ] Breaking change

Release Notes

Background

Mbed Crypto is our implementation of the PSA Crypto APIs. We shipped an implementations of prerelease versions of the PSA Crypto API in Mbed OS 5.11, 5.12, and 5.13, with a warning in the documentation that these APIs were subject to change and that we did not intend to maintain backwards compatibility with them. The PSA Crypto API has continued to develop and change over the past few months and a number of breaking changes have been made since the pre-release version we based our Mbed Crypto implementation on.

What is being broken?

The PSA Crypto APIs in Mbed OS as shipped in Mbed OS 5.13 are breaking.

PSA Crypto API 1.0b3 API breaking changes, addressed in Mbed OS 5.14 (see #11315)

  • Use key attributes structures for key creation
  • Make generating or importing a key also allocate the key
  • Update key derivation functions to accept chunked inputs
  • Update key agreement API
  • Align PSA Crypto error codes with other PSA error codes
  • Rename functions for consistency with each other and the rest of PSA
  • Be consistent in use of stdint types

Expected PSA Crypto API 1.0 API breaking changes, to be addressed in Mbed OS 5.15

  • None

Why is it being broken?

Mbed Crypto needs to track the upstream PSA Crypto API as maintained by ATG in order to pass ATG's PSA Compliance Kit tests. The PSA Crypto API was not yet finalized in Mbed OS 5.13 and continues to evolve. We shipped an implementation in Mbed OS 5.13 to enable other teams working on Mbed OS to develop their services a top APIs that should be mostly similar to the final APIs, rather than save up all the integration pain for a later Mbed OS release when the APIs are finalized.

Analysis of impact on users

There should be no surprises to users based on our statements of PSA API instability. We've worked with Mbed TLS, Pelion Client, Storage (ITS), SPM, and Attestation teams to ensure the message of API instability was understood and to coordinate our changes to the API. There are potentially other users of the PSA Crypto API, and our documentation states the stability level of the API for these users.

Alternatives

We could continue to provide the version of the PSA Crypto API shipped with Mbed OS 5.13, but to save flash size and reduce the maintenance burden of maintaining an API we clearly communicated we'd be breaking in the next release, this was deemed not worth the cost.

Mitigation and migration path for users

Users must update to use the new version of the API.

Renaming of key pair names

Replace KEY_PAIR in names that would have used KEYPAIR in the previous API version. For example, PSA_KEY_TYPE_ECC_KEY_PAIR replaces PSA_KEY_TYPE_ECC_KEYPAIR.

Using persistent keys

Use psa_open_key() to open a persistent key. Previously, volatile keys could also be opened. With PSA Crypto API 1.0b3, keys are implicitly opened for you upon import, generation, or derivation.

psa_status_t psa_open_key(psa_key_id_t id,
                          psa_key_handle_t *handle);

Only persistent keys can be opened, so there is no need to pass the lifetime anymore.

It is no longer necessary to call psa_create_key() to make a key persistent. A key is persistent if it is created with a lifetime other than PSA_KEY_LIFETIME_VOLATILE. As part of key creation, use psa_set_key_id() to set both the key's persistent identifier and to set the lifetime to persistent and then call the key creation routine: like psa_generate_key() or psa_import_key()

Old New
psa_open_key() Only use for opening previously created persistent keys
psa_create_key() psa_set_key_id() Keys with IDs are made persistent implicitly upon creation

Allocating keys

Key creation will implicitly allocate resources as necessary, so psa_allocate_key() has been removed from the API and is no longer needed.

Old New
psa_allocate_key() Not necessary. Delete calls to psa_allocate_key().

Importing keys

Previously, you had create a policy structure and pass many function arguments to communicate the properties you wanted the imported key to have. Now, you describe them entirely within the attributes structure, passing only the attributes and data to psa_import_key().

Old New
psa_key_policy_init() psa_key_attributes_init()
psa_key_policy_set_usage() psa_set_key_usage_flags(), psa_set_key_algorithm()
Pass key type to psa_import_key() psa_set_key_type()
psa_set_key_policy() Pass the attributes to psa_import_key()
psa_import_key() psa_import_key()

Generating keys

Previously, you had create a policy structure and pass many function arguments to communicate the properties you wanted the imported key to have. Now, you describe them entirely within the attributes structure, passing only the attributes and data to psa_generate_key().

Old New
psa_key_policy_init() psa_key_attributes_init()
psa_key_policy_set_usage() psa_set_key_usage_flags(), psa_set_key_algorithm()
Pass key type to psa_import_key() psa_set_key_type()
psa_set_key_policy() Pass the attributes to psa_import_key()
psa_generate_key() psa_generate_key()

Reading key policy or information

What used to be two functions with many parameters each is now one function that returns the attributes in one structure, in the same format you'd use to create new keys.

Old New
psa_get_key_policy(), psa_get_key_information() psa_get_key_attributes()

Deriving keys

The previous "generator" class of functions has been renamed to "key_derivation". The psa_crypto_generator_t structure was previously used to derive keys. Use of the psa_key_derivation_operation_t structure replaces psa_crypto_generator_t for deriving keys.

Old New
psa_crypto_generator_t psa_key_derivation_operation_t
psa_generator_abort() psa_key_derivation_abort()
psa_get_generator_capacity() psa_key_derivation_get_capacity()
Function parameter psa_key_derivation_set_capacity()
psa_generator_read() psa_key_derivation_output_bytes()
Use of generator with PSA_ALG_SELECT_RAW psa_raw_key_agreement()
psa_key_derivation() Deriving keys now uses key derivation objects and consists of multiple parts. See the getting started guide for details.

Key agreement

Old New
psa_key_agreement() psa_key_derivation_setup(), psa_key_derivation_key_agreement(), psa_key_derivation_output_key()

Hashing

A few new functions have been added to help with hashing. Specifically, functions to perform one-shot computation or comparision of hashes.

Old New
Many hash function calls psa_hash_compute()
Many hash function calls psa_hash_compare()

Computing or verifying a MAC

A few new functions have been added to help with working with MACs. Specifically, functions to perform one-shot computation or comparision of MACs.

Old New
Many hash function calls psa_mac_compute()
Many hash function calls psa_mac_verify()

Symmetric cryptography

A few new functions have been added to help with working with symmetric ciphers. Specifically, functions to perform one-shot encryption or decryption. The types used by psa_cipher_generate_iv(), psa_cipher_set_iv(), and psa_cipher_update() have changed from unsigned char to uint8_t.

Old New
Many hash function calls psa_cipher_encrypt()
Many hash function calls psa_cipher_decrypt()

Authenticated encryption

The PSA Crypto API 1.0b3 introduces multi-part authenticated encryption functions. The original one-shot AEAD functions still remain and aren't being replaced.

New functions for multipart AEAD
  • psa_aead_operation_init()
  • psa_aead_encrypt_setup()
  • psa_aead_decrypt_setup()
  • psa_aead_generate_nonce()
  • psa_aead_set_nonce()
  • psa_aead_set_lengths()
  • psa_aead_update_ad()
  • psa_aead_update()
  • psa_aead_finish()
  • psa_aead_verify()
  • psa_aead_abort()

Mbed Crypto entropy injection

Use of uint8_t replaces unsigned char in mbedtls_psa_inject_entropy(). The macro MBEDTLS_PSA_INJECT_ENTROPY replaces MBEDTLS_PSA_ENTROPY_INJECTION.

Patater added 8 commits Aug 20, 2019
We'll use the new ARM-software/psa-arch-tests directly instead for PSA
Crypto API 1.0b3. This commit removes the crypto compliance tests only
for now, leaving attestation and storage tests.
Remove PSA_SUCCESS redefinitions in the PSA compliance test PAL.
Make crypto_struct_ipc.h (for use with PSA Crypto clients) match style
with the file it is based on, crypto_struct.h (from Mbed Crypto). This
helps to keep the file diff minimal so it's easy to see the meaningful
(non-style) changes.
PSA_CRYPTO_INIT is defined as a service identifier. We don't need to
redefine it as a secure function. There is only one function under this
service identifier, so no secure function definitions are necessary to
differentiate between.
Rename the reference count variable to something shorter and avoid using
the mispelling "refence".
Order the IPC functions in the same order as the functions are listed in
crypto.h for improved readability.
Make the service implementation use the same names as the API as
declared in the crypto.h header. This improves readability and
maintainability of the service.
@Patater Patater force-pushed the Patater:psa-crypto-api-1.0b3 branch from 0194997 to c9569c6 Aug 23, 2019
@Patater

This comment has been minimized.

Copy link
Contributor Author

commented Aug 23, 2019

Rebased to fix astyle issues.

@Patater Patater force-pushed the Patater:psa-crypto-api-1.0b3 branch from c9569c6 to a116293 Aug 23, 2019
@Patater

This comment has been minimized.

Copy link
Contributor Author

commented Aug 23, 2019

Rebased to actually fix astyle issues.

@ciarmcom ciarmcom requested review from ashok-rao, maclobdell and ARMmbed/mbed-os-maintainers Aug 23, 2019
@ciarmcom

This comment has been minimized.

@0xc0170

This comment has been minimized.

Copy link
Member

commented Aug 26, 2019

First CI run started

@mbed-ci

This comment has been minimized.

Copy link

commented Aug 26, 2019

Test run: FAILED

Summary: 4 of 4 test jobs failed
Build number : 1
Build artifacts

Failed test jobs:

  • jenkins-ci/mbed-os-ci_unittests
  • jenkins-ci/mbed-os-ci_build-ARM
  • jenkins-ci/mbed-os-ci_build-GCC_ARM
  • jenkins-ci/mbed-os-ci_build-IAR
@0xc0170

This comment has been minimized.

Copy link
Member

commented Aug 26, 2019

Build failures look related

@0xc0170 0xc0170 added needs: work and removed needs: review labels Aug 26, 2019
@Patater

This comment has been minimized.

Copy link
Contributor Author

commented Aug 27, 2019

Thanks for testing. I'll work on fixing.

Mbed Crypto also supplies include files. Ensure that our unit tests can
find those headers.
@Patater Patater force-pushed the Patater:psa-crypto-api-1.0b3 branch from a116293 to 69d7178 Aug 28, 2019
@0xc0170 0xc0170 requested a review from bulislaw Aug 30, 2019
@mbed-ci

This comment has been minimized.

Copy link

commented Aug 30, 2019

Test run: FAILED

Summary: 2 of 4 test jobs failed
Build number : 5
Build artifacts

Failed test jobs:

  • jenkins-ci/mbed-os-ci_build-GCC_ARM
  • jenkins-ci/mbed-os-ci_build-IAR
@0xc0170

This comment has been minimized.

Copy link
Member

commented Aug 30, 2019

CI aborted and restarted.

@0xc0170

This comment has been minimized.

Copy link
Member

commented Aug 30, 2019

Last test and we have the very last PR for 5.14.0-rc1 in 🎉

@0xc0170

This comment has been minimized.

Copy link
Member

commented Aug 30, 2019

Looks like tests are failing, related to this changeset. @ARMmbed/mbed-os-crypto Please review

@mbed-ci

This comment has been minimized.

Copy link

commented Aug 30, 2019

Test run: FAILED

Summary: 1 of 11 test jobs failed
Build number : 6
Build artifacts

Failed test jobs:

  • jenkins-ci/mbed-os-ci_greentea-test
@Patater

This comment has been minimized.

Copy link
Contributor Author

commented Sep 2, 2019

@adbridge

This comment has been minimized.

Copy link
Contributor

commented Sep 2, 2019

@Patater any status update on this ?

@Patater

This comment has been minimized.

Copy link
Contributor Author

commented Sep 2, 2019

@Patater any status update on this ?

We've discovered that master doesn't build. ;)

@Patater

This comment has been minimized.

Copy link
Contributor Author

commented Sep 2, 2019

@Patater any status update on this ?

We've discovered that master doesn't build. ;)

#11389

Patater added 5 commits Aug 16, 2019
The PSA Crypto API has moved on from 1.0b2 to 1.0b3, bringing along with
it some breaking changes. Update Mbed OS to use the 1.0b3 API.
The release script must be run from mbed-os root, otherwise the test
partition binaries will not be created. Add a note in the tool's README
to make this clear and hopefully help save someone some debugging time
in the future.
If psa_attestation_inject_key() is called twice, exit with a non-fatal
error status without attempting to create another attestation key. The
key already exists and doesn't need to be added again.
Be robust when keys can't be opened for deletion by erasing storage and
thereby all keys.
Rebuild the TF-M binaries for the LPC55S69 and MUSCA_A1 using the latest
service updates. This allows the boards to use the PSA Crypto API 1.0b3.
@Patater Patater force-pushed the Patater:psa-crypto-api-1.0b3 branch from c80c32d to a848cd6 Sep 2, 2019
@Patater

This comment has been minimized.

Copy link
Contributor Author

commented Sep 2, 2019

Rebased to initialize attributes variable, which contains a pointer, domain_parameters. When psa_reset_key_attributes() calls free(attributes->domain_parameters) is called on a garbage initialized attributes, bad things like hard faults or worse can happen.

@adbridge

This comment has been minimized.

Copy link
Contributor

commented Sep 2, 2019

CI started

@mbed-ci

This comment has been minimized.

Copy link

commented Sep 2, 2019

Test run: SUCCESS

Summary: 11 of 11 test jobs passed
Build number : 7
Build artifacts

@0xc0170

This comment has been minimized.

Copy link
Member

commented Sep 3, 2019

All good, merging now 👍

@0xc0170

This comment has been minimized.

Copy link
Member

commented Sep 3, 2019

The PSA Crypto APIs in Mbed OS as shipped in Mbed OS 5.13 are breaking.

Is this still true in the release notes after changes done in this PR?

@0xc0170 0xc0170 merged commit e001216 into ARMmbed:master Sep 3, 2019
25 checks passed
25 checks passed
continuous-integration/jenkins/pr-head This commit looks good
Details
jenkins-ci/build-ARM Success
Details
jenkins-ci/build-GCC_ARM Success
Details
jenkins-ci/build-IAR Success
Details
jenkins-ci/cloud-client-test Success
Details
jenkins-ci/dynamic-memory-usage RTOS ROM(+0 bytes) RAM(+72 bytes)
Details
jenkins-ci/exporter Success
Details
jenkins-ci/greentea-test Success
Details
jenkins-ci/mbed2-build-ARM Success
Details
jenkins-ci/mbed2-build-GCC_ARM Success
Details
jenkins-ci/mbed2-build-IAR Success
Details
jenkins-ci/unittests Success
Details
travis-ci/astyle Success!
Details
travis-ci/docs Success!
Details
travis-ci/doxy-spellcheck Success!
Details
travis-ci/events Success! Runtime is 8643 cycles.
Details
travis-ci/gitattributestest Success!
Details
travis-ci/include_check Success!
Details
travis-ci/licence_check Success!
Details
travis-ci/littlefs Success! Code size is 8464B.
Details
travis-ci/psa-autogen Success!
Details
travis-ci/tools-py2.7 Success!
Details
travis-ci/tools-py3.5 Success!
Details
travis-ci/tools-py3.6 Success!
Details
travis-ci/tools-py3.7 Success!
Details
@Patater

This comment has been minimized.

Copy link
Contributor Author

commented Sep 3, 2019

The PSA Crypto APIs in Mbed OS as shipped in Mbed OS 5.13 are breaking.

Is this still true in the release notes after changes done in this PR?

Yes, the PSA Crypto APIs are different from what we shipped in Mbed OS 5.13.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.