Support volatile keys in external SE

[stevew817]

Description

Implemented changes requested in #3288 to support volatile keys located in Secure Element drivers, and to not have the PSA core store information about those volatile keys persistently.

Status

READY

Requires Backporting

NO

Migrations

NO

Additional comments

Todos

• Tests
• Documentation

Steps to test or reproduce

Tested through the updated SE HAL test suite.

[danh-arm]

Thanks @stevew817 for the PR. This doesn't build in some configs, e.g.

******************************************************************

* build_crypto_full: build: make, crypto only, full config 

* Mon Jun  1 16:51:26 UTC 2020

******************************************************************

  CC    test_suite_gcm.aes128_en.c

suites/test_suite_psa_crypto_se_driver_hal.function: In function 'test_key_creation_import_export':

suites/test_suite_psa_crypto_se_driver_hal.function:898:54: error: expected statement before ')' token

         if( ! check_no_persistent_data( location ) ) )

                                                      ^

suites/main_test.function: In function 'get_expression':

suites/main_test.function:136:30: error: 'TEST_SE_VOLATILE_LIFETIME' undeclared (first use in this function)

  *                  function parameters.

                              ^

suites/main_test.function:136:30: note: each undeclared identifier is reported only once for each function it appears in

[stevew817]

Interesting. I'm running with tests/scripts/basic-in-docker.sh, which does not throw this error.

Is this script somehow only testing a previous version, and not updating with the latest in the working directory? Is there some script that needs to be run before it to get it to regenerate the output of the .function and .data files?

[gilles-peskine-arm]

basic-in-docker.sh only runs basic-build-test.sh which builds the library in one configuration and runs tests. You need all-in-docker.sh which runs tests/scripts/all.sh. Compared with all-in-docker.sh, the CI additionally runs some Windows and Arm Compiler jobs.

You can pass component names (tests/scripts/all.sh --list-components) to all.sh or all-in-docker.sh to run one or more component without spending the hours of the whole script.

[gilles-peskine-arm]

Overall this mostly looks good, but a few areas need work.

The CI results aren't very useful right now because builds without SE support break on p_drv being unused in psa_start_key_creation.

[gilles-peskine-arm]

The code for persistent/volatile and local/external should be mostly orthogonal. As it stands, some validation is missing on persistent SE keys.

If MBEDTLS_PSA_CRYPTO_SE_C is enabled and MBEDTLS_PSA_CRYPTO_STORAGE_C is disabled, a persistent external key should be rejected. This configuration is currently rejected in check_config.h, but only because there is currently no support for volatile external keys, which this PR changes.

To keep this PR small, it's ok to keep the restriction in check_config.h and do the following later:

• Remove the restriction in check_config.h.
• In all.sh, no longer systematically disable MBEDTLS_PSA_CRYPTO_SE_C when MBEDTLS_PSA_CRYPTO_STORAGE_C is disabled.
• Fix any failing test.

But please fix the existing bug that the key id isn't validated for keys in a secure element and add a non-regression test.

[stevew817]

Is there any intent to support persistent SE-stored keys without having to require PSA Crypto Storage?

[gilles-peskine-arm]

No, because the metadata is always in internal storage. Due to the interface design (the application only remembers the key id, not its lifetime), it's difficult to have metadata spread over multiple storage areas.

[gilles-peskine-arm]

Sorry I forgot to mention this earlier, but for future reference, please don't mark GitHub conversations as resolved when you're the author of the code: that's for the commenter to do when they've decided that the comment was resolved satisfactorily.

In Mbed TLS, we have a convention for the coder use the 🚀 emoji to indicate “I've handled this comment in my local copy”, if you want to keep track of which comments you've handled.

[stevew817]

@gilles-peskine-arm Do you have insight into why those two CI jobs are still failing?

[gilles-peskine-arm]

The remaining CI failure is from the ABI compatibility checker. It says that the symbol psa_validate_persistent_key_parameters has disappeared. Since this is an internal library function, this is not a change of the ABI and no update to the soversion is needed.

So that's a pass from CI. I'll review shortly.

[gilles-peskine-arm]

I'm happy with the library code and with the tests. The documentation needs a little improvement.

The commit history is hard to follow, so I'm requesting a rewrite of the history. The commits “Update PSA core to not persist SE keys declared volatile” and “Fixups after review” do too many things. Please break them up. At this point, you might as well remove the back-and-forth between these two commits. In general, please don't do mix refactoring and adding new things in the same commit.

And please use more descriptive commit messages. For example, “Enable figuring out number of cores when running on OS X” is a good commit message: it explains exactly what's going on. “Fix error from #3302” is not so good; it's acceptable because the diff is very simple, but “Fix typo in a macro that isn't used yet” would be more informative. “Update PSA core to not persist SE keys declared volatile” is not bad in itself but the commit does too much which isn't described in the commit message. “Simplify key checking” lacks explanations and “Fixups after review” is not informative at all.

Please add a changelog entry in ChangeLog.d.

To facilitate review, please save a copy of the current state in your fork (I use the naming convention original-branch-name-1, original-branch-name-2, etc. for successive versions) before force-pushing. Please avoid combining new content with a force-push that rewrites the history. Also, please don't rebase on top of the current target branch if there's no strong reason for it (like a force push). The reason for these requests is to make the force-push diff easy to review (in particular, if you're only restructuring commits or rewriting commit messages, the force-push diff should be empty).

[stevew817]

Force-push updated commit message length as requested by @ronald-cron-arm

[stevew817]

@gilles-peskine-arm again, all failing CI is inside the ARM network and I cannot see what would've cause failure.

[danh-arm]

The CI failures seem mainly to be in our Mbed OS testing:

Compile [  0.1%]: mbed_tz_context.c

Compile [  0.2%]: rf_configuration.c

Compile [  0.3%]: fslittle_test.c

Compile [  0.4%]: MCR20Drv.c

Compile [  0.4%]: main.cpp

[Fatal Error] pk.h@49,10: psa/crypto.h: No such file or directory

[ERROR] In file included from ./mbed-os/features/mbedtls/inc/mbedtls/ssl_ciphersuites.h:33,

                 from ./mbed-os/features/mbedtls/inc/mbedtls/ssl.h:36,

                 from ./mbed-os/features/netsocket/TLSSocketWrapper.h:29,

                 from ./mbed-os/features/netsocket/nsapi.h:42,

                 from ./mbed-os/mbed.h:26,

                 from ./main.cpp:20:

./mbed-os/features/mbedtls/inc/mbedtls/pk.h:49:10: fatal error: psa/crypto.h: No such file or directory

   49 | #include "psa/crypto.h"

      |          ^~~~~~~~~~~~~~

compilation terminated.

However, I don't know enough about this change or that testing to suggest what the problem is.

There's also the ABI/API compatibility check failure, which Gilles said earlier can be ignored.

[ronald-cron-arm]

Mbed OS Testing is currently broken thus just ignore it. Sorry for the inconvenience. Thus this PR does not have any CI issues currently.

[ronald-cron-arm]

Thank you for addressing my comments. LGTM.

[gilles-peskine-arm]

CI passed except for:

• Mbed OS which is currently failing for unrelated reasons.
• ABI changes only in internal functions (false positives).

So this is as good as a pass.

[gilles-peskine-arm]

Sorry for missing this at the time, but psa_key_lifetime_is_external is wrong here. This allows any application to create a key with a key id that's reserved for the implementation, as long as the key is in a secure element. This wasn't a big issue at the time of this pull request because Mbed TLS didn't use the implementation key id range. But with #3547 (comment) it's beginning to use the range and we're going to have to backtrack this change.