Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixup or re-enable tests with Use PSA #5742

Merged

Conversation

superna9999
Copy link
Contributor

@superna9999 superna9999 commented Apr 14, 2022

Description

There are a number of tests that currently depend on !MBEDTLS_USE_PSA_CRYPTO. Since this USE_PSA is meant to become the default, and even the only option at some point, it is should enjoy at least as much test coverage as the non-PSA variant.

Resolves #5669

Status

READY

Requires Backporting

NO

Migrations

NO

Additional comments

N/A

Todos

  • Tests

Steps to test or reproduce

test_suite_ssl must run clean

Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
…te_mfl tests with PSA

Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
@superna9999 superna9999 added needs-review Every commit must be reviewed by at least two team members, needs-ci Needs to pass CI tests Community needs-reviewer This PR needs someone to pick it up for review component-test Test framework and CI scripts labels Apr 15, 2022
@superna9999
Copy link
Contributor Author

@mpg the only remaining test with !MBEDTLS_USE_PSA_CRYPTO is :

 Verify ext RSA #5 (PKCS1 v2.1, wrong salt_len)

in tests/suites/test_suite_pk.data

But there is an alternate one for PSA, because the wrong salt len doesn't return PSA_ERROR_INVALID_PADDING but PSA_ERROR_INVALID_SIGNATURE so pk return can't be MBEDTLS_ERR_RSA_INVALID_PADDING

Where this should be documented, can we add comments into tests/suites/test_suite_pk.data ?

@superna9999 superna9999 marked this pull request as ready for review April 15, 2022 07:17
@superna9999 superna9999 added needs-work and removed needs-review Every commit must be reviewed by at least two team members, needs-reviewer This PR needs someone to pick it up for review labels Apr 15, 2022
@mpg mpg self-requested a review April 15, 2022 07:20
Those were disabled in original submission, but it works fine
with PSA crypto enabled.

Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
@superna9999 superna9999 added needs-review Every commit must be reviewed by at least two team members, needs-reviewer This PR needs someone to pick it up for review and removed needs-work labels Apr 15, 2022
@superna9999
Copy link
Contributor Author

Found a very explicit comment in

/* Mbed TLS distinguishes "invalid padding" from "valid padding but
* the rest of the signature is invalid". This has little use in
* practice and PSA doesn't report this distinction. */
about the problem, so I copied the comment in tests/suites/test_suite_pk.data to document the PSA specific test need

@superna9999 superna9999 force-pushed the 5669-review-test-incompatible-psa branch from 62a19c7 to 9b3899e Compare April 15, 2022 07:47
@mpg
Copy link
Contributor

mpg commented Apr 15, 2022

I think a comment in the data file explaining why there are two versions of the test's data is good, but actually I'd prefer to handle that in the test function, for example:

#if defined(MBEDTLS_USE_PSA_CRYTPO)
if( result == MBEDTLS_ERR_RSA_INVALID_PADDING )
    result = MBEDTLS_ERR_RSA_VERIFY_FAILED;
#endif

(with a comment explaining why). Do you think that would work as well?

The reason I think this is preferable is that having no test case that depends on !MBEDTLS_USE_PSA_CRYPTO makes things easier to audit: we can just grep for that see that there are no occurrences, instead of finding a small number of occurrences and then checking each of them to see if there's a corresponding PSA test.

Actually, this makes me think: perhaps we should add a component in all.sh that checks that there is no occurrence of 'depends_on.*!MBEDTLS_USE_PSA_CRYPTO' in the code base. That way, we don't risk having to do that task again in a couple of months...

@superna9999
Copy link
Contributor Author

@mpg Ok ! I'll move it to the function instead with the same comment.

I'll also try to detect the non-PSA test cases in all.sh

@mpg
Copy link
Contributor

mpg commented Apr 15, 2022

Cool, thanks!

Also, if you don't mind me expanding the scope of the task a bit: we should check in ssl-opt.sh too.

I just did, and found two tests that have requires_config_disabled MBEDTLS_USE_PSA_CRYPTO. As far as I can see, that's just a remnant from the time where TLS 1.3 wasn't compatible with that. I tried just removing that line, and the tests passed in a build with USE_PSA enabled, so I think those negative depends are no longer useful and we just forgot to remove them.

Can you remove those and also add that to the all.sh check?

@superna9999
Copy link
Contributor Author

@mpg sure will do

@superna9999 superna9999 added needs-work and removed needs-review Every commit must be reviewed by at least two team members, needs-reviewer This PR needs someone to pick it up for review labels Apr 15, 2022
@mpg mpg added the needs-review Every commit must be reviewed by at least two team members, label Apr 25, 2022
Copy link
Contributor

@mpg mpg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with the code, but I'm requesting further tuning of the comment.

* error depending on which path was taken.
* If the PSA path is used, it won't because Mbed TLS
* distinguishes "invalid padding" from "valid padding but
* the rest of the signature is invalid". This has little use in
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for picking on grammar, but I don't think we should have a full stop here, the explanation introduced by the "because" is not complete yet - the reason is "and PSA doesn't report this distinction". While editing this comment, I think we might want to expand on why a non-PSA path might be taken while USE_PSA is active.

Here's a suggestion:

    /* Mbed TLS distinguishes "invalid padding" from "valid padding but
     * the rest of the signature is invalid". This has little use in
     * practice and PSA doesn't report this distinction.
     * In this case, PSA returns PSA_ERROR_INVALID_SIGNATURE translated
     * to MBEDTLS_ERR_RSA_VERIFY_FAILED.
     * However, currently `mbedtls_pk_verify_ext()` may use either the PSA or the
     * Mbed TLS API, depending on the PSS options used. So, it may return either
     * INVALID_PADDING or INVALID_SIGNATURE.
     */

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with that, I'll update the comment

Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
mpg
mpg previously approved these changes Apr 26, 2022
tests/scripts/all.sh Outdated Show resolved Hide resolved
tests/scripts/all.sh Outdated Show resolved Hide resolved
@gilles-peskine-arm gilles-peskine-arm added needs-work and removed needs-review Every commit must be reviewed by at least two team members, labels Apr 26, 2022
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
@superna9999 superna9999 added needs-review Every commit must be reviewed by at least two team members, and removed needs-work labels Apr 27, 2022
@mpg mpg added approved Design and code approved - may be waiting for CI or backports and removed needs-review Every commit must be reviewed by at least two team members, labels Apr 28, 2022
@mpg mpg merged commit ad47487 into Mbed-TLS:development Apr 28, 2022
mpg added a commit to mpg/mbedtls that referenced this pull request Jun 28, 2022
We've decided not to check it, see
Mbed-TLS#5277

Also, remove the check about no test depending on !USE_PSA - here it's
clearly the right thing to do, and adding this check to all.sh was
already controversial in the first place [1] so let's not keep it if it
becomes an annoyance.

[1]: Mbed-TLS#5742 (comment)

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
mpg added a commit to mpg/mbedtls that referenced this pull request Sep 13, 2022
We've decided not to check it, see
Mbed-TLS#5277

Also, remove the check about no test depending on !USE_PSA - here it's
clearly the right thing to do, and adding this check to all.sh was
already controversial in the first place [1] so let's not keep it if it
becomes an annoyance.

[1]: Mbed-TLS#5742 (comment)

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
mpg added a commit to mpg/mbedtls that referenced this pull request Dec 6, 2022
The previous commit declared that some tests cases in ssl-opt.sh depend
on USE_PSA being disabled, which is the right thing to do.

We had a check that forbade that - it was mainly meant to prevent
accidental re-introduction of such dependencies after we cleaned up a
number of cases where it was not warranted, but already at the time that
was controversial [1]. Now it's preventing us from doing the right
thing, so let's just remove it.

[1]: Mbed-TLS#5742 (comment)

See also Mbed-TLS#5907 which also
removes this for a similar reason.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
mpg added a commit to mpg/mbedtls that referenced this pull request Dec 6, 2022
The previous commit declared that some tests cases in ssl-opt.sh depend
on USE_PSA being disabled, which is the right thing to do.

We had a check that forbade that - it was mainly meant to prevent
accidental re-introduction of such dependencies after we cleaned up a
number of cases where it was not warranted, but already at the time that
was controversial [1]. Now it's preventing us from doing the right
thing, so let's just remove it.

[1]: Mbed-TLS#5742 (comment)

See also Mbed-TLS#5907 which also
removes this for a similar reason.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
mpg added a commit to mpg/mbedtls that referenced this pull request Dec 9, 2022
The previous commit declared that some tests cases in ssl-opt.sh depend
on USE_PSA being disabled, which is the right thing to do.

We had a check that forbade that - it was mainly meant to prevent
accidental re-introduction of such dependencies after we cleaned up a
number of cases where it was not warranted, but already at the time that
was controversial [1]. Now it's preventing us from doing the right
thing, so let's just remove it.

[1]: Mbed-TLS#5742 (comment)

See also Mbed-TLS#5907 which also
removes this for a similar reason.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Design and code approved - may be waiting for CI or backports component-test Test framework and CI scripts
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Review tests incompatible with Use PSA
3 participants