Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests fail after 2024-01-19 #1961

Closed
bmwiedemann opened this issue Jan 6, 2023 · 10 comments · Fixed by #2180
Closed

Tests fail after 2024-01-19 #1961

bmwiedemann opened this issue Jan 6, 2023 · 10 comments · Fixed by #2180
Assignees
Labels
Milestone

Comments

@bmwiedemann
Copy link
Contributor

Description

When building the rnp-0.16.2 openSUSE package after 2024-01-19, three tests fail

Steps to Reproduce

  1. on Debian or openSUSE do
  2. osc co openSUSE:Factory/rnp && cd $_
  3. osc build --vm-type=kvm --noservice --clean --build-opt=--vm-custom-opt="-rtc base=2024-01-19T12:00:00" standard

Expected Behavior

Tests should keep passing in future

Actual Behavior

 The following tests FAILED: 
         75 - rnp_tests.test_key_add_userid (Failed)
        207 - rnp_tests.test_ffi_security_profile (Failed)
        252 - cli_tests-EncryptElgamal (Failed)
 Errors while running CTest

The failure is probably from src/lib/sec_profile.cpp SHA-1 depecation

Background:
As part of my work on reproducible builds for openSUSE, I check that software still gives identical build results in the future.
The usual offset is +16 years, because that is how long I expect some software will be used in some places.
This showed up failing tests in our package build.
See https://reproducible-builds.org/ for why this matters.

@ronaldtse
Copy link
Contributor

Thank you @bmwiedemann for reporting this and for suggesting a cause, which is probably correct.

Ping @ni4 and @antonsviridenko on what should be the desired behavior here for the tests.

@ronaldtse ronaldtse moved this from TRIAGE to In Progress in RNP Jan 9, 2023
@ronaldtse ronaldtse moved this from In Progress to Medium priority in RNP Jan 9, 2023
@ni4
Copy link
Contributor

ni4 commented Jan 9, 2023

@bmwiedemann Thanks for reporting!
@ronaldtse I'm thinking on what correct behaviour we should have. We still have tests with expiring keys (and MUST have those), which would expose different behaviour in a number of years.
So I see the following options:

  • change tests behaviour and check current time. However this could bring some problems (for instance, regressions where key would not be considered as expired while it should since test disables this check in 3 years)
  • set timestamp via FFI to some certain value while running the certain test(s) (this would also save us from regenerating test keys every few years)
  • add option like DISABLE_TIME_SENSITIVE_TESTS to run only time-independent tests.

@bmwiedemann
Copy link
Contributor Author

IMHO, treating timestamp as a variable is the most versatile approach. This way you can declare in tests:

  • a key expiring in 2024 should be valid in 2023
  • same key expiring in 2024 should be invalid in 2025

Another approach for keys that are meant to "not be expired" is to set expiry to +1000 years. It does not help with SHA-1 deprecation, though.

@antonsviridenko
Copy link
Contributor

We can generate new keys on each test run with appropriate expiration dates depending on current time, but it will take more time to run the test.

@ronaldtse
Copy link
Contributor

Is most of the time used for key generation due to lack of entropy? Can we supply a mock entropy source for tests for reproducibility and speed?

@antonsviridenko
Copy link
Contributor

or we can just update the expiration date on an already generated key...

@bmwiedemann
Copy link
Contributor Author

10 months to go - what is your plan to fix this issue?

@ni4
Copy link
Contributor

ni4 commented Mar 21, 2023

@bmwiedemann It's on my todo list already, just didn't get to the implementation yet.

@ni4 ni4 added this to the v0.18.0 milestone Apr 28, 2023
@bmwiedemann
Copy link
Contributor Author

3 months to go.

@ronaldtse
Copy link
Contributor

Thank you @bmwiedemann for the reminder...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

4 participants