Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AES GCM without additional authenticated data crashes SoftHSM #664

Merged
merged 4 commits into from Jan 5, 2022

Conversation

saper
Copy link
Contributor

@saper saper commented Jan 1, 2022

When building SoftHSMv2 with C++ library assertions enabled (as this is the case of SoftHSMv2 RPM for Red Hat Enterprise Linux), AES GCM encryption fails when no additional authenticated data are supplied:

    * thread #1, name = 'p11test', stop reason = signal SIGABRT
      * frame #0: 0x0000000800aa569a libc.so.7`__sys_thr_kill + 10
        frame #1: 0x0000000800aa3af4 libc.so.7`__raise + 52
        frame #2: 0x0000000800a19719 libc.so.7`abort + 73
        frame #3: 0x00000008007df9a2 libc++.so.1`std::__1::__libcpp_abort_debug_function(std::__1::__libcpp_debug_info const&) + 82
        frame #4: 0x00000000003beb6a p11test`std::__1::vector<unsigned char, SecureAllocator<unsigned char> >::operator[](unsigned long) + 122
        frame #5: 0x00000000003be096 p11test`ByteString::operator[](unsigned long) + 38
        frame #6: 0x00000000003412a6 p11test`SoftHSM::SymEncryptInit(unsigned long, _CK_MECHANISM*, unsigned long) + 3190
        frame #7: 0x000000000034267b p11test`SoftHSM::C_EncryptInit(unsigned long, _CK_MECHANISM*, unsigned long) + 75
        frame #8: 0x00000000003363b7 p11test`C_EncryptInit + 55
        frame #9: 0x0000000000283cfb p11test`SymmetricAlgorithmTests::encryptDecrypt(_CK_MECHANISM, unsigned long, unsigned long, unsigned long, unsigned long, bool) + 555
        frame #10: 0x000000000028c682 p11test`SymmetricAlgorithmTests::testAesEncryptDecrypt() + 3378

(this backtrace is from FreeBSD using clang and c++)

This refactors the AES tests and enables additional assertions in the Travis CI build.

Refactor encryptDecrypt to use mechanism details supplied from the caller.
This way it can be used to try various variable parameters for AES modes.
With C++ library assertions enabled, this crashes the p11test:

* thread opendnssec#1, name = 'p11test', stop reason = signal SIGABRT
  * frame #0: 0x0000000800aa569a libc.so.7`__sys_thr_kill + 10
    frame opendnssec#1: 0x0000000800aa3af4 libc.so.7`__raise + 52
    frame opendnssec#2: 0x0000000800a19719 libc.so.7`abort + 73
    frame opendnssec#3: 0x00000008007df9a2 libc++.so.1`std::__1::__libcpp_abort_debug_function(std::__1::__libcpp_debug_info const&) + 82
    frame opendnssec#4: 0x00000000003beb6a p11test`std::__1::vector<unsigned char, SecureAllocator<unsigned char> >::operator[](unsigned long) + 122
    frame opendnssec#5: 0x00000000003be096 p11test`ByteString::operator[](unsigned long) + 38
    frame opendnssec#6: 0x00000000003412a6 p11test`SoftHSM::SymEncryptInit(unsigned long, _CK_MECHANISM*, unsigned long) + 3190
    frame opendnssec#7: 0x000000000034267b p11test`SoftHSM::C_EncryptInit(unsigned long, _CK_MECHANISM*, unsigned long) + 75
    frame opendnssec#8: 0x00000000003363b7 p11test`C_EncryptInit + 55
    frame opendnssec#9: 0x0000000000283cfb p11test`SymmetricAlgorithmTests::encryptDecrypt(_CK_MECHANISM, unsigned long, unsigned long, unsigned long, unsigned long, bool) + 555
    frame opendnssec#10: 0x000000000028c682 p11test`SymmetricAlgorithmTests::testAesEncryptDecrypt() + 3378
@saper saper mentioned this pull request Jan 3, 2022
@rijswijk rijswijk merged commit 28c67fe into opendnssec:develop Jan 5, 2022
1 check failed
@saper
Copy link
Contributor Author

saper commented Jan 6, 2022

Thanks @rijswijk for taking care of this so quickly! 🚀

@Leont
Copy link

Leont commented Mar 28, 2023

Just ran into this same issue. Could a new version with this fix be released so it actually percolates to distributions?

@saper
Copy link
Contributor Author

saper commented Mar 28, 2023

good question, looks some recent changes broke some parts of the build (like Windows stuff), I wouldn't hold my breath.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants