Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a DRBG for s2n #61

Merged
merged 16 commits into from
Apr 3, 2015
Merged

Create a DRBG for s2n #61

merged 16 commits into from
Apr 3, 2015

Conversation

colmmacc
Copy link
Contributor

This pull request adds a NIST SP800-90A conformant CTR_DRBG, which removes /dev/urandom as our bottleneck. Some high-lights:

  • It uses AES128 CTR as the generator. I'd like to replace this with ChaCha20, but this is not yet in regular OpenSSL.
  • Every thread gets two DRBG instances: one for private data and one for public. These are in thread local storage and new threads will get their own DRBGs (there are tests for this). Process inheritance is handled with pthread_atfork, or minherit() where available - in either case the DRBGs are re-initialized on fork().
  • NIST test vectors have been added and the DRBG produces output that matches these test vectors.
  • The DRBG is approximately 200 times faster than reading from /dev/urandom on my platforms.

This change refactors s2n's entropy handling to be able to use a
DRBG. The DRBG included is simply the stream output of AES256-CTR
and is not yet conformant with NIST SP 800-90A, which will come
in a subsequent commit.

Highlights:

    s2n_drbg emulates the NIST DRBG API and generates bits and
    automatically reseeds itself.

    s2n_get_public_random_data / s2n_get_private_random_data
    have been added, each backed by different DRBG instantiations.

    s2n_get_random_data -> s2n_get_urandom_data

    all of the random generators have been factored to produce
    blobs

    The TLS code has been modified to use the public random data
    where appropriate.

    The OpenSSL over-ride now uses the private random data.

    The public/private DRBGs are kept in thread-local storage:
    so every thread gets its own automatically.

    Defensiveness against fork()'ing has been added, including
    support for minherit() where available.

    Tests have been added and tweak, including full coverage
    of the multi-thread and multi-process cases.
This change makes our DRBG conformant with NIST SP800-90A and now
matches the NIST test vectors.
Conflicts:
	error/s2n_errno.c
	error/s2n_errno.h
{
AES_KEY ctx;

AES_set_encrypt_key(key, 128, &ctx);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to call aes_free(&ctx) when we are done with it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So there's no aes_free() because it's a flat struct that isn't malloc()'d, but this is a good call out. I've restructured things to move the AES_KEY into the drbg struct. I've added an s2n_drbg_wipe call to zero that key and all other DRBG context, and I now wipe the DRBGs from s2n_cleanup().

@baldwinmatt
Copy link
Contributor

first pass i was mainly looking at the code layout, looks good.

i'll go through again with the NIST docs at my side - oh and i trust you constructed your test vectors correctly! :)

This change also creates a define for S2N_DRBG_BLOCK_SIZE for the
underlying CTR block size.
This change moves the AES_KEY context to the drbg struct, rather than on a function stack,
and adds an s2n_drbg_wipe() call to explicitly wipe all of the DRBG state. The public/private
DRBGs are now also wiped when s2n_cleanup is called.
This commit makes the DRBG useable with BoringSSL and fixes a few
Ubuntu GCC warnings/errors:

    * DRBG code now uses the EVP interface, which is the only
      working interface that BoringSSL provides

    * Fix several zero initialization and cast warnings detected
      by Ubuntu GCC
Conflicts:
	tests/unit/s2n_handshake_test.c
@baldwinmatt
Copy link
Contributor

Just need to check your nist vectors :)

i'll email you the results of the valgrind, but looks like we are leaking the cipher ctx, in the test at least. a call to s2n_drbg_wipe might be in order at the end of the test, here is an example:

==32339== 3,432 bytes in 13 blocks are definitely lost in loss record 9 of 9
==32339== at 0x4C2CFA7: malloc (vg_replace_malloc.c:296)
==32339== by 0x50739C5: EVP_CipherInit_ex (cipher.c:166)
==32339== by 0x5073CF4: EVP_EncryptInit_ex (cipher.c:239)
==32339== by 0x5067D18: s2n_drbg_update (in /home/local/ANT/baldwinm/dev/awslabs/s2n-baldwinm/lib/libs2n.so)
==32339== by 0x506809A: s2n_drbg_generate (in /home/local/ANT/baldwinm/dev/awslabs/s2n-baldwinm/lib/libs2n.so)
==32339== by 0x4010AE: main (s2n_drbg_test.c:173)
==32339==
{
<insert_a_suppression_name_here>
Memcheck:Leak
match-leak-kinds: definite
fun:malloc
fun:EVP_CipherInit_ex
fun:EVP_EncryptInit_ex
fun:s2n_drbg_update
fun:s2n_drbg_generate
fun:main
}

GUARD(s2n_drbg_update(drbg, &blob));

drbg->bytes_used = 0;
drbg->generation += 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to ensure we don't overflow this?

although unlikely, it's still possible. what is the impact?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generation variable isn't exposed by any API and it's only used in one place: the unit test. The reason it's there is so that a unit test could cover the case where we cross the 2^35 boundary and check that we reseed the DRBG. I think it's ok to let it overflow. It will overflow only after a DRBG has created 2^97 bits of data.

@baldwinmatt
Copy link
Contributor

Ok, I like that better - tested your test vectors and they are all correct :)

merging!

baldwinmatt added a commit that referenced this pull request Apr 3, 2015
@baldwinmatt baldwinmatt merged commit 7483b20 into aws:master Apr 3, 2015
@colmmacc
Copy link
Contributor Author

colmmacc commented Apr 3, 2015

Thanks! I appreciate the review and especially for digging through the NIST docs and test vectors. I know it makes the review at least 10 times more work than reading the code alone!

colmmacc added a commit that referenced this pull request Jun 29, 2015
This change also creates a define for S2N_DRBG_BLOCK_SIZE for the
underlying CTR block size.
colmmacc added a commit that referenced this pull request Jun 29, 2015
baldwinmatt added a commit that referenced this pull request Jun 29, 2015
feliperodri pushed a commit to feliperodri/s2n that referenced this pull request Dec 6, 2021
1.  Add necessary cbmc-proofs.txt file to contracts-based proof directories.
2.  Update PROOF_UID for some contracts-based proofs.
3.  Remove occurrences where text was appended to variables, such as this one:

    FUNCTION_NAME = __CPROVER_file_local_s2n_hash_c_s2n_evp_hash_copy
    PROOF_UID = $(FUNCTION_NAME)_with_contracts
    HARNESS_ENTRY = $(PROOF_UID)_harness
    HARNESS_FILE = $(HARNESS_ENTRY).c

    The above approach doesn't yield the expected variables of:

    PROOF_UID = __CPROVER_file_local_s2n_hash_c_s2n_evp_hash_copy_with_contracts

    I changed all "$(VARIABLE_NAME)" occurrences with the string itself.
feliperodri pushed a commit to feliperodri/s2n that referenced this pull request Dec 6, 2021
feliperodri pushed a commit to feliperodri/s2n that referenced this pull request Dec 6, 2021
1.  Add necessary cbmc-proofs.txt file to contracts-based proof directories.
2.  Update PROOF_UID for some contracts-based proofs.
3.  Remove occurrences where text was appended to variables, such as this one:

    FUNCTION_NAME = __CPROVER_file_local_s2n_hash_c_s2n_evp_hash_copy
    PROOF_UID = $(FUNCTION_NAME)_with_contracts
    HARNESS_ENTRY = $(PROOF_UID)_harness
    HARNESS_FILE = $(HARNESS_ENTRY).c

    The above approach doesn't yield the expected variables of:

    PROOF_UID = __CPROVER_file_local_s2n_hash_c_s2n_evp_hash_copy_with_contracts

    I changed all "$(VARIABLE_NAME)" occurrences with the string itself.
feliperodri pushed a commit to feliperodri/s2n that referenced this pull request Dec 6, 2021
feliperodri pushed a commit to feliperodri/s2n that referenced this pull request Feb 15, 2022
1.  Add necessary cbmc-proofs.txt file to contracts-based proof directories.
2.  Update PROOF_UID for some contracts-based proofs.
3.  Remove occurrences where text was appended to variables, such as this one:

    FUNCTION_NAME = __CPROVER_file_local_s2n_hash_c_s2n_evp_hash_copy
    PROOF_UID = $(FUNCTION_NAME)_with_contracts
    HARNESS_ENTRY = $(PROOF_UID)_harness
    HARNESS_FILE = $(HARNESS_ENTRY).c

    The above approach doesn't yield the expected variables of:

    PROOF_UID = __CPROVER_file_local_s2n_hash_c_s2n_evp_hash_copy_with_contracts

    I changed all "$(VARIABLE_NAME)" occurrences with the string itself.
feliperodri pushed a commit to feliperodri/s2n that referenced this pull request Feb 15, 2022
feliperodri pushed a commit to feliperodri/s2n that referenced this pull request Jun 16, 2022
1.  Add necessary cbmc-proofs.txt file to contracts-based proof directories.
2.  Update PROOF_UID for some contracts-based proofs.
3.  Remove occurrences where text was appended to variables, such as this one:

    FUNCTION_NAME = __CPROVER_file_local_s2n_hash_c_s2n_evp_hash_copy
    PROOF_UID = $(FUNCTION_NAME)_with_contracts
    HARNESS_ENTRY = $(PROOF_UID)_harness
    HARNESS_FILE = $(HARNESS_ENTRY).c

    The above approach doesn't yield the expected variables of:

    PROOF_UID = __CPROVER_file_local_s2n_hash_c_s2n_evp_hash_copy_with_contracts

    I changed all "$(VARIABLE_NAME)" occurrences with the string itself.
feliperodri pushed a commit to feliperodri/s2n that referenced this pull request Jun 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants