Skip to content

Conversation

@rhenium
Copy link
Member

@rhenium rhenium commented Jul 21, 2025

The ossl_*_new() functions are confusingly inconsistent in how they handle the lifetime of the passed OpenSSL object. This PR makes the behavior consistent, remove dead code, and fix potential memory leaks due to allocating Ruby objects after allocating OpenSSL objects.


bn: avoid ossl_bn_new(NULL)

Currently, calling ossl_bn_new() with a NULL argument allocates a new
OpenSSL::BN instance representing 0. This behavior is confusing. Raise
an exception if this is attempted, instead.


x509: disallow ossl_x509{,attr,crl,ext,revoked,name}*_new(NULL)

These functions are not actually called with NULL. It also doesn't make
sense to do so, so let's simplify the definitions.


pkcs7: disallow ossl_pkcs7{si,ri}_new(NULL)

These functions are not actually called with NULL.


ocsp: refactor ossl_ocspsres_new()

Similar to most of the other ossl_*_new() functions, let it take a const
pointer and make a copy of the object.

This also fixes a potential memory leak when the wrapper object
allocation fails.


ocsp: refactor ossl_ocspcertid_new()

Likewise, let it take a const pointer and not the ownership of the
OpenSSL object.

This fixes potential memory leak in OpenSSL::OCSP::BasicResponse#status.


pkey: rename ossl_pkey_new() to ossl_pkey_wrap()

Among functions named ossl_*_new(), ossl_pkey_new() is now the only one
that takes ownership of the passed OpenSSL object instead of making a
copy or incrementing its reference counter. Rename it to make this
behavior easier to understand.

rhenium added 6 commits July 21, 2025 23:35
Currently, calling ossl_bn_new() with a NULL argument allocates a new
OpenSSL::BN instance representing 0. This behavior is confusing. Raise
an exception if this is attempted, instead.
These functions are not actually called with NULL. It also doesn't make
sense to do so, so let's simplify the definitions.
These functions are not actually called with NULL.
Similar to most of the other ossl_*_new() functions, let it take a const
pointer and make a copy of the object.

This also fixes a potential memory leak when the wrapper object
allocation fails.
Likewise, let it take a const pointer and not the ownership of the
OpenSSL object.

This fixes potential memory leak in OpenSSL::OCSP::BasicResponse#status.
Among functions named ossl_*_new(), ossl_pkey_new() is now the only one
that takes ownership of the passed OpenSSL object instead of making a
copy or incrementing its reference counter. Rename it to make this
behavior easier to understand.
@rhenium rhenium merged commit f7114e9 into ruby:master Jul 27, 2025
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant