-
-
Notifications
You must be signed in to change notification settings - Fork 9.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]No supported data to decode. Input type: PEM #16696
Comments
Calling DH setters on a RSA key is not a good idea.. Use the RSA setters.. You also dont check the return value from any of these calls.. |
@slontis
But still getting :
|
I just took your example RSA private key data and put i in a pem file..
|
Did you find your issue? |
@slontis After the private key generation in code above, I am calling the following :
But always getting :
I have setup a breakpoint on SSL_CTX_use_PrivateKey_file, before reaches SSL_CTX_use_PrivateKey_file, the szFileName or Server.key is already exist, and I can open it in Notepad++ while when reading it through SSL_CTX_use_PrivateKey_file is failed with above error. I don't have these problem whenever using 1.1.1 branch and below. And also I cann't reproduce my report if I am restarting my app and the RSA certificate, private, public is already generated/exist. Can you try to generate the certificates (RSA certificate, private, public) and calling function above?, especially the SSL_CTX_use_PrivateKey_file EDIT :
|
You could try the following code which works for me locally...
It might be windoze file related... |
In my part it created only 0 kb file, could it be because I have custom certificate things goin on, here is the complete part of the code :
|
@slontis
The code :
Regarding the issue of :
Could it be regarding line ending that are read & write by OpenSSL 3.0?, in old 1.1.1 OpenSSL 1.1.1 windows is creating and reading Unix UTF-8 line ending (LF, detected by Notepad++) while on OpenSSL 3.0, it using Windows UTF-8 Line endings (CR LF). I have changed my code into :
The "No supported data to decode" seems doesn't happened on first example codes that is copied above, but strangely it only happened in my codebase, also I have tested changed my Windows 10 version from 21H1 into 21H2 the problem still persist with OpenSSL 3.0 only, so I doubt it's a windows problems. |
That's not a 3.0 specific issue - applink has been around for a long time. You need to compile and link the file applink.c into your application: https://www.openssl.org/docs/man3.0/man3/OPENSSL_Applink.html |
@mattcaswell
Is happened in my codebase. |
Is there actually any data in the rsa.pem file? This looks suspicious in your code:
You are opening a BIO for the rsa.pem file (which under the covers calls "fopen" on the file). Then entirely separately you are "fopen"ing the same file and writing data to it. Then you "BIO_free" the BIO which (which under the covers calls "fclose" on the file) and then finally you call fclose again. That looks very strange. Since you never seem to use the "out" BIO it can be removed. All the stuff in your reproducer about creating an X509 and then signing it, as well as all the stuff about generating DH parameters all seem irrelevant to the reproducer since you never do anything with the results. I would remove all of that from your reproducer and shrink it down to the simplest possible code that demonstrates the problem. |
Yes, the rsa.key should not have any content/created before, because I am using code to pre-generated the .key at startup. I have posted before that I have changed the code into the following :
I have tried to simplified the code but haven't been able to reproduce it isolated/simple code/case. Could it be regarding line ending that are read & write by OpenSSL 3.0?, in old 1.1.1 OpenSSL 1.1.1 windows is creating and reading Unix UTF-8 line ending (LF, detected by Notepad++) while on OpenSSL 3.0, it using Windows UTF-8 Line endings (CR LF). |
What happens if you remove all of you code except for the bit calling I would expect the line endings to be handled properly. But it is a plausible theory that something is going wrong there. |
Restarting the app is managed to load/check the private key with SSL_CTX_use_PrivateKey_file , my problem is only happening whenever after generating the key,pem,dhp and then straightly checking it out with SSL_CTX_use_PrivateKey_file , so without app restart/shutdown. Cannot reproduce it with this code :
But from my codebase for 3.0, getting :
This issue doesn't happened with OpenSSL 1.1.1 branch :-( |
I am trying to 'bypass' reading private key from the file through OpenSSL by using fopen into a buffer and convert it back into EVP_KEY, the code is below :
But it failed the SSL_CTX_check_private_key! [EDITED]
But weirdly I am still getting :
|
Managed to reproduce it with minimal test code :
It will failed at "SSL_CTX_use_PrivateKey"
Perhaps relevant with SSL_CTX_use_PrivateKey_file error above :
|
A couple things to try: print the value of |
@kaduk |
My suggestion was not to just add the sleep. My suggestion was to go and look at the file on disk during the sleep. |
@kaduk |
Already tried, the file seems perfectly created and ended, but it's line endings is Windows UTF-8, not Unix UTF-8 which is default in OpenSSL 1.1.1 |
I have posted the reproducible code in #16696 (comment) , should I upload the complete VS 2019 project files?. So the issue is somewhere before SSL_CTX_use_PrivateKey & SSL_CTX_use_PrivateKey_file because using privatekey from either buffer "SSL_CTX_use_PrivateKey" or file read "SSL_CTX_use_PrivateKey_file" getting the same :
Although I cannot get reproducible test code with SSL_CTX_use_PrivateKey_file, but isn't it this is worth to look for, at-least it can be reproducible with buffer way "SSL_CTX_use_PrivateKey"? |
I had this issue with nginx on a freebsd server .. looks like it has nothing to do with the openssl3 upgrade (even though I did that at that server too) but the fix, at least for nginx and probably elsewhere, is to simply rename the cert .pem to *.crt and the key needs a *.key extension then nginx is happy. Nginx likes the certificate to end with .crt and the key as .key. It doesn't understand what a .pem file is. nginx version: nginx/1.24.0 |
Marking as inactive, to be closed at the end of 3.4 dev, barring further input |
I have a RSA private key being generated by the following :
example generated RSA Private key :
the certficate file :
When calling it through function below :
I am getting the following error from OpenSSL :
The Same exact function is working properly on OpenSSL 1.1.1 branch, is this a bug or wrong code?
The text was updated successfully, but these errors were encountered: