Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Concurrent calls to psycopg2.connect() crash on Windows with both binary and source builds of psycopg2 #836
In one of our Python scripts that is being run on Windows, we have switched from MySQL (
Steps to reproduce
The script runs successfully.
The script crashes inside
The stacktrace above indicates that the crash happens inside OpenSSL. Do you have any hints what might be causing this issue or how to proceed with the debugging?
This was referenced
Apr 9, 2019
So I tested out the development package. When testing against a PostgreSQL 11.2 Windows server, it fails to connect up in SSL mode. So I fixed the AppVeyor config to setup the local PostgreSQL server in SSL mode, and it connects up and passes those tests. But when I download the packages that pass the AppVeyor tests, it still does not connect up to the Windows PostgreSQL server.
I am going to have to do some more testing against different versions, as we are not there yet, but I'm hoping it is user (my) error.
BTW, I've issued a pull request to you for the SSL changes.
Not much farther along with a solution, but wanted to document where I'm at.
The error I see when using psycopg2 with the OpenSSL 1.1.1 build outside of Appveyor is:
This error is passed through from libpq. Ironically, there is not a lot out there on this error with the exception of this stackoverflow question. With this issue, someone had an issue with an Anaconda version of psycopg2, and they solved it by uninstalling it and installing the default pip version.
I see this error with both 32 and 64bit Python, and with Python 2.7 and Python 3.7. I also see the error when connecting up to a Unix PostgreSQL server (version 9.6) and a Windows PostgreSQL server (version 11.2). If I 'downgrade' Psycopg2 to the 2.8.1 version (or even 22.214.171.124), it works.
Another item to note: Modifying the Appveyor build to require SSL connections with the PostgreSQL server does NOT have the error show up.
This leaves me with a few questions:
Great google-fu there @dvarrazzo! Using the information in that npm post, I think it explain what is seen.
In my test environments, if I create the environment variable OPENSSL_CONF and point it to a file that exists (could be an empty file), then your build works as expected. In the Appveyor environment, the OPENSSL_CONF environment variable is set, which explains why it works there. The good news is we can unset the variable in Appveyor and have it fail.
The challenge now is how to work around it requiring the config file. I'll see what I can do with that.
Cute that if it doesn't find a file it tells you it doesn't find a process <3
I thought as well that as a test strategy it could be good to unset that env var and see it fail.
How to not make it fail... Should we bundle an
I don't know if this from libpq ssl support docs offers further info.
The default is whatever set on config via
Think I have a working solution.
There is a #define for the PostgreSQL build,
The solution is defining HAVE_OPENSSL_INIT_SSL, which takes the newer code path which the bug/feature does not show up in. I was initially surprised that it wasn't defined before, but then I realized this define was in the PostgreSQL build, which must not be able to determine what version of OpenSSL it is building against (at least in Windows).
I'll issue a pull request once it runs through and tests all the Windows builds.