Do not leak psycopg2 connections during testing which can lead to fla…

[gkokolatos]

…kyness

According to psycopg2 documentation connections are not closed on context exit.
This is in contrast with connection cursors that actually do so.

It is not an inherent problem in itself, however, calls to get_state() in a
loop, such as from wait_until_*, can overflow the connection limits and cause
the test suite to fail. This is particularly noticable when the system is slow
to respond, for example if it is run under valgrind.

This might address other objerved flakyness in the tests due to connection
exhaustion with messages as:
"psycopg2.OperationalError: FATAL: sorry, too many clients already"

Run into this repeatedly while profiling in order to address #582
Might be related with the comment in #512#issuecomment-767205595

[DimCitus]

Do we still need to have a context handler on the connection object if the context handler doesn't close the connection anyway? What it is used for then?

[gkokolatos]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Wednesday, March 17, 2021 10:56 AM, Dimitri Fontaine ***@***.***> wrote: @DimCitus approved this pull request. --------------------------------------------------------------- In [tests/pgautofailover_utils.py](#628 (comment)): > @@ -359,14 +359,21 @@ def run_sql_query(self, query, *args): Runs the given sql query with the given arguments in this postgres node and returns the results. Returns None if there are no results to fetch. """ - with psycopg2.connect(self.connection_string()) as conn: - cur = conn.cursor() - cur.execute(query, args) - try: - result = cur.fetchall() - return result - except psycopg2.ProgrammingError: - return None + result = None + conn = psycopg2.connect(self.connection_string()) + + with conn: Do we still need to have a context handler on the connection object if the context handler doesn't close the connection anyway? What it is used for then?

I understand your question. I am not going to claim that I am python nor a psycopg2 expert. However the documentation of psycopg2 ([https://www.psycopg.org](https://www.psycopg.org/docs/connection.html)) gives this format as best practice so obliged. In either case it does not hurt, does it?

…

— You are receiving this because you authored the thread. Reply to this email directly, [view it on GitHub](#628 (review)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/ALBTIO7VXHMPHJRTIC7LW73TEB4FRANCNFSM4ZI65TDQ).

[DimCitus]

Ah yeah that's per psycopg2 docs. Perfect then, many thanks again for your contributions!

[gkokolatos]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Wednesday, March 17, 2021 11:52 AM, Dimitri Fontaine ***@***.***> wrote: Ah yeah that's per psycopg2 docs. Perfect then, many thanks again for your contributions!

Thank you for the project!

…

— You are receiving this because you were assigned. Reply to this email directly, [view it on GitHub](#628 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/ALBTIO6ADL5NUZYDGBST32TTECCXHANCNFSM4ZI65TDQ).