Attempt to fix inconsistent error

[AndrewJackson2020]

This changes server_login_retry to also cache the error message. This makes it much easier to debug an authentication problem between PgBouncer and Postgres, because now the client always sees the message, not just the first time. Before you'd have to check the PgBouncer logs to find the actual error message.

Fixes #1149

[JelteF]

I think we'd still want the fast fail logic, to be able to quickly surface an error to all waiting clients. But we'd want want that error to contain the original error message too, in addition to the error about the fast failure being triggered.

[AndrewJackson2020]

Yeah I think that makes sense. Will figure out how to make that happen.

[AndrewJackson2020]

Just pushed a new commit that restores the fast fail logic but saves the previous error message to present to clients. Let me know if this is more in the right direction.

Also not sure why this is failing on freebsd and one of the Linux test sets. Will look into this.

[JelteF]

I think this is good overall. For the FreeBSD failure, feel free to just get it to pass by skipping the test. The error reporting on FreeBSD to the client is known to be bad/broken in the test suite.

[AndrewJackson2020]

I think this is good overall. For the FreeBSD failure, feel free to just get it to pass by skipping the test. The error reporting on FreeBSD to the client is known to be bad/broken in the test suite.

Changed the test so that it skips FreeBSD. I'm still seeing FreeBSD fail with the message ld-elf.so.1: /lib/libc.so.7: version FBSD_1.8 required by /usr/local/libexec/cabal/pandoc not found on this PR, #1138, and #1137.

[eulerto]

Indeed. I asked Cirrus Labs about the FreeBSD image and it seems we are using an EOL image. I created ande merged the PR #1183 to fix it.

Revert the skip and fix the other tests.

[AndrewJackson2020]

Indeed. I asked Cirrus Labs about the FreeBSD image and it seems we are using an EOL image. I created ande merged the PR #1183 to fix it.

Revert the skip and fix the other tests.

Can confirm that the image update fixed #1138 and #1137. Still seeing some flakiness on some of the test runs in this PR though.

[AndrewJackson2020]

@JelteF I believe that I was able to get the tests to work on test/test_auth.py::{test_change_server_password_reconnect,test_change_server_password_server_lifetime} by adding a sleep statement at the end. Possibly that test needed more time in order for the final error message to hit the log before the log_contains could register it. Either that or the test is flaky for some reason.

Either way let me know if this needs more work or it looks good to you.