Race condition or failure after postgresql startup

[kbarber]

We keep getting intermittent failures in our CI system, here is one example: https://jenkins.puppetlabs.com/view/B_PuppetDB/job/PuppetDB%20Acceptance%20IN%20THE%20CLOUD/680/install_mode=install,label=ec2-coordinator,options_file=postgres,platform=ec2-west-el5-64mda-el5-64a/

The race is here:

�[mNotice: /Stage[main]/Postgresql::Server/Exec[reload_postgresql]: Triggered 'refresh' from 1 events
�[mNotice: /Stage[main]/Postgresql::Server/Service[postgresqld]/ensure: ensure changed 'stopped' to 'running'
Error: /Stage[main]/Puppetdb::Database::Postgresql_db/Postgresql::Db[puppetdb]/Postgresql::Database[puppetdb]/Postgresql_psql[Check for existence of db 'puppetdb']: Could not evaluate: Error evaluating 'unless' clause: 'psql: FATAL:  the database system is starting up
'
�[mNotice: /Stage[main]/Puppetdb::Database::Postgresql_db/Postgresql::Db[puppetdb]/Postgresql::Database[puppetdb]/Exec[/usr/bin/createdb --owner='postgres' --template=template0 --encoding 'UTF8'  'puppetdb']: Dependency Postgresql_psql[Check for existence of db 'puppetdb'] has failures: true

It looks to me like the postgresql_psql provider needs to be made aware of the 'psql: the database system is starting up' state and back-off for a retry later perhaps?

[kbarber]

Another one: https://jenkins.puppetlabs.com/job/PuppetDB%20Acceptance%20IN%20THE%20CLOUD/682/install_mode=upgrade,label=ec2-coordinator,options_file=postgres,platform=ec2-west-el5-64mda-el5-64a/console

This one specifically fails on a missing socket file, but I believe this is also symptomatic with a race condition as well.

[chelnak]

Hello! We are doing some house keeping and noticed that this issue has been open for a long time.

We're going to close it but please do raise another issue if the issue still persists. 😄