Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DB Retry Behavior #144

Merged
merged 3 commits into from
Jan 24, 2020
Merged

DB Retry Behavior #144

merged 3 commits into from
Jan 24, 2020

Conversation

tiger5226
Copy link
Contributor

Don't just error on db connections. DBs lose connections, it happens. Retry periodically, with increasing intervals before finally returning an error to connect to the database.

This is also beneficial for our container compositions. If postgres does not startup in time, it will retry.

Screen Shot 2020-01-03 at 9 46 21 PM

internal/storage/conn.go Outdated Show resolved Hide resolved
internal/storage/conn.go Outdated Show resolved Hide resolved
@lbry-bot lbry-bot assigned tiger5226 and unassigned anbsky Jan 6, 2020
break
}
secondsToWait := i + 1
c.logger.Log().Warning("Database Connection Err: ", err)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit of a nitpick, but those two log entries are not guaranteed to reach the physical log stream together. Why not put it all in a single message?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No particular reason, besides I thought you would want them separated out because one is an error and one is an info statement. You never specified what you wanted, so I had to assume brother. so I went with a warning message because the error was causing a failure. I don't really have a strong opinion. So be specific and I will change it to whatever you want it to be.

Copy link
Collaborator

@anbsky anbsky Jan 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A single Warning containing both a database error details and a retry details is fine here

@lbry-bot lbry-bot assigned tiger5226 and unassigned tiger5226 Jan 9, 2020
@tiger5226
Copy link
Contributor Author

@sayplastic How does coverage decrease if the function adjusted has a unit test covering it?

@anbsky
Copy link
Collaborator

anbsky commented Jan 14, 2020

@sayplastic How does coverage decrease if the function adjusted has a unit test covering it?

Test coverage is not binary, you added a logical branch to the function that the existing test does not cover.

@kauffj
Copy link

kauffj commented Jan 14, 2020

@tiger5226, @sayplastic says you can and should merge this

@tiger5226
Copy link
Contributor Author

All clear. I was just curious. I did not know if was that detailed of a check. Good to know.

… Retry periodically, with increasing intervals before finally returning an error to connect to the database.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants