Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Postgres connection error in multi-process case (DM-26539) #69

Merged
merged 1 commit into from Aug 31, 2020

Conversation

andy-slac
Copy link
Collaborator

My previous workaround for passing butler to subprocess was broken
because new connection was made in the parent process. Replacing with a
new workaround that passes butler in pickle form across fork.

Copy link
Member

@timj timj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change because pickling guarantees that each fork gets a new butler, whereas before every fork was getting the same butler?

python/lsst/ctrl/mpexec/mpGraphExecutor.py Outdated Show resolved Hide resolved
My previous workaround for passing butler to subprocess was broken
because new connection was made in the parent process. Replacing with a
new workaround that passes butler in pickle form across fork.
@andy-slac
Copy link
Collaborator Author

Is this change because pickling guarantees that each fork gets a new butler, whereas before every fork was getting the same butler?

Not quite. Before this each fork also had a new butler but that instance was shared with parent process (butler was instantiated by parent) and parent and child had a connection to database which parent closed immediately after "passing" butler to child process. That clearly confused server because child tried to use connection that was closed already (whatever that means for Postgres server). New code avoids instantiating butler in parent process, butler is created by child process now, parent and child only share pickle bytes which has no state.

@andy-slac andy-slac merged commit 18fd799 into master Aug 31, 2020
@andy-slac andy-slac deleted the tickets/DM-26539 branch November 11, 2020 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants