[exec,lib] partial revert of using main_session for KlioConfig #147
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a partial revert for f58caf7.
In some cases for unknown reasons,
RunConfig.get()
will begin raising an exception that the config is no longer set. This should not be happening, and in some instances the problem doesn't show up until a streaming job has been running for hours, but there is likely some underlying issue in Beam/Dataflow that is leading the main session variable to be deleted.This PR is not intended to be a complete fix, but rather a mitigation that reverts back to
klioexec
getting its config via theeffective-klio-job.yaml
that should be included with the packaged code.I've left the
klio-cli
changes untouched, meaning it will continue to write a temp file and bind it for use byklioexec
.klioexec
will also still set theKlioConfig
to the main session variable withRunConfig.set()
, even though for now it won't be used.Checklist for PR author(s)
[cli] Fixes bugs in 'klio job fake-cmd'
.docs/src
.