Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DBRunner enhancements for ingest, update-only, force (Closes #9) #26

Merged
merged 16 commits into from Sep 30, 2020

Conversation

jtniehof
Copy link
Member

@jtniehof jtniehof commented Sep 4, 2020

NOTE: This PR conflicts with #24, in that it includes everything from #24. Either this can be merged and #24 closed, or when #24 is merged I'll rebase this one to the new master.

This PR enhances DBRunner in several ways:

  • Allows execution of processes with no inputs
  • Adds option to ingest the created files
  • Adds option to bump the version to avoid DB conflicts, or only produce files which are out of date

This closes #9 by implementing the checklist/design there.

I had #24 open to get the testing-related changes out faster and hopefully have them reviewed before I did the functionality here. So if someone has time to review this one, it will get everything...if only a little time, then #24 can come first (and this one will have to wait for the rebase.)

PR Checklist

  • Pull request has descriptive title
  • Pull request gives overview of changes
  • New code has inline comments where necessary
  • Any new modules, functions or classes have docstrings consistent with dbprocessing style
  • Major new functionality has appropriate Sphinx documentation
  • Added an entry to CHANGELOG if fixing a major bug or providing a major new feature (Ooops, we don't have a CHANGELOG set up yet, should probably do so and/or edit template)
  • New features and bug fixes should have unit tests
  • Relevant issues are linked in the description (e.g. See issue # or Closes #)

@jtniehof jtniehof added the enhancement New feature or feature request label Sep 4, 2020
@jtniehof
Copy link
Member Author

jtniehof commented Sep 8, 2020

Updated DBRunner to take process name as well as process ID on the command line, and fixed up a few bugs I found in the meantime.

@jtniehof
Copy link
Member Author

Rebased since #24 was merged; this still needs review/merge.

 * Should be no change to functionality.
 * --force flag to always bump version.
 * -u/--update flag to only run for out-of-date files (not there
   or new code).
 * This required a fix for runMe's "new parent" check when no inputs.
 * Also fix rendering of DBRunner script documentation.
 * Not functional yet.
 * There are new arguments that mean files are made sometimes, not always.
 * The actual runMe runner doesn't check this, so if not running in
   pure-force mode, check it to make sure not making files that conflict
   with the database.
 * 4cc7b44 removed runMe from list if
   ableToRun was false, test wasn't updated for that.
@dnadeau-lanl dnadeau-lanl merged commit 5c05a16 into spacepy:master Sep 30, 2020
@jtniehof jtniehof deleted the no_inputs branch September 30, 2020 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or feature request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support products with no input
2 participants