Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom source loading #278

Merged
merged 19 commits into from
Sep 14, 2020
Merged

Add custom source loading #278

merged 19 commits into from
Sep 14, 2020

Conversation

shyamd
Copy link
Contributor

@shyamd shyamd commented Sep 11, 2020

This PR enables adding a custom source file such as .py or .ipynb file. The only requirements are as follows:

  1. The source file has to be in a subfolder from where mrun is called. This is a limitation of the pathfinding that has to convert a module import command into a path
  2. The custom python file has to include either a __builder__ or __builders__ defined somewhere in the file. This is what gets loaded by mrun to run.

Addresses #277

Examples:
mrun -n 2 -v builder_test.py
mrun -n 2 -v builder_test.ipynb
mrun -n 2 -v main_builders.json builder_test.py

@lgtm-com
Copy link

lgtm-com bot commented Sep 12, 2020

This pull request introduces 2 alerts when merging 828727b into b45d769 - view on LGTM.com

new alerts:

  • 2 for Unused import

@mkhorton
Copy link
Member

This is going to be so useful :)

@lgtm-com
Copy link

lgtm-com bot commented Sep 12, 2020

This pull request introduces 1 alert when merging 5e74854 into b45d769 - view on LGTM.com

new alerts:

  • 1 for Unused import

@codecov
Copy link

codecov bot commented Sep 14, 2020

Codecov Report

Merging #278 into master will increase coverage by 0.19%.
The diff coverage is 92.38%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #278      +/-   ##
==========================================
+ Coverage   88.30%   88.49%   +0.19%     
==========================================
  Files          27       28       +1     
  Lines        2026     2121      +95     
==========================================
+ Hits         1789     1877      +88     
- Misses        237      244       +7     
Impacted Files Coverage Δ
src/maggma/cli/source_loader.py 91.95% <91.95%> (ø)
src/maggma/cli/__init__.py 93.47% <92.30%> (+1.17%) ⬆️
src/maggma/cli/multiprocessing.py 94.59% <100.00%> (+0.04%) ⬆️
src/maggma/stores/aws.py 82.50% <100.00%> (ø)
src/maggma/stores/gridfs.py 89.20% <100.00%> (ø)
src/maggma/stores/mongolike.py 93.62% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6ba742b...f10552d. Read the comment docs.

@lgtm-com
Copy link

lgtm-com bot commented Sep 14, 2020

This pull request introduces 3 alerts when merging 62ee2c9 into 6ba742b - view on LGTM.com

new alerts:

  • 2 for Unused import
  • 1 for Module is imported with 'import' and 'import from'

@lgtm-com
Copy link

lgtm-com bot commented Sep 14, 2020

This pull request fixes 1 alert when merging f10552d into 6ba742b - view on LGTM.com

fixed alerts:

  • 1 for Unused import

@shyamd shyamd merged commit af978d6 into master Sep 14, 2020
@shyamd shyamd deleted the source_loader branch November 17, 2020 21:25
@shyamd shyamd mentioned this pull request Nov 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants