New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG] Better dag handling #69

Merged
merged 8 commits into from Feb 4, 2019

Conversation

Projects
None yet
3 participants
@bluegenes
Copy link
Collaborator

bluegenes commented Jan 30, 2019

Don't print to stdout if building a dag. This change means that we can redirect the output directly to a dot file, or use graphviz to get an image (rather than needing to delete the extra lines first).

./run_eelpond examples/nema.yaml default --dag > dagfile.dot

or

./run_eelpond examples/nema.yaml default --dag | dot -Tpng > dagfile.png

addresses #11 - provides all the functionality, though it doesn't enable an easy --dagfile FILENAME flag. Thoughts @charlesreid1?

@bluegenes bluegenes changed the title Better dag handling [MRG] Better dag handling Jan 30, 2019

add flags --dagfile and --dagpng
This wraps the Snakemake API call in a
custom context manager class that checks
if we are printing a DAG, and if we are,
when we enter the context, sets up stdout
to be a file buffer, and assembles the
buffer contents into a list when done.
Show resolved Hide resolved run_eelpond Outdated
@charlesreid1

This comment has been minimized.

Copy link
Contributor

charlesreid1 commented Jan 31, 2019

I think having the --dag flag to dump straight to output is a good idea, so that you can pipe it to scripts or other command line utilities, but I also think it's good to have a flag to dump to a file or to a png.

I implemented a context manager that will capture stdout when the user is creating the directed acyclic graph, and either prints it to the screen (in the case of --dag, which will continue to work as you implemented it), or prints it to a file (if the user passes the --dagfile flag), or uses dot to convert the dotfile into a png file (if the user passes the --dagpng flag).

I opened a pull request (#73) against this pull request so you can take a look. I'll also add a few tests of the new functionality.

charlesreid1 and others added some commits Jan 31, 2019

Add eelpond tests/ directory (#75)
* Add eelpond tests/ directory

This adds individual unit tests for
each of eelpond's flags.

Not all of these flag unit tests are
implemented yet, but the --dag and
--dagfile flags are tested, as are
the --help and --dry-run flags,
to demonstrate how to do it.

* fill in --dagpng test. clean up test_flags test

* add pytest to environment.yml, add pytest command to travis
Merge pull request #73 from dib-lab/cmr_better_dag_handling
Further improvements to DAG handling
@bluegenes

This comment has been minimized.

Copy link
Collaborator Author

bluegenes commented Feb 4, 2019

@ctb - @charlesreid1 added a testing framework! Works and looks good to me, but your review would be helpful

@ctb

This comment has been minimized.

Copy link
Member

ctb commented Feb 4, 2019

@bluegenes

This comment has been minimized.

Copy link
Collaborator Author

bluegenes commented Feb 4, 2019

it was merged into this branch/PR, not into master i think...?

@ctb

This comment has been minimized.

Copy link
Member

ctb commented Feb 4, 2019

@bluegenes bluegenes merged commit b22a38a into master Feb 4, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@charlesreid1

This comment has been minimized.

Copy link
Contributor

charlesreid1 commented Feb 5, 2019

it was merged into this branch/PR, not into master i think...?

yup, that's right - the inspiration for the tests was testing the new --dagfile and --dagpng flags so I added it into the DAG-related pull requests. Thanks for merging!

@bluegenes bluegenes deleted the better_dag_handling branch Feb 11, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment