New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pydoit compatibility for nbflow #5

Open
wants to merge 14 commits into
base: master
from

Conversation

Projects
None yet
4 participants
@camillescott
Contributor

camillescott commented Jul 15, 2016

Hello there!

I like to use pydoit as my build system; it supports Python3 and is quite extensible. This PR adds a module to nbflow for pydoit.

The folder doit-example shows how it works. It uses a dodo.py file, which follows a similar pattern to the scons example:

from nbflow.doit import create_build_tasks

def task_nbflow():
    yield from create_build_tasks(['analyses'])

Major kudos for your arch making this really easy to write!

Todos:

  • Needs tests ;)
  • Works only with Python 3, which conflicts with he scons module and example notebooks. See here for instructions on putting Python 2 and 3 kernels side by side.
  • Should possibly be its own package, something like nbflow-doit
  • A couple bugs have been fixed which I need to push downstream (more PRs incoming)
  • If keeping in same project, update README

Cheers,
Camille

@jhamrick

This comment has been minimized.

Show comment
Hide comment
@jhamrick

jhamrick Jul 15, 2016

Owner

Thanks! Let's loop @joestubbs too who was thinking of working on exactly this as well.

Owner

jhamrick commented Jul 15, 2016

Thanks! Let's loop @joestubbs too who was thinking of working on exactly this as well.

Show outdated Hide outdated nbflow/doit.py
Show outdated Hide outdated nbflow/extractor.py
Show outdated Hide outdated requirements.txt
@joestubbs

This comment has been minimized.

Show comment
Hide comment
@joestubbs

joestubbs Jul 15, 2016

Thanks for looping me in! I wrote a pydoit.py module with a setup function as we discussed, and the original workflow example builds successfully, but I hadn't added any additional tests. My implementation doesn't add pydoit to the requirements; rather, it assumes the user has installed it themselves and has their own dodo.py file (or task loader) that they can complement with the setup function.

If this would be of use I'm happy to make it available - just let me know.

joestubbs commented Jul 15, 2016

Thanks for looping me in! I wrote a pydoit.py module with a setup function as we discussed, and the original workflow example builds successfully, but I hadn't added any additional tests. My implementation doesn't add pydoit to the requirements; rather, it assumes the user has installed it themselves and has their own dodo.py file (or task loader) that they can complement with the setup function.

If this would be of use I'm happy to make it available - just let me know.

@ajasja

This comment has been minimized.

Show comment
Hide comment
@ajasja

ajasja Aug 8, 2016

Hi!
Just watched the Scipy 2016 talk. Great Idea!
I would also much prefer to use pydoit instead of Scons.
Best,
Ajasja

ajasja commented Aug 8, 2016

Hi!
Just watched the Scipy 2016 talk. Great Idea!
I would also much prefer to use pydoit instead of Scons.
Best,
Ajasja

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment