New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Jupyter support #52

Open

sebhoerl opened this issue Nov 13, 2020 · 1 comment

Contributor

sebhoerl commented Nov 13, 2020

It would be nice to have functions to run things with synpp from Jupyter, for instance stage by stage and to examine the results directly, e.g.

pipeline = synpp.setup("config_ile_de_france.yml")
df_census = pipeline.run("data.census.cleaned")
# Do something with df_census

df_census = pipeline.run("data.census.cleaned", stages = {
  "data.census.raw": SOME_DATA # Override stage output
})

davibicudo mentioned this issue

Feature/stageclasses #56

Merged

Contributor

davibicudo commented Aug 18, 2021

With stage decoration maybe a workaround could be:

df_census = synpp.run([synpp.stage("data.census.cleaned", **{"data.census.raw": synpp.stage(lambda: SOME_DATA)})])

The dependency "data.census.raw" is replaced by another fake stage produced with the lambda.

I haven't tested it though, it would probably break the cache...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment