Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make examples manageable for demo'ing and exploring #2776

Closed
schrockn opened this issue Aug 4, 2020 · 3 comments
Closed

Make examples manageable for demo'ing and exploring #2776

schrockn opened this issue Aug 4, 2020 · 3 comments
Labels
area: docs Related to documentation in general area: docs-infra Related to documentation site infra and tooling integration: notebook Related to dagstermill
Projects

Comments

@schrockn
Copy link
Member

schrockn commented Aug 4, 2020

While our new examples hierarchy is nicer organizationally it's been a substantial regression in terms of navigating around to different example pipelines. You end up having to kill and restart dagit with different workspaces all the time, which is a huge bummer.

I believe we need a new workspace.yaml that can merge together all the pipelines reasonably. However doing this naively may result in a a ton of repository locations which also has its downsides. But right now it is really hard to hop around the codebase (I struggled in vain to be able to load the longitidunal_pipeline without changing code).

@natekupp
Copy link
Contributor

natekupp commented Aug 4, 2020

It might be worth having a separate repo w/ examples intended for live demos / presentations vs. examples intended to showcase the full surface area of our public APIs:

  1. For demos/presentations, I assume the audience is new to Dagster, and the goals should be to elegantly and systematically demonstrate the capabilities of the system, starting from "hello, world" and building towards specific use cases—ETL on AWS, ML in dagstermill, etc.
  2. For user examples, I assume the user is already reasonably familiar with the fundamentals of Dagster and is trying to solve a specific problem—so the examples can be highly targeted, MVP examples of "how can I do X with Dagster?"

Definitely overlapping use cases, but I wonder if we can have a more streamlined demo/preso experience by addressing it separately. Thoughts?

@natekupp natekupp added this to the 0.10.0 (planned) milestone Aug 4, 2020
@mgasner mgasner added the area: docs Related to documentation in general label Aug 6, 2020
@natekupp natekupp added this to To do in Docs Infra Oct 6, 2020
@yuhan yuhan added the area: docs-infra Related to documentation site infra and tooling label Oct 28, 2020
@yuhan
Copy link
Contributor

yuhan commented Oct 28, 2020

#2451

@nancydyc nancydyc moved this from To do to Backlog in Docs Infra Nov 2, 2020
@yuhan yuhan moved this from Backlog to Infra Backlog in Docs Infra Nov 2, 2020
@sryza sryza removed this from the 0.10.0 milestone Apr 1, 2021
@yuhan yuhan added the integration: notebook Related to dagstermill label May 14, 2021
@yuhan
Copy link
Contributor

yuhan commented May 9, 2022

folding this into this an internal tracker.

@yuhan yuhan closed this as completed May 9, 2022
Docs Infra automation moved this from Site Infra to Done May 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: docs Related to documentation in general area: docs-infra Related to documentation site infra and tooling integration: notebook Related to dagstermill
Projects
Development

No branches or pull requests

6 participants