Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[dagster-managed-elements] Getting started guide #10224

Merged
merged 16 commits into from
Dec 6, 2022
Merged

[dagster-managed-elements] Getting started guide #10224

merged 16 commits into from
Dec 6, 2022

Conversation

benpankow
Copy link
Member

@benpankow benpankow commented Oct 27, 2022 •
edited
Loading

Summary

Implements a first pass at a managed elements/config guide for dagster-airbyte, walking through implementing a basic Airbyte connection.

Test Plan

vercel

@vercel
Copy link

vercel bot commented Oct 27, 2022 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated
dagster ✅ Ready (Inspect) Visit Preview ✅ 4 resolved Dec 6, 2022 at 7:56PM (UTC)
2 Ignored Deployments
Name Status Preview Comments Updated
dagit-storybook ⬜️ Ignored (Inspect) 💬 Add your feedback Dec 6, 2022 at 7:56PM (UTC)
dagster-oss-cloud-consolidated ⬜️ Ignored (Inspect) 💬 Add your feedback Dec 6, 2022 at 7:56PM (UTC)

@benpankow
Copy link
Member Author

benpankow commented Oct 27, 2022 •
edited
Loading

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@benpankow benpankow requested review from sryza, OwenKephart and yuhan and removed request for sryza October 27, 2022 15:23
@sryza
Copy link
Contributor

sryza commented Oct 27, 2022

Thoughts on making this part of the Airbyte integration docs or including Airbyte in the title? I think it comes off a little weird to present it as general functionality when it only works with a single integration.

@erinkcochran87
Copy link
Contributor

erinkcochran87 commented Nov 1, 2022 •
edited
Loading

Thoughts on making this part of the Airbyte integration docs or including Airbyte in the title? I think it comes off a little weird to present it as general functionality when it only works with a single integration.

A few thoughts:

  • If managed elements will be made available to other integrations, I think it's okay to keep the title and location as-is for now.

  • We should link to this from somewhere in the Airbyte docs

  • If/when this feature is expanded to include other integrations, we'll need to think about how to structure integration-specific sections. Right now there's only Airbyte, but this doc could quickly get unruly.
    We could eventually take this Airbyte-specific section, make it its own guide under the rest of the Airbyte docs ("Using Managed Elements with Airbyte" or something similar), and add a section to this guide that links out to the integration-specific content. That way the content will still be easily accessible from this location.

    Or we could do this now and save ourselves future work, unless the plan is to just keep this to Airbyte.

@slopp slopp mentioned this pull request Nov 1, 2022
Open
@benpankow benpankow force-pushed the benpankow/managed-elements-guide branch from 1cac966 to 1057808 Compare November 16, 2022 16:56
@benpankow benpankow force-pushed the benpankow/managed-elements-guide branch from a51dd75 to 92ed2ab Compare November 17, 2022 21:40
@benpankow benpankow changed the base branch from master to benpankow/managed-elements-apidoc November 17, 2022 21:46
@benpankow benpankow force-pushed the benpankow/managed-elements-guide branch from 92ed2ab to a9ed4d5 Compare November 17, 2022 21:46
@benpankow benpankow mentioned this pull request Nov 17, 2022
Merged
@benpankow benpankow force-pushed the benpankow/managed-elements-guide branch from a9ed4d5 to f08af60 Compare November 17, 2022 22:15
@benpankow benpankow force-pushed the benpankow/managed-elements-guide branch from f08af60 to 4990bc8 Compare November 17, 2022 22:35
@benpankow benpankow force-pushed the benpankow/managed-elements-guide branch from 4990bc8 to 05fc509 Compare November 17, 2022 22:45
erinkcochran87
erinkcochran87 reviewed Nov 21, 2022
View reviewed changes
docs/content/guides/dagster/airbyte-managed-elements.mdx Outdated Show resolved Hide resolved
docs/content/guides/dagster/airbyte-managed-elements.mdx Outdated Show resolved Hide resolved
docs/content/guides/dagster/airbyte-managed-elements.mdx Outdated Show resolved Hide resolved
docs/content/guides/dagster/airbyte-managed-elements.mdx Outdated Show resolved Hide resolved
docs/content/guides/dagster/airbyte-managed-elements.mdx Outdated Show resolved Hide resolved
docs/content/guides/dagster/airbyte-managed-elements.mdx Outdated Show resolved Hide resolved
docs/content/guides/dagster/airbyte-managed-elements.mdx Outdated Show resolved Hide resolved
docs/content/guides/dagster/airbyte-managed-elements.mdx Outdated Show resolved Hide resolved
docs/content/guides/dagster/airbyte-managed-elements.mdx Outdated Show resolved Hide resolved
docs/content/guides/dagster/airbyte-managed-elements.mdx Outdated Show resolved Hide resolved
yuhan
yuhan previously requested changes Nov 30, 2022
View reviewed changes
Copy link
Contributor

@yuhan yuhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q management

since the DD agenda announcement is out, should we officially rename Managed Elements to Data Ingestion as Code? also, thoughts on making the guide a subpage under airbyte? so the left nav would be something like:
image

@benpankow benpankow force-pushed the benpankow/managed-elements-apidoc branch from 35c40f8 to d841460 Compare December 5, 2022 23:19
@benpankow benpankow force-pushed the benpankow/managed-elements-apidoc branch from 7e78f42 to 045f9b4 Compare December 6, 2022 17:51
@benpankow benpankow force-pushed the benpankow/managed-elements-guide branch from 4db9d3f to 1012a4b Compare December 6, 2022 17:51
@benpankow benpankow force-pushed the benpankow/managed-elements-apidoc branch from 045f9b4 to eb42a6b Compare December 6, 2022 19:21
@benpankow benpankow force-pushed the benpankow/managed-elements-guide branch from 1012a4b to b4debd6 Compare December 6, 2022 19:21
Base automatically changed from benpankow/managed-elements-apidoc to master December 6, 2022 19:39
@benpankow benpankow force-pushed the benpankow/managed-elements-guide branch from b4debd6 to 9860819 Compare December 6, 2022 19:46
@benpankow benpankow merged commit 96560ad into master Dec 6, 2022
@benpankow benpankow deleted the benpankow/managed-elements-guide branch December 6, 2022 19:59
yuhan pushed a commit that referenced this pull request Dec 6, 2022
@benpankow @erinkcochran87 @yuhan
[dagster-managed-elements] Getting started guide (#10224)
a313b8f 
* [dagster-managed-elements] Getting started guide

* Start converting to airbyte

* apidoc

* link out to apidoc

* First pass

* Title + add Experimental callout

Using the Experimental badge in headings causes link anchor issues. Moving it to a callout instead.

* Prereqs

* Adjust headings and dividers

* Add Related section

* Copy edits for grammar, spelling, clarity

* Add to Integrations in sidenav

* Fix links in Related

* Reformat Related + add link to new Airbyte guide

* add link in apidoc

* docs rebuild

* fix guide

Co-authored-by: Erin Cochran <erin.k.cochran@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@erinkcochran87 erinkcochran87 erinkcochran87 approved these changes

@yuhan yuhan yuhan left review comments

@rexledesma rexledesma Awaiting requested review from rexledesma

@OwenKephart OwenKephart Awaiting requested review from OwenKephart

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@benpankow @sryza @erinkcochran87 @yuhan