-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add documentation around LoadOptions with example DAGs #1567
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Codecov ReportBase: 97.71% // Head: 94.24% // Decreases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## main #1567 +/- ##
==========================================
- Coverage 97.71% 94.24% -3.47%
==========================================
Files 22 89 +67
Lines 787 4448 +3661
Branches 0 444 +444
==========================================
+ Hits 769 4192 +3423
- Misses 18 173 +155
- Partials 0 83 +83
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
@sunank200 Just a minor comment, we need to add these options in specific sections
It may be tough to list relevant options as per specific sections, maybe adding a generic section and referencing it makes more sense. WDYT? |
@utkarsharma2 referenced different locations as mentioned. da148c4 Also addressed comments from this PR: #1566 |
# Description ## What is the current behavior? <!-- Please describe the current behavior that you are modifying. --> In Astro SDK 1.4 we introduce a series of LoadOptions classes for classes such as databricks, pandas, and snowflake. Each of these classes is unique and we should create documentation that best showcases how users can use these classes to create the loading settings they want. <!-- Issues are required for both bug fixes and features. Reference it using one of the following: closes: #ISSUE related: #ISSUE --> closes: #1556 ## What is the new behavior? <!-- Please describe the behavior or changes that are being added by this PR. --> - Add example DAG using `PandasLoadOptions`, `SnowflakeLoadOptions` and `DeltaLoadOptions` - Add the documentation for `PandasLoadOptions`, `SnowflakeLoadOptions` and `DeltaLoadOptions`. Add detailed documentation of how a user can use these classes. - Add examples for PandasOptions, DeltaOptions, and Snowflake Options. ## Does this introduce a breaking change? No ### Checklist - [x] Created tests which fail without the change (if possible) - [x] Extended the README/documentation, if necessary
Description
What is the current behavior?
In Astro SDK 1.4 we introduce a series of LoadOptions classes for classes such as databricks, pandas, and snowflake. Each of these classes is unique and we should create documentation that best showcases how users can use these classes to create the loading settings they want.
closes: #1556
What is the new behavior?
PandasLoadOptions
,SnowflakeLoadOptions
andDeltaLoadOptions
PandasLoadOptions
,SnowflakeLoadOptions
andDeltaLoadOptions
. Add detailed documentation of how a user can use these classes.Does this introduce a breaking change?
No
Checklist