New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🚨🚨 ✨ Source Google Search Console: adding data_state
parameter
#26452
Conversation
Before Merging a Connector Pull RequestWow! What a great pull request you have here! 🎉 To merge this PR, ensure the following has been done/considered for each connector added or updated:
If the checklist is complete, but the CI check is failing,
|
data_state
parameter data_state
parameter
c322861
to
da5e08f
Compare
data_state
parameter data_state
parameter
/test connector=connectors/source-google-search-console
Build PassedTest summary info:
|
@lazebnyi would it be too much trouble to take a quick look at this PR and give your approval if it's okay to merge? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
@@ -98,6 +101,10 @@ The Google Search Console Source connector supports the following [ sync modes]( | |||
The granularity for the cursor is 1 day, so Incremental Sync in Append mode may result in duplicating the data. | |||
::: | |||
|
|||
:::note | |||
Parameter `data_state='all'` should not be used with Incremental Sync mode as it may cause data loss. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe it would be beneficial to include this information in the field description within the spec file.
…bytehq#26452) * feat: adding parameter * bump dockerfile and metadata.yaml * docs: update google-search-console readme with new changelog * @lazebnyi code review suggestion applied --------- Co-authored-by: sajarin <sajarindider@gmail.com>
What
Adding
data_state
, which could make data fresher from the google search console APIIssue: #10402
How
Adding a parameter that is set for a whole source.
Recommended reading order
airbyte-integrations/connectors/source-google-search-console/source_google_search_console/spec.jsona
airbyte-integrations/connectors/source-google-search-console/source_google_search_console/source.py
airbyte-integrations/connectors/source-google-search-console/source_google_search_console/streams.py
airbyte-integrations/connectors/source-google-search-console/unit_tests/unit_test.py
🚨 User Impact 🚨
There should be no user impact on that change as the default value is set of
data_state
tofinal
.This should be a minor change.
Pre-merge Actions
Community member or Airbyter
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.