Remove custom fields #4
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
macmv
force-pushed
the
remove-custom-fields
branch
from
e319eae
to
557b60a
Compare
cleve-fauna
reviewed
Sep 2, 2022
airbyte-integrations/connectors/source-fauna/source_fauna/source.py
Outdated
Show resolved
Hide resolved
cleve-fauna
reviewed
Sep 2, 2022
cleve-fauna
approved these changes
Sep 2, 2022
macmv
pushed a commit
that referenced
this pull request
Sep 23, 2022
* Replaced sidebar menu with headlessui * Removed unnecessary tag li * Removed unnecessary Sidebar Popout * Changed export type * Added keyboard support * Removed styled-components * Removed commented code * Disabled ESLint rule css-modules/no-undef-class * Review fixes * Fixed shorthand property * Review fixes * Review fixes #2 * Review fixes #3 * Review fixes #4 * Review fixes airbytehq#5 * Review fixes airbytehq#6 * Review fixes airbytehq#7 * Update airbyte-webapp/src/packages/cloud/views/layout/SideBar/SideBar.module.scss Co-authored-by: Vladimir <volodymyr.s.petrov@globallogic.com> * Update airbyte-webapp/src/packages/cloud/views/layout/SideBar/SideBar.module.scss Co-authored-by: Vladimir <volodymyr.s.petrov@globallogic.com> Co-authored-by: Vladimir <volodymyr.s.petrov@globallogic.com>
macmv
added a commit
that referenced
this pull request
Sep 23, 2022
* Remove custom fields from source.py * Remove custom fields from spec.yaml * Collections that support incremental sync are found correctly * Run formatter * Index values and termins are verified * Stripped additional_columns from collection config and check() * We now search for an index at the start of each sync * Add default for missing data in collection * Add a log message about the index chosen to sync an incremental stream * Add an example for a configured incremental catalog * Check test now validates the simplified check function * Remove collection name from spec.yaml and CollectionConfig * Update test_util.py to ahere to the new config * Update the first discover test to validate that we can find indexes correctly * Remove other discover tests, as they no longer apply * Full refresh test now works with simplified expanded columns * Remove unused imports * Incremental test now adheres to the find_index_for_stream system * Database test passes, so now all unit tests pass again * Remove extra fields from required section * ttl is nullable * Data defaults to an empty object * Update tests to reflect ttl and data select changes * Fix expected records. All unit tests and acceptance tests pass * Cleanup docs for find_index_for_stream * Update setup guide to reflect multiple collections
macmv
added a commit
that referenced
this pull request
Oct 26, 2022
* Add fauna source * Update changelog to include the correct PR * Improve docs (#1) * Applied suggestions to improve docs (#2) * Applied suggestions to improve docs * Cleaned up the docs * Apply suggestions from code review Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com> * Update airbyte-integrations/connectors/source-fauna/source_fauna/spec.yaml Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com> Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com> * Flake Checker (#3) * Run ./gradlew :airbyte-integrations:connectors:source-fauna:flakeCheck * Fix all the warnings * Set additionalProperties to true to adhere to acceptance tests * Remove custom fields (#4) * Remove custom fields from source.py * Remove custom fields from spec.yaml * Collections that support incremental sync are found correctly * Run formatter * Index values and termins are verified * Stripped additional_columns from collection config and check() * We now search for an index at the start of each sync * Add default for missing data in collection * Add a log message about the index chosen to sync an incremental stream * Add an example for a configured incremental catalog * Check test now validates the simplified check function * Remove collection name from spec.yaml and CollectionConfig * Update test_util.py to ahere to the new config * Update the first discover test to validate that we can find indexes correctly * Remove other discover tests, as they no longer apply * Full refresh test now works with simplified expanded columns * Remove unused imports * Incremental test now adheres to the find_index_for_stream system * Database test passes, so now all unit tests pass again * Remove extra fields from required section * ttl is nullable * Data defaults to an empty object * Update tests to reflect ttl and data select changes * Fix expected records. All unit tests and acceptance tests pass * Cleanup docs for find_index_for_stream * Update setup guide to reflect multiple collections * Add docs to install the fauna shell * Update examples and README to conform to the removal of additional columns Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>
macmv
added a commit
that referenced
this pull request
Oct 26, 2022
* Add fauna source * Update changelog to include the correct PR * Improve docs (#1) * Applied suggestions to improve docs (#2) * Applied suggestions to improve docs * Cleaned up the docs * Apply suggestions from code review Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com> * Update airbyte-integrations/connectors/source-fauna/source_fauna/spec.yaml Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com> Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com> * Flake Checker (#3) * Run ./gradlew :airbyte-integrations:connectors:source-fauna:flakeCheck * Fix all the warnings * Set additionalProperties to true to adhere to acceptance tests * Remove custom fields (#4) * Remove custom fields from source.py * Remove custom fields from spec.yaml * Collections that support incremental sync are found correctly * Run formatter * Index values and termins are verified * Stripped additional_columns from collection config and check() * We now search for an index at the start of each sync * Add default for missing data in collection * Add a log message about the index chosen to sync an incremental stream * Add an example for a configured incremental catalog * Check test now validates the simplified check function * Remove collection name from spec.yaml and CollectionConfig * Update test_util.py to ahere to the new config * Update the first discover test to validate that we can find indexes correctly * Remove other discover tests, as they no longer apply * Full refresh test now works with simplified expanded columns * Remove unused imports * Incremental test now adheres to the find_index_for_stream system * Database test passes, so now all unit tests pass again * Remove extra fields from required section * ttl is nullable * Data defaults to an empty object * Update tests to reflect ttl and data select changes * Fix expected records. All unit tests and acceptance tests pass * Cleanup docs for find_index_for_stream * Update setup guide to reflect multiple collections * Add docs to install the fauna shell * Update examples and README to conform to the removal of additional columns * add seed file * auto-bump connector version [ci skip] * add icon in seed Co-authored-by: Neil Macneale <neil.macneale.v@gmail.com> Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Removes additional columns.
After the user selects the collections they want, we can't actually see anything more than simply the name of the stream they selected, and the sync mode they chose. This means we need to go find the index they wanted to use.
I've implemented the index finding by just iterating over all indexes at the start of each sync.