Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🎉 Source Freshdesk: Added new streams #1

Closed
wants to merge 13 commits into from
Closed

🎉 Source Freshdesk: Added new streams #1

wants to merge 13 commits into from

Conversation

lgomezm
Copy link
Owner

@lgomezm lgomezm commented May 1, 2022 •
edited
Loading

What

  • There are several Freshdesk endpoints that don't have any matching stream yet.

Closes airbytehq#1387.

Note: This PR depends on airbytehq#12334. I will update the target branch accordingly when it's merged.

How

  • Adds new streams to the Freshdesk source connector.
  • All new streams work on a full refresh basis.

Recommended reading order

  1. streams.py
  2. test_streams.py
  3. Files in /schemas directory.

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

  • Community member? Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • docs/SUMMARY.md
    • docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
    • docs/integrations/README.md
    • airbyte-integrations/builds.md
  • PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • If new credentials are required for use in CI, add them to GSM. Instructions.
  • /test connector=connectors/<name> command is passing
  • New Connector version released on Dockerhub by running the /publish command described here
  • After the connector is published, connector added to connector index as described here
  • Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here
Updating a connector

Community member or Airbyter

  • Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • If new credentials are required for use in CI, add them to GSM. Instructions.
  • /test connector=connectors/<name> command is passing
  • New Connector version released on Dockerhub and connector version bumped by running the /publish command described here
Connector Generator
  • Issue acceptance criteria met
  • PR name follows PR naming conventions
  • If adding a new generator, add it to the list of scaffold modules being tested
  • The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
  • Documentation which references the generator is updated as needed

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

@lgomezm lgomezm force-pushed the lgomez/freshdesk_migration branch from 24d1581 to 5751b4a Compare May 18, 2022 02:31
@lgomezm lgomezm force-pushed the lgomez/freshdesk_migration branch from 77ea41b to 9cf2ba1 Compare May 27, 2022 02:59
lgomezm pushed a commit that referenced this pull request Jul 13, 2022
@harshithmullapudi @ptiurin
Harshith/test pr 12536 (airbytehq#13874)
074adef 
* 🎉 New Source: Firebolt (#1)

* fix: Boolean type cast

* test: Improve format testing and doc

* refactor: Move some db functionality

* docs: Adding types doc link in utils

* feat: Use future-proof Auth in SDK

* fix: integration tests are failing

* chore: update seed file

Co-authored-by: Petro Tiurin <93913847+ptiurin@users.noreply.github.com>
Co-authored-by: ptiurin <petro.tiurin@firebolt.io>
@lgomezm lgomezm closed this Jul 22, 2022
lgomezm pushed a commit that referenced this pull request Oct 10, 2022
@macmv @faunaee
🎉 New source: Fauna (airbytehq#15274)
65e6168 
* Add fauna source

* Update changelog to include the correct PR

* Improve docs (#1)

* Applied suggestions to improve docs (airbytehq#2)

* Applied suggestions to improve docs

* Cleaned up the docs

* Apply suggestions from code review

Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>

* Update airbyte-integrations/connectors/source-fauna/source_fauna/spec.yaml

Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>

Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>

* Flake Checker (airbytehq#3)

* Run ./gradlew :airbyte-integrations:connectors:source-fauna:flakeCheck

* Fix all the warnings

* Set additionalProperties to true to adhere to acceptance tests

* Remove custom fields (airbytehq#4)

* Remove custom fields from source.py

* Remove custom fields from spec.yaml

* Collections that support incremental sync are found correctly

* Run formatter

* Index values and termins are verified

* Stripped additional_columns from collection config and check()

* We now search for an index at the start of each sync

* Add default for missing data in collection

* Add a log message about the index chosen to sync an incremental stream

* Add an example for a configured incremental catalog

* Check test now validates the simplified check function

* Remove collection name from spec.yaml and CollectionConfig

* Update test_util.py to ahere to the new config

* Update the first discover test to validate that we can find indexes correctly

* Remove other discover tests, as they no longer apply

* Full refresh test now works with simplified expanded columns

* Remove unused imports

* Incremental test now adheres to the find_index_for_stream system

* Database test passes, so now all unit tests pass again

* Remove extra fields from required section

* ttl is nullable

* Data defaults to an empty object

* Update tests to reflect ttl and data select changes

* Fix expected records. All unit tests and acceptance tests pass

* Cleanup docs for find_index_for_stream

* Update setup guide to reflect multiple collections

* Add docs to install the fauna shell

* Update examples and README to conform to the removal of additional columns

Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>
lgomezm pushed a commit that referenced this pull request Oct 10, 2022
@marcosmarxm @macmv
@faunaee @octavia-squidington-iii
Source Fauna: add seed file information (airbytehq#17355)
8fc5dcb 
* Add fauna source

* Update changelog to include the correct PR

* Improve docs (#1)

* Applied suggestions to improve docs (airbytehq#2)

* Applied suggestions to improve docs

* Cleaned up the docs

* Apply suggestions from code review

Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>

* Update airbyte-integrations/connectors/source-fauna/source_fauna/spec.yaml

Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>

Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>

* Flake Checker (airbytehq#3)

* Run ./gradlew :airbyte-integrations:connectors:source-fauna:flakeCheck

* Fix all the warnings

* Set additionalProperties to true to adhere to acceptance tests

* Remove custom fields (airbytehq#4)

* Remove custom fields from source.py

* Remove custom fields from spec.yaml

* Collections that support incremental sync are found correctly

* Run formatter

* Index values and termins are verified

* Stripped additional_columns from collection config and check()

* We now search for an index at the start of each sync

* Add default for missing data in collection

* Add a log message about the index chosen to sync an incremental stream

* Add an example for a configured incremental catalog

* Check test now validates the simplified check function

* Remove collection name from spec.yaml and CollectionConfig

* Update test_util.py to ahere to the new config

* Update the first discover test to validate that we can find indexes correctly

* Remove other discover tests, as they no longer apply

* Full refresh test now works with simplified expanded columns

* Remove unused imports

* Incremental test now adheres to the find_index_for_stream system

* Database test passes, so now all unit tests pass again

* Remove extra fields from required section

* ttl is nullable

* Data defaults to an empty object

* Update tests to reflect ttl and data select changes

* Fix expected records. All unit tests and acceptance tests pass

* Cleanup docs for find_index_for_stream

* Update setup guide to reflect multiple collections

* Add docs to install the fauna shell

* Update examples and README to conform to the removal of additional columns

* add seed file

* auto-bump connector version [ci skip]

* add icon in seed

Co-authored-by: Neil Macneale <neil.macneale.v@gmail.com>
Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
area/connectors
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@lgomezm