Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 New Connector: Fleetio source #35633

Merged
merged 60 commits into from
Jun 3, 2024

Conversation

jmmizerany
Copy link
Contributor

What

  • Adding in a new source connector to pull data from the Fleetio API
  • For context on what fleetio is, the website is linked here
  • The only UI change is adding the Fleetio Source connector , along with our Fleetio icon, to the source collection for connectors

How

  • This connector was built using the airbyte low-code CDK
  • Currently this support full table replication only, but in future version we will introduce incremental syncing

Recommended reading order

  1. x.java
  2. y.python

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user?

  • No breaking changes to Airbyte or this connector

For connector PRs, use this section to explain which type of semantic versioning bump occurs as a result of the changes. Refer to our Semantic Versioning for Connectors guidelines for more information. Breaking changes to connectors must be documented by an Airbyte engineer (PR author, or reviewer for community PRs) by using the Breaking Change Release Playbook.

If there are breaking changes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Actions

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

  • Community member? Grant edit access to maintainers (instructions)
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Connector version is set to 0.0.1
    • Dockerfile has version 0.0.1
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • docs/integrations/<source or destination>/<name>.md including changelog with an entry for the initial version. See changelog example
    • docs/integrations/README.md

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • If new credentials are required for use in CI, add them to GSM. Instructions.

@jmmizerany
Copy link
Contributor Author

Hey all, is there an update on the changes I have pushed? We are wanting to get this live as we have a beta test with some of our end users coming up

Copy link
Contributor

@natikgadzhi natikgadzhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me, high level. @marcosmarxm, can you get this over the finish line? What are our requirements for new connectors? Should we obtain a sandbox account and store it internally for tests?

@jmmizerany, would you be able to provide us with a sandbox account that Airbyte could use to run acceptance tests for this connector? I'm not sure how we usually share those credentials ;)

@jmmizerany
Copy link
Contributor Author

jmmizerany commented Mar 5, 2024

This looks good to me, high level. @marcosmarxm, can you get this over the finish line? What are our requirements for new connectors? Should we obtain a sandbox account and store it internally for tests?

@jmmizerany, would you be able to provide us with a sandbox account that Airbyte could use to run acceptance tests for this connector? I'm not sure how we usually share those credentials ;)

@natikgadzhi Thanks! We have a sandbox account ready to go. Let me know how we can coordinate the next steps

@marcosmarxm
Copy link
Member

I ran tests with provided sandbox credentials and some tests are failing. Later this week I'll return with a review and update of changes.

@jmmizerany
Copy link
Contributor Author

I ran tests with provided sandbox credentials and some tests are failing. Later this week I'll return with a review and update of changes.

Thank you @marcosmarxm Please let me know what I can do to help

Copy link
Member

@marcosmarxm marcosmarxm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the spec.yaml file as you're setting the spec inside the manifest.yaml

@@ -0,0 +1,240 @@
{
"$schema": "http://json-schema.org/schema#",
"properties": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all objects type must have additionalProperties: true

Copy link
Contributor

@natikgadzhi natikgadzhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's get this over the finish line, but then — we should also inline all schemas from json files into the manifest. We have a new tool that can assist with that, airbyte-ci connectors --name source-fleetio migrate_to_inline_schemas in #37649.

@jmmizerany
Copy link
Contributor Author

Let's get this over the finish line, but then — we should also inline all schemas from json files into the manifest. We have a new tool that can assist with that, airbyte-ci connectors --name source-fleetio migrate_to_inline_schemas in #37649.

Thanks!

Copy link
Contributor

@natikgadzhi natikgadzhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for a low-code connector with a bunch of integration tests, this is good enough. For failing cats, this might stop us from doing new changes to this connector (which I would advice — like inlining schemas).

We will reach out if we need to fix them up.

@natikgadzhi natikgadzhi merged commit f4c69be into airbytehq:master Jun 3, 2024
31 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation community connectors/source/fleetio new-connector
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants