🎉 New Destination: Timeplus

[jovezhong]

(this is to replace #20391 Cannot share secrets to maintainer if the PR is sent from a github org not personal account)

What

A new destination connector for Timeplus (works for any cloud deployment or onprem deployment of Timeplus)

close #20195

How

This python based destination connector will send data to Timeplus. It will create streams in Timeplus if necessary, with the exactly same schema from the source (kind of normalization, but not based on DBT). For example, if there are 4 columns in the CSV, then there will be 4 columns in the Timeplus stream.

Recommended reading order

airbyte-integrations/connectors/destination-timeplus/destination_timeplus/destination.py

🚨 User Impact 🚨

no breaking change

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

• Community member? Grant edit access to maintainers (instructions)
• Secrets in the connector's spec are annotated with airbyte_secret
• Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• Code reviews completed
• Documentation updated
  • Connector's README.md
  • Connector's bootstrap.md. See description and examples
  • docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • docs/integrations/README.md
  • airbyte-integrations/builds.md
• PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

• Create a non-forked branch based on this PR and test the below items on it
• Build is successful
• If new credentials are required for use in CI, add them to GSM. Instructions.
• /test connector=connectors/<name> command is passing
• New Connector version released on Dockerhub by running the /publish command described here
• After the connector is published, connector added to connector index as described here
• Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Updating a connector

Community member or Airbyter

• Grant edit access to maintainers (instructions)
• Secrets in the connector's spec are annotated with airbyte_secret
• Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• Code reviews completed
• Documentation updated
  • Connector's README.md
  • Connector's bootstrap.md. See description and examples
  • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
• PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

• Create a non-forked branch based on this PR and test the below items on it
• Build is successful
• If new credentials are required for use in CI, add them to GSM. Instructions.
• /test connector=connectors/<name> command is passing
• New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

Connector Generator

• Issue acceptance criteria met
• PR name follows PR naming conventions
• If adding a new generator, add it to the list of scaffold modules being tested
• The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
• Documentation which references the generator is updated as needed

To test this connector, you can create a free account at https://us.timeplus.cloud, sign in with Google/MS SSO, then create a workspace, create an API key, then set them in the secret/UI to push data to it (tested with CSV and db)

Tests

Unit

pytest -s unit_tests
=================================================== test session starts ===================================================
platform darwin -- Python 3.9.16, pytest-7.2.0, pluggy-1.0.0 -- /Users/jove/Dev/jovezhong/airbyte/airbyte-integrations/connectors/destination-timeplus/.venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/jove/Dev/jovezhong/airbyte, configfile: pytest.ini
collected 1 item

unit_tests/unit_test.py::test_type_mapping PASSED

=================================================== 1 passed in 0.71s ===================================================

Integration

=================================================== test session starts ===================================================
platform darwin -- Python 3.9.16, pytest-7.2.0, pluggy-1.0.0 -- /Users/jove/Dev/jovezhong/airbyte/airbyte-integrations/connectors/destination-timeplus/.venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/jove/Dev/jovezhong/airbyte, configfile: pytest.ini
collected 2 items

integration_tests/integration_test.py::test_check_valid_config {"type": "LOG", "log": {"level": "INFO", "message": "Successfully connected to https://dev.timeplus.cloud/latest"}}
PASSED
integration_tests/integration_test.py::test_check_invalid_config PASSED

Acceptance

Put your acceptance tests output here.

[jovezhong]

Any maintainer look into this PR? It has been quite a while

[jovezhong]

still waiting for a reviewer

[jovezhong]

Again, waiting for reviewers. What is the process to assign such review tickets?

[CLAassistant]

All committers have signed the CLA.

[jovezhong]

Hi @marcosmarxm , thanks for the reviews. I am not quite sure about the process. I resolved the review comments. Feel free to reopen them if they need more changes. I added 3 commits to address your review comments one by one. Hopefully it should make it a bit easier to check the code change.

[jovezhong]

Hi @marcosmarxm, could you please take the other look? Thanks

[jovezhong]

I don't want to ask this every week, but please please give some feedback @marcosmarxm

[marcosmarxm]

Sorry the long delay @jovezhong I added this to our team backlog. Hope to have a slot this week.

[marcosmarxm]

@jovezhong can you fix:
❌ - check_documentation_file_exists
❌ - check_changelog_entry_is_updated
❌ - check_connector_icon_is_available

[jovezhong]

Sure, @marcosmarxm , happy to add those files. I checked the log and cannot find the exact file they are looking for? Can you share any example or docs about those 3 checks? Thanks.

❌ - check_documentation_file_exists
❌ - check_changelog_entry_is_updated
❌ - check_connector_icon_is_available

[jovezhong]

Never mind. I got the checker code tools/ci_connector_ops/ci_connector_ops/qa_checks.py
I will add commits soon

[marcosmarxm]

/test connector=connectors/destination-timeplus

⚠️ The test slash command is now deprecated.

The connector tests are automatically triggered as CI checks.

Please use /legacy-test if you need to test CDK or CAT changes.

Please join post to #pipeline-complaint-hotline slack channel if something is not working as expected.

[jovezhong]

Thank you @marcosmarxm for the review and merge!