Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🐛 Fix date-time for airbyte types #25965

Merged
merged 12 commits into from May 12, 2023
Merged

🐛 Fix date-time for airbyte types #25965

merged 12 commits into from May 12, 2023

Conversation

evantahler
Copy link
Contributor

@evantahler evantahler commented May 10, 2023 •
edited

Closes #25564

This PR fixes sources which were using the wrong data type in their specs and tests. It's date-time, not datetime!

Most changes were about test fixtures, but the following 4 connectors need to be re-published:

  • source-file
  • source-firebolt
  • source-google-ads
  • source-stripe

@evantahler evantahler changed the title Fix airbyte types Fix date-time airbyte types May 10, 2023
@evantahler evantahler mentioned this pull request May 10, 2023
Closed
@github-actions
Copy link
Contributor

github-actions bot commented May 10, 2023 •
edited by evantahler

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

  • PR name follows PR naming conventions
  • Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan and you've followed all steps in the Breaking Changes Checklist
  • Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
  • Secrets in the connector's spec are annotated with airbyte_secret
  • All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
  • Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
  • You, or an Airbyter, have run /test successfully on this PR - or on a non-forked branch
  • You, or an Airbyter, have run /publish successfully on this PR - or on a non-forked branch
  • You've updated the connector's metadata.yaml file (new!)
  • The Octavia bot updated the source_definitions.yaml or destination_definitions.yaml, or you ran processResources manually (deprecated)

If the checklist is complete, but the CI check is failing,

  1. Check for hidden checklists in your PR description

  2. Toggle the github label checklist-action-run on/off to re-run the checklist CI.

@evantahler
Copy link
Contributor Author

evantahler commented May 11, 2023 •
edited by github-actions bot

/test connector=connectors/source-stripe

🕑 connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/4942732701
✅ connectors/source-stripe https://github.com/airbytehq/airbyte/actions/runs/4942732701
Python tests coverage:

Name                                     Stmts   Miss  Cover
------------------------------------------------------------
source_stripe/source.py                     22      0   100%
source_stripe/__init__.py                    2      0   100%
source_stripe/streams.py                   332     34    90%
source_stripe/availability_strategy.py      19      2    89%
------------------------------------------------------------
TOTAL                                      375     36    90%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================== 39 passed, 1 skipped in 288.25s (0:04:48) ===================

@evantahler
Copy link
Contributor Author

evantahler commented May 11, 2023 •
edited by github-actions bot

/test connector=connectors/source-firebolt

🕑 connectors/source-firebolt https://github.com/airbytehq/airbyte/actions/runs/4942733796
✅ connectors/source-firebolt https://github.com/airbytehq/airbyte/actions/runs/4942733796
Python tests coverage:

Name                          Stmts   Miss  Cover
-------------------------------------------------
source_firebolt/utils.py         34      0   100%
source_firebolt/__init__.py       2      0   100%
source_firebolt/source.py        44      1    98%
source_firebolt/database.py      36      4    89%
-------------------------------------------------
TOTAL                           116      5    96%
Name                          Stmts   Miss  Cover
-------------------------------------------------
source_firebolt/source.py        44      0   100%
source_firebolt/__init__.py       2      0   100%
source_firebolt/utils.py         34      1    97%
source_firebolt/database.py      36      5    86%
-------------------------------------------------
TOTAL                           116      6    95%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 36 passed, 2 skipped, 38 warnings in 37.84s ==================

@evantahler evantahler changed the title Fix date-time airbyte types 🐞 Fix date-time airbyte types May 11, 2023
@evantahler
Copy link
Contributor Author

evantahler commented May 11, 2023 •
edited by github-actions bot

/test connector=connectors/source-file

🕑 connectors/source-file https://github.com/airbytehq/airbyte/actions/runs/4950120559
✅ connectors/source-file https://github.com/airbytehq/airbyte/actions/runs/4950120559
Python tests coverage:

Name                      Stmts   Miss  Cover
---------------------------------------------
source_file/__init__.py       2      0   100%
source_file/utils.py         13      1    92%
source_file/source.py        81      7    91%
source_file/client.py       322     56    83%
---------------------------------------------
TOTAL                       418     64    85%
Name                      Stmts   Miss  Cover
---------------------------------------------
source_file/__init__.py       2      0   100%
source_file/client.py       322     51    84%
source_file/utils.py         13      8    38%
source_file/source.py        81     60    26%
---------------------------------------------
TOTAL                       418    119    72%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: Incremental syncs are not supported on this connector.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:578: The previous and actual discovered catalogs are identical.
======================== 35 passed, 3 skipped in 49.47s ========================

@evantahler
Copy link
Contributor Author

evantahler commented May 11, 2023 •
edited by github-actions bot

/test connector=connectors/source-google-ads

🕑 connectors/source-google-ads https://github.com/airbytehq/airbyte/actions/runs/4951008799
✅ connectors/source-google-ads https://github.com/airbytehq/airbyte/actions/runs/4951008799
Python tests coverage:

Name                                       Stmts   Miss  Cover
--------------------------------------------------------------
source_google_ads/models.py                   18      0   100%
source_google_ads/__init__.py                  2      0   100%
source_google_ads/utils.py                    56      1    98%
source_google_ads/streams.py                 195      6    97%
source_google_ads/source.py                   95      7    93%
source_google_ads/custom_query_stream.py      56      5    91%
source_google_ads/google_ads.py               73     12    84%
--------------------------------------------------------------
TOTAL                                        495     31    94%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================== 40 passed, 1 skipped in 619.55s (0:10:19) ===================

@evantahler evantahler marked this pull request as ready for review May 11, 2023 18:32
@evantahler evantahler requested review from bazarnov, marcosmarxm, erohmensing and edgao and removed request for bazarnov May 11, 2023 18:32
@bazarnov bazarnov changed the title 🐞 Fix date-time airbyte types 🐛 Fix date-time for airbyte types May 11, 2023
bazarnov
bazarnov approved these changes May 11, 2023
View reviewed changes
Copy link
Collaborator

@bazarnov bazarnov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

airbyte-integrations/connectors/source-google-ads/Dockerfile Show resolved Hide resolved
@evantahler
Copy link
Contributor Author

evantahler commented May 12, 2023 •
edited by github-actions bot

/publish connector=connectors/source-stripe

🕑 Publishing the following connectors:
connectors/source-stripe
https://github.com/airbytehq/airbyte/actions/runs/4953828554


Connector Version Did it publish? Were definitions generated?
connectors/source-stripe 3.4.3 ✅ ✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@evantahler
Copy link
Contributor Author

evantahler commented May 12, 2023 •
edited by github-actions bot

/publish connector=connectors/source-file

🕑 Publishing the following connectors:
connectors/source-file
https://github.com/airbytehq/airbyte/actions/runs/4953831491


Connector Version Did it publish? Were definitions generated?
connectors/source-file 0.3.4 ✅ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@evantahler
Copy link
Contributor Author

evantahler commented May 12, 2023 •
edited by github-actions bot

/publish connector=connectors/source-google-ads

🕑 Publishing the following connectors:
connectors/source-google-ads
https://github.com/airbytehq/airbyte/actions/runs/4953832665


Connector Version Did it publish? Were definitions generated?
connectors/source-google-ads 0.2.16 ✅ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@evantahler
Copy link
Contributor Author

evantahler commented May 12, 2023 •
edited by github-actions bot

/publish connector=connectors/source-firebolt

🕑 Publishing the following connectors:
connectors/source-firebolt
https://github.com/airbytehq/airbyte/actions/runs/4953872739


Connector Version Did it publish? Were definitions generated?
connectors/source-firebolt 0.2.1 ✅ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@evantahler evantahler enabled auto-merge (squash) May 12, 2023 00:51
@evantahler evantahler merged commit 3679dcd into master May 12, 2023
24 of 27 checks passed
@evantahler evantahler deleted the evan/fix-datetime branch May 12, 2023 01:15
marcosmarxm pushed a commit to natalia-miinto/airbyte that referenced this pull request Jun 8, 2023
@evantahler @octavia-squidington-iii @marcosmarxm
🐛 Fix date-time for airbyte types (airbytehq#25965)
ae6925a 
* Fix airbyte types

* update firebolt test fixture

* Bump changelog PR ids

* update metadata files

* fix firebolt test

* fix file-source for date-time overload

* fix source-file changelog

* it's a date, not date-time

* another date

* auto-bump connector version

* manually bump definitions

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bazarnov bazarnov bazarnov approved these changes

@marcosmarxm marcosmarxm Awaiting requested review from marcosmarxm

@erohmensing erohmensing Awaiting requested review from erohmensing

@edgao edgao Awaiting requested review from edgao

Assignees
No one assigned
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/destination/convex connectors/destination/cumulio connectors/destination/duckdb connectors/destination/firebolt connectors/destination/sqlite connectors/source/file connectors/source/firebolt connectors/source/google-ads connectors/source/hellobaton connectors/source/stripe
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Destinations: Fix integration tests referring to format: datetime
3 participants
@evantahler @bazarnov @octavia-squidington-iii