Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source Woocommerce: fix pagination and removed lookback window #23599

Merged
merged 6 commits into from
Mar 3, 2023

Conversation

andrewlreeve
Copy link
Contributor

@andrewlreeve andrewlreeve commented Feb 28, 2023

What

  1. The current version (0.2.0) of the Woocommerce connector has broken pagination and it is not fetching paginated pages.

  2. A conversion window config has been included in the source that isn't required. This looks like a copy and paste hangover from a Google Ads connector.

How

  1. Connector was using page instead of offset for the page_token_option. Woocommerce has to use an offset approach as the page index is 1 based.

🚨 User Impact 🚨

Removes a setting config that is superfluous.

Pre-merge Checklist

Community member or Airbyter

  • Community member? Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
    • docs/integrations/README.md
    • airbyte-integrations/builds.md
  • PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • If new credentials are required for use in CI, add them to GSM. Instructions.
  • /test connector=connectors/<name> command is passing
  • New Connector version released on Dockerhub by running the /publish command described here
  • After the connector is published, connector added to connector index as described here
  • Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here
Updating a connector

Community member or Airbyter

  • Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • If new credentials are required for use in CI, add them to GSM. Instructions.
  • /test connector=connectors/<name> command is passing
  • New Connector version released on Dockerhub and connector version bumped by running the /publish command described here
Connector Generator
  • Issue acceptance criteria met
  • PR name follows PR naming conventions
  • If adding a new generator, add it to the list of scaffold modules being tested
  • The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
  • Documentation which references the generator is updated as needed

@CLAassistant
Copy link

CLAassistant commented Feb 28, 2023

CLA assistant check
All committers have signed the CLA.

@andrewlreeve andrewlreeve changed the title Fixed source-woocommerce pagination and removed lookback window Source Woocommerce: fix pagination and removed lookback window Feb 28, 2023
@marcosmarxm
Copy link
Member

marcosmarxm commented Mar 2, 2023

/test connector=connectors/source-woocommerce

🕑 connectors/source-woocommerce https://github.com/airbytehq/airbyte/actions/runs/4317218130
✅ connectors/source-woocommerce https://github.com/airbytehq/airbyte/actions/runs/4317218130
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:509: The previous and actual discovered catalogs are identical.
================== 36 passed, 1 skipped in 346.36s (0:05:46) ===================

Copy link
Member

@marcosmarxm marcosmarxm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @andrewlreeve I'll publish tomorrow

@octavia-squidington-iii octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Mar 3, 2023
@marcosmarxm
Copy link
Member

marcosmarxm commented Mar 3, 2023

/publish connector=connectors/source-woocommerce

🕑 Publishing the following connectors:
connectors/source-woocommerce
https://github.com/airbytehq/airbyte/actions/runs/4324525251


Connector Did it publish? Were definitions generated?
connectors/source-woocommerce

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@marcosmarxm marcosmarxm merged commit 9f7ee2c into airbytehq:master Mar 3, 2023
jbfbell pushed a commit that referenced this pull request Mar 6, 2023
* fix: pagination and lookback

* chore: fix newline

* bump connector version

* remove files and eof

* auto-bump connector version

---------

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
danielduckworth pushed a commit to danielduckworth/airbyte that referenced this pull request Mar 13, 2023
…tehq#23599)

* fix: pagination and lookback

* chore: fix newline

* bump connector version

* remove files and eof

* auto-bump connector version

---------

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
@andrewlreeve andrewlreeve deleted the source-woocommerce branch April 19, 2023 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation bounty community connectors/source/woocommerce
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants