Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standard Tests: allow specifying exact record matches #2186

Closed
sherifnada opened this issue Feb 23, 2021 · 2 comments · Fixed by #2960
Closed

Standard Tests: allow specifying exact record matches #2186

sherifnada opened this issue Feb 23, 2021 · 2 comments · Fixed by #2960
Assignees
Labels
Acceptance Tests https://docs.google.com/document/d/1pFLR8eLP5VOj6vDtWe69KZL0aWWHEA6Vkbod6NYJon4/edit# type/enhancement New feature or request

Comments

@sherifnada
Copy link
Contributor

Tell us about the problem you're trying to solve

standard tests are currently smoke tests which don't allow verifying that exact records were produced. We want to have this functionality so we can write more robust tests.

@sherifnada sherifnada added type/enhancement New feature or request Acceptance Tests https://docs.google.com/document/d/1pFLR8eLP5VOj6vDtWe69KZL0aWWHEA6Vkbod6NYJon4/edit# labels Feb 23, 2021
@keu
Copy link
Contributor

keu commented Mar 23, 2021

for this we should provide files with expected records,
test options are:

Options Default Value Meaning Estimation
exact_order False Ensure that records produced in exact same order 2h
extra_fields True Allow records to have other fields 1h
extra_records True Allow connector to produce extra records, but still enforce all records from the expected file to be produced 1h

by default test configured to be as much flexible as possible.

base implementation - 2h
Total dev estimation - 6+h

@keu keu self-assigned this Apr 19, 2021
@keu keu changed the title Allow specifying exact record matches in standard tests v2 Standard Tests: allow specifying exact record matches Apr 19, 2021
@keu keu linked a pull request Apr 20, 2021 that will close this issue
@keu
Copy link
Contributor

keu commented Apr 20, 2021

some initial match was implemented in the first version of Standard Tests. All options were added by they are NOP

@keu keu added the zazmic label Apr 29, 2021
@keu keu closed this as completed in #2960 Apr 30, 2021
keu added a commit that referenced this issue Apr 30, 2021
* records matching

+ extra_fields
+ exact_order
+ extra_records

* fix serialization before comparision

* fix pydantic validation

* fix reporting and enable for hubspot

* format

* update docs

* update examples

* fix template

* Update docs/contributing-to-airbyte/building-new-connector/source-acceptance-tests.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update docs/contributing-to-airbyte/building-new-connector/source-acceptance-tests.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Acceptance Tests https://docs.google.com/document/d/1pFLR8eLP5VOj6vDtWe69KZL0aWWHEA6Vkbod6NYJon4/edit# type/enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants