Skip to content

Commit

Permalink
馃悰 SAT: fix minor issues with connector tests (#20237)
Browse files Browse the repository at this point in the history
  • Loading branch information
bazarnov committed Dec 15, 2022
1 parent 3f0cffb commit f1546c2
Show file tree
Hide file tree
Showing 6 changed files with 50 additions and 28 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -259,15 +259,15 @@
documentationUrl: https://docs.airbyte.com/integrations/sources/close-com
icon: close.svg
sourceType: api
releaseStage: alpha
releaseStage: beta
- name: CoinGecko Coins
sourceDefinitionId: 9cdd4183-d0ba-40c3-aad3-6f46d4103974
dockerRepository: airbyte/source-coingecko-coins
dockerImageTag: 0.1.0
documentationUrl: https://docs.airbyte.com/integrations/sources/coingecko-coins
icon: coingeckocoins.svg
sourceType: api
releaseStage: beta
releaseStage: alpha
- name: Cockroachdb
sourceDefinitionId: 9fa5862c-da7c-11eb-8d19-0242ac130003
dockerRepository: airbyte/source-cockroachdb
Expand Down
Original file line number Diff line number Diff line change
@@ -1,20 +1,28 @@
# See [Source Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/source-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-coingecko-coins:dev
tests:
acceptance_tests:
spec:
- spec_path: "source_coingecko_coins/spec.yaml"
tests:
- spec_path: "source_coingecko_coins/spec.yaml"
connection:
- config_path: "secrets/config.json"
status: "succeed"
- config_path: "integration_tests/invalid_config.json"
status: "failed"
tests:
- config_path: "secrets/config.json"
status: "succeed"
timeout_seconds: 120
- config_path: "integration_tests/invalid_config.json"
status: "failed"
discovery:
- config_path: "secrets/config.json"
tests:
- config_path: "secrets/config.json"
basic_read:
- config_path: "secrets/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
empty_streams: []
tests:
- config_path: "secrets/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
full_refresh:
- config_path: "secrets/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
tests:
- config_path: "secrets/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
# we ignore actively changing array with numbers to pass the `test two sequential reads`
ignored_fields:
market_chart: ["prices", "market_caps", "total_volumes"]
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from setuptools import find_packages, setup

MAIN_REQUIREMENTS = [
"airbyte-cdk~=0.1",
"airbyte-cdk~=0.2",
]

TEST_REQUIREMENTS = [
Expand Down
10 changes: 10 additions & 0 deletions airbyte-integrations/connectors/source-file/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,16 @@ docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/sample_files:/sample_files
1. To run additional integration tests, place your integration tests in a new directory `integration_tests` and run them with `python -m pytest -s integration_tests`.
Make sure to familiarize yourself with [pytest test discovery](https://docs.pytest.org/en/latest/goodpractices.html#test-discovery) to know how your test files and methods should be named.

#### Acceptance Tests
Customize `acceptance-test-config.yml` file to configure tests. See [Source Acceptance Tests](https://docs.airbyte.io/connector-development/testing-connectors/source-acceptance-tests-reference) for more information.
If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
To run your integration tests with acceptance tests, from the connector root, run
```
docker build . --no-cache -t airbyte/source-file:dev \
&& python -m pytest -p source_acceptance_test.plugin
```
To run your integration tests with docker

## Dependency Management
All of your dependencies should go in `setup.py`, NOT `requirements.txt`. The requirements file is only used to connect internal Airbyte dependencies in the monorepo for local development.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,19 +1,24 @@
# See [Source Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/source-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-file:dev
tests:
acceptance_tests:
spec:
- spec_path: "source_file/spec.json"
tests:
- spec_path: "source_file/spec.json"
connection:
- config_path: "integration_tests/config.json"
status: "succeed"
- config_path: "integration_tests/invalid_config.json"
status: "failed"
tests:
- config_path: "integration_tests/config.json"
status: "succeed"
- config_path: "integration_tests/invalid_config.json"
status: "failed"
discovery:
- config_path: "integration_tests/config.json"
tests:
- config_path: "integration_tests/config.json"
basic_read:
- config_path: "integration_tests/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
tests:
- config_path: "integration_tests/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
full_refresh:
- config_path: "integration_tests/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
tests:
- config_path: "integration_tests/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
Original file line number Diff line number Diff line change
Expand Up @@ -149,12 +149,11 @@ def test_discover(source, config, client):

def test_check_wrong_reader_options(source, config):
config["reader_options"] = '{encoding":"utf_16"}'
with pytest.raises(Exception):
source.check(logger=logger, config=config)
assert source.check(logger=logger, config=config) == AirbyteConnectionStatus(
status=Status.FAILED, message="reader_options is not valid JSON"
)


def test_check_google_spreadsheets_url(source, config):
config["url"] = "https://docs.google.com/spreadsheets/d/"
assert source.check(logger=logger, config=config) == AirbyteConnectionStatus(
Expand Down

0 comments on commit f1546c2

Please sign in to comment.