Standard Tests: fix incremental test and docs #3161
Conversation
+ update docs and templates
keu
requested review from
sherifnada
and removed request for
michel-tricot and
ChristopheDuong
|
|
||
| from setuptools import find_packages, setup | ||
| TEST_REQUIREMENTS = [ | ||
| "pytest==6.1.2", |
There was a problem hiding this comment.
wouldn't it be better to do something like ^6.1 rather than pin to a specific version?
airbyte-integrations/connector-templates/source-python-http-api/integration_tests/acceptance.py
Outdated
Show resolved
Hide resolved
| @@ -0,0 +1,29 @@ | |||
| # See [Source Acceptance Tests](https://docs.airbyte.io/contributing-to-airbyte/building-new-connector/source-acceptance-tests.md) | |||
There was a problem hiding this comment.
not needed for this PR but the number of common files between these templates is growing. Maybe we should have a common directory connector-templates/common which contains all common files.
There was a problem hiding this comment.
if we will migrate to cookiecuttter we can have a single template for all types of connectors, solving this problem in such case
| def parse_value(value, type_): | ||
| if type_ in ("datetime", "date-time"): | ||
| def parse_value(value: Any, type_: Set[str]): | ||
| if type_ & {"datetime", "date-time"}: |
There was a problem hiding this comment.
this can be expressed like this:
if type_.intersect({"datetime", "date-time"}):There was a problem hiding this comment.
I think I slightly prefer that just because it's less cognitive load (intersect tells you exactly what it is doing)
| try: | ||
| return self.get_property(path)["type"] | ||
| field = self.get_property(path) | ||
| type_ = field.get("format", field["type"]) |
There was a problem hiding this comment.
can we call this format? very confusing to call it type
|
|
||
| def get_cursor_value(self, record, cursor_path): | ||
| type_ = self.get_type_for_key_path(path=cursor_path) | ||
| value = reduce(lambda data, key: data[key], cursor_path, record) | ||
| return self.parse_value(value, type_) | ||
|
|
||
| @staticmethod | ||
| def parse_value(value, type_): | ||
| if type_ in ("datetime", "date-time"): | ||
| def parse_value(value: Any, type_: Set[str]): |
There was a problem hiding this comment.
are these parameters well named? it seems like value is actually JSON Schema type and type_ is format?
There was a problem hiding this comment.
type is something in between I would say, but ok, lets make format
value is an actual value, it can be 1, '1', 'sometext', 20.0, '2010-10-11'
format is what tells us how to parse it
...integrations/bases/source-acceptance-test/source_acceptance_test/utils/json_schema_helper.py
Outdated
Show resolved
Hide resolved
…i/integration_tests/acceptance.py Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
…tance_test/utils/json_schema_helper.py Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
What
Describe what the change is solving
It helps to add screenshots if it affects the frontend.
How
Describe the solution
Pre-merge Checklist
Recommended reading order
test.javacomponent.ts