Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Source faker: change some fields to be integer #29213

Merged
merged 10 commits into from
Aug 9, 2023
Merged

Source faker: change some fields to be integer #29213

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
0bb8960
change some fields to be integer
edgao Aug 8, 2023
84ccc76
logistics
edgao Aug 8, 2023
3a78858
fix tests?
edgao Aug 8, 2023
2f60817
more fix tests?
edgao Aug 8, 2023
78251b3
fine, breaking change :P
edgao Aug 8, 2023
e30621c
migration doc
edgao Aug 8, 2023
a7b8747
metadata
edgao Aug 8, 2023
843e6f7
cat config
edgao Aug 8, 2023
386045d
uh
edgao Aug 8, 2023
c1e8cf5
Merge branch 'master' into edgao/faker_schema
edgao Aug 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-faker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,5 @@ COPY source_faker ./source_faker
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=4.0.0
LABEL io.airbyte.version=5.0.0
LABEL io.airbyte.name=airbyte/source-faker
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-faker/acceptance-test-config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ acceptance_tests:
tests:
- config_path: secrets/config.json
backward_compatibility_tests_config:
disable_for_version: "2.1.0" # We changed the cursor field of all streams to be "updated_at"
disable_for_version: "4.0.0" # We changed the '*_id' and products.year fields to be integers instead of number
basic_read:
tests:
- config_path: secrets/config.json
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-faker/integration_tests/catalog.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"id": { "type": "number" },
"id": { "type": "integer" },
"created_at": {
"type": "string",
"format": "date-time",
Expand Down
12 changes: 6 additions & 6 deletions airbyte-integrations/connectors/source-faker/integration_tests/configured_catalog.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"id": { "type": "number" },
"id": { "type": "integer" },
"created_at": {
"type": "string",
"format": "date-time",
Expand Down Expand Up @@ -45,9 +45,9 @@
"name": "purchases",
"json_schema": {
"properties": {
"id": { "type": "number" },
"user_id": { "type": "number" },
"product_id": { "type": "number" },
"id": { "type": "integer" },
"user_id": { "type": "integer" },
"product_id": { "type": "integer" },
"created_at": {
"type": "string",
"format": "date-time",
Expand Down Expand Up @@ -87,10 +87,10 @@
"name": "products",
"json_schema": {
"properties": {
"id": { "type": "number" },
"id": { "type": "integer" },
"make": { "type": "string" },
"model": { "type": "string" },
"year": { "type": "number" },
"year": { "type": "integer" },
"price": { "type": "number" },
"created_at": {
"type": "string",
Expand Down
5 changes: 4 additions & 1 deletion airbyte-integrations/connectors/source-faker/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ data:
connectorSubtype: api
connectorType: source
definitionId: dfd88b22-b603-4c3d-aad7-3701784586b1
dockerImageTag: 4.0.0
dockerImageTag: 5.0.0
dockerRepository: airbyte/source-faker
githubIssueLabel: source-faker
icon: faker.svg
Expand Down Expand Up @@ -32,6 +32,9 @@ data:
- language:python
releases:
breakingChanges:
5.0.0:
message: "ID and products.year fields are changing to be integers instead of floats."
upgradeDeadline: "2023-08-31"
Comment on lines +35 to +37
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @erohmensing - a wild breaking change has appeared

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I swear I'm not trying to be a conn ops early adopter, it just keeps happening :p

4.0.0:
message: "This is a breaking change message"
upgradeDeadline: "2023-07-19"
Expand Down
4 changes: 2 additions & 2 deletions airbyte-integrations/connectors/source-faker/source_faker/schemas/products.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"id": { "type": "number" },
"id": { "type": "integer" },
"make": { "type": "string" },
"model": { "type": "string" },
"year": { "type": "number" },
"year": { "type": "integer" },
"price": { "type": "number" },
"created_at": {
"type": "string",
Expand Down
6 changes: 3 additions & 3 deletions airbyte-integrations/connectors/source-faker/source_faker/schemas/purchases.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"id": { "type": "number" },
"user_id": { "type": "number" },
"product_id": { "type": "number" },
"id": { "type": "integer" },
"user_id": { "type": "integer" },
"product_id": { "type": "integer" },
"created_at": {
"type": "string",
"format": "date-time",
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-faker/source_faker/schemas/users.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"id": { "type": "number" },
"id": { "type": "integer" },
"created_at": {
"type": "string",
"format": "date-time",
Expand Down
32 changes: 16 additions & 16 deletions airbyte-integrations/connectors/source-faker/unit_tests/unit_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@


class MockLogger:
def debug(a,b, **kwargs):
def debug(a, b, **kwargs):
return None

def info(a,b, **kwargs):
def info(a, b, **kwargs):
return None

def exception(a,b,**kwargs):
def exception(a, b, **kwargs):
print(b)
return None

Expand Down Expand Up @@ -46,7 +46,7 @@ def test_source_streams():

assert len(schemas) == 3
assert schemas[1]["properties"] == {
"id": {"type": "number"},
"id": {"type": "integer"},
"created_at": {"type": "string", "format": "date-time", "airbyte_type": "timestamp_with_timezone"},
"updated_at": {"type": "string", "format": "date-time", "airbyte_type": "timestamp_with_timezone"},
"name": {"type": "string"},
Expand All @@ -62,18 +62,18 @@ def test_source_streams():
"height": {"type": "string"},
"blood_type": {"type": "string"},
"weight": {"type": "integer"},
'address': {
'type': 'object',
'properties': {
'city': {'type': 'string'},
'country_code': {'type': 'string'},
'postal_code': {'type': 'string'},
'province': {'type': 'string'},
'state': {'type': 'string'},
'street_name': {'type': 'string'},
'street_number': {'type': 'string'}
}
}
"address": {
"type": "object",
"properties": {
"city": {"type": "string"},
"country_code": {"type": "string"},
"postal_code": {"type": "string"},
"province": {"type": "string"},
"state": {"type": "string"},
"street_name": {"type": "string"},
"street_number": {"type": "string"},
},
},
}


Expand Down
5 changes: 4 additions & 1 deletion docs/integrations/sources/faker-migrations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# Sample Data (Faker) Migration Guide

## Upgrading to 5.0.0
Some columns are narrowing from `number` to `integer`. You may need to force normalization to rebuild your destination tables by manually dropping the SCD and final tables, refreshing the connection schema (skipping the reset), and running a sync. Alternatively, you can just run a reset.

## Upgrading to 4.0.0

Nothing to do here - this was a test breaking change.
Nothing to do here - this was a test breaking change.
1 change: 1 addition & 0 deletions docs/integrations/sources/faker.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ None!

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:----------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------|
| 5.0.0 | 2023-08-08 | [29213](https://github.com/airbytehq/airbyte/pull/29213) | Change all `*id` fields and `products.year` to be integer |
| 4.0.0 | 2023-07-19 | [28485](https://github.com/airbytehq/airbyte/pull/28485) | Bump to test publication |
| 3.0.2 | 2023-07-07 | [27807](https://github.com/airbytehq/airbyte/pull/28060) | Bump to test publication |
| 3.0.1 | 2023-06-28 | [27807](https://github.com/airbytehq/airbyte/pull/27807) | Fix bug with purchase stream updated_at |
Expand Down
Loading