Skip to content

Commit

Permalink
source-faker: unique state messages (#34344)
Browse files Browse the repository at this point in the history
  • Loading branch information
evantahler committed Jan 18, 2024
1 parent a9f2b29 commit 1b0ad24
Show file tree
Hide file tree
Showing 4 changed files with 18 additions and 12 deletions.
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-faker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,5 @@ COPY source_faker ./source_faker
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=5.0.1
LABEL io.airbyte.version=5.0.2
LABEL io.airbyte.name=airbyte/source-faker
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-faker/metadata.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ data:
connectorSubtype: api
connectorType: source
definitionId: dfd88b22-b603-4c3d-aad7-3701784586b1
dockerImageTag: 5.0.1
dockerImageTag: 5.0.2
dockerRepository: airbyte/source-faker
documentationUrl: https://docs.airbyte.com/integrations/sources/faker
githubIssueLabel: source-faker
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -119,9 +119,9 @@ def read_records(self, **kwargs) -> Iterable[Mapping[str, Any]]:
if records_remaining_this_loop == 0:
break

self.state = {"seed": self.seed, "updated_at": updated_at}
self.state = {"seed": self.seed, "updated_at": updated_at, "loop_offset": loop_offset}

self.state = {"seed": self.seed, "updated_at": updated_at}
self.state = {"seed": self.seed, "updated_at": updated_at, "loop_offset": loop_offset}


class Purchases(Stream, IncrementalMixin):
Expand Down Expand Up @@ -180,6 +180,6 @@ def read_records(self, **kwargs) -> Iterable[Mapping[str, Any]]:
if records_remaining_this_loop == 0:
break

self.state = {"seed": self.seed, "updated_at": updated_at}
self.state = {"seed": self.seed, "updated_at": updated_at, "loop_offset": loop_offset}

self.state = {"seed": self.seed, "updated_at": updated_at}
self.state = {"seed": self.seed, "updated_at": updated_at, "loop_offset": loop_offset}
18 changes: 12 additions & 6 deletions docs/integrations/sources/faker.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@

## Sync overview

The Sample Data (Faker) source generates sample data using the python [`mimesis`](https://mimesis.name/en/master/) package.
The Sample Data (Faker) source generates sample data using the python
[`mimesis`](https://mimesis.name/en/master/) package.

### Output schema

This source will generate an "e-commerce-like" dataset with users, products, and purchases. Here's what is produced at a Postgres destination connected to this source:
This source will generate an "e-commerce-like" dataset with users, products, and purchases. Here's
what is produced at a Postgres destination connected to this source:

```sql
CREATE TABLE "public"."users" (
Expand Down Expand Up @@ -84,9 +86,12 @@ CREATE TABLE "public"."purchases" (
| Incremental Sync | Yes | |
| Namespaces | No | |

Of note, if you choose `Incremental Sync`, state will be maintained between syncs, and once you hit `count` records, no new records will be added.
Of note, if you choose `Incremental Sync`, state will be maintained between syncs, and once you hit
`count` records, no new records will be added.

You can choose a specific `seed` (integer) as an option for this connector which will guarantee that the same fake records are generated each time. Otherwise, random data will be created on each subsequent sync.
You can choose a specific `seed` (integer) as an option for this connector which will guarantee that
the same fake records are generated each time. Otherwise, random data will be created on each
subsequent sync.

### Requirements

Expand All @@ -95,8 +100,9 @@ None!
## Changelog

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:----------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------|
| 5.0.1 | 2023-01-08 | [34033](https://github.com/airbytehq/airbyte/pull/34033) | Add standard entrypoints for usage with AirbyteLib |
| :------ | :--------- | :-------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------- |
| 5.0.2 | 2024-01-17 | [34344](https://github.com/airbytehq/airbyte/pull/34344) | Ensure unique state messages |
| 5.0.1 | 2023-01-08 | [34033](https://github.com/airbytehq/airbyte/pull/34033) | Add standard entrypoints for usage with AirbyteLib |
| 5.0.0 | 2023-08-08 | [29213](https://github.com/airbytehq/airbyte/pull/29213) | Change all `*id` fields and `products.year` to be integer |
| 4.0.0 | 2023-07-19 | [28485](https://github.com/airbytehq/airbyte/pull/28485) | Bump to test publication |
| 3.0.2 | 2023-07-07 | [27807](https://github.com/airbytehq/airbyte/pull/28060) | Bump to test publication |
Expand Down

0 comments on commit 1b0ad24

Please sign in to comment.