Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

source-faker: unique state messages #34344

Merged
merged 3 commits into from
Jan 18, 2024
Merged

source-faker: unique state messages #34344

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6d24e83
source-faker: unique state messages
evantahler Jan 18, 2024
d4fcb91
fix doc
evantahler Jan 18, 2024
cea93f9
dockerfile bump
evantahler Jan 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-faker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,5 @@ COPY source_faker ./source_faker
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=5.0.1
LABEL io.airbyte.version=5.0.2
LABEL io.airbyte.name=airbyte/source-faker
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-faker/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ data:
connectorSubtype: api
connectorType: source
definitionId: dfd88b22-b603-4c3d-aad7-3701784586b1
dockerImageTag: 5.0.1
dockerImageTag: 5.0.2
dockerRepository: airbyte/source-faker
documentationUrl: https://docs.airbyte.com/integrations/sources/faker
githubIssueLabel: source-faker
Expand Down
8 changes: 4 additions & 4 deletions airbyte-integrations/connectors/source-faker/source_faker/streams.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,9 +119,9 @@ def read_records(self, **kwargs) -> Iterable[Mapping[str, Any]]:
if records_remaining_this_loop == 0:
break

self.state = {"seed": self.seed, "updated_at": updated_at}
self.state = {"seed": self.seed, "updated_at": updated_at, "loop_offset": loop_offset}

self.state = {"seed": self.seed, "updated_at": updated_at}
self.state = {"seed": self.seed, "updated_at": updated_at, "loop_offset": loop_offset}


class Purchases(Stream, IncrementalMixin):
Expand Down Expand Up @@ -180,6 +180,6 @@ def read_records(self, **kwargs) -> Iterable[Mapping[str, Any]]:
if records_remaining_this_loop == 0:
break

self.state = {"seed": self.seed, "updated_at": updated_at}
self.state = {"seed": self.seed, "updated_at": updated_at, "loop_offset": loop_offset}

self.state = {"seed": self.seed, "updated_at": updated_at}
self.state = {"seed": self.seed, "updated_at": updated_at, "loop_offset": loop_offset}
18 changes: 12 additions & 6 deletions docs/integrations/sources/faker.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@

## Sync overview

The Sample Data (Faker) source generates sample data using the python [`mimesis`](https://mimesis.name/en/master/) package.
The Sample Data (Faker) source generates sample data using the python
[`mimesis`](https://mimesis.name/en/master/) package.

### Output schema

This source will generate an "e-commerce-like" dataset with users, products, and purchases. Here's what is produced at a Postgres destination connected to this source:
This source will generate an "e-commerce-like" dataset with users, products, and purchases. Here's
what is produced at a Postgres destination connected to this source:

```sql
CREATE TABLE "public"."users" (
Expand Down Expand Up @@ -84,9 +86,12 @@ CREATE TABLE "public"."purchases" (
| Incremental Sync | Yes | |
| Namespaces | No | |

Of note, if you choose `Incremental Sync`, state will be maintained between syncs, and once you hit `count` records, no new records will be added.
Of note, if you choose `Incremental Sync`, state will be maintained between syncs, and once you hit
`count` records, no new records will be added.

You can choose a specific `seed` (integer) as an option for this connector which will guarantee that the same fake records are generated each time. Otherwise, random data will be created on each subsequent sync.
You can choose a specific `seed` (integer) as an option for this connector which will guarantee that
the same fake records are generated each time. Otherwise, random data will be created on each
subsequent sync.

### Requirements

Expand All @@ -95,8 +100,9 @@ None!
## Changelog

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:----------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------|
| 5.0.1 | 2023-01-08 | [34033](https://github.com/airbytehq/airbyte/pull/34033) | Add standard entrypoints for usage with AirbyteLib |
| :------ | :--------- | :-------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------- |
| 5.0.2 | 2024-01-17 | [34344](https://github.com/airbytehq/airbyte/pull/34344) | Ensure unique state messages |
| 5.0.1 | 2023-01-08 | [34033](https://github.com/airbytehq/airbyte/pull/34033) | Add standard entrypoints for usage with AirbyteLib |
| 5.0.0 | 2023-08-08 | [29213](https://github.com/airbytehq/airbyte/pull/29213) | Change all `*id` fields and `products.year` to be integer |
| 4.0.0 | 2023-07-19 | [28485](https://github.com/airbytehq/airbyte/pull/28485) | Bump to test publication |
| 3.0.2 | 2023-07-07 | [27807](https://github.com/airbytehq/airbyte/pull/28060) | Bump to test publication |
Expand Down
Loading