Skip to content

Source Facebook Marketing: fix start_date selection behaviour - #36025

Merged
Artem Inzhyyants (artem1205) merged 24 commits into
masterfrom
artem1205/source-facebook-OC-4495-2
Mar 20, 2024
Merged

Source Facebook Marketing: fix start_date selection behaviour#36025
Artem Inzhyyants (artem1205) merged 24 commits into
masterfrom
artem1205/source-facebook-OC-4495-2

Conversation

@artem1205

Copy link
Copy Markdown
Contributor

What

Resolve https://github.com/airbytehq/oncall/issues/4495

How

fix logic of start_date to sync from;
considering lookback window do not sync data prior start_date

Recommended reading order

  1. airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/streams/base_insight_streams.py

🚨 User Impact 🚨

no breaking changes

Pre-merge Actions

Updating a connector

Community member or Airbyter

  • Grant edit access to maintainers (instructions)
  • Unit & integration tests added

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • If new credentials are required for use in CI, add them to GSM. Instructions.

Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
@vercel

vercel Bot commented Mar 13, 2024

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Mar 20, 2024 1:02pm

Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
@artem1205
Artem Inzhyyants (artem1205) marked this pull request as ready for review March 13, 2024 19:32
start_date = self._start_date

if start_date < self._start_date:
logger.warning(f"Ignore provided state and start sync from start_date ({self._start_date}).")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we have the message like this?

logger.warning(f"Starting the sync from: ({self._start_date}). The provided state: {%} was ignored.")

Not sure about the state mention in this context, but it seems like one of [start_date, self._start_date, oldest_date] ?

@erohmensing

Copy link
Copy Markdown
Contributor

Artem Inzhyyants (@artem1205) I want to check - when we are moving up the start date, are the end dates for the slices adjusted accordingly? Looking into this and I want to make sure it is addressed (since the start and end date that were overlapping were both before the user's prescribed start date, I'm hoping that this was caused by the same root cause but feel free to redirect me if you think otherwise)

@artem1205

Copy link
Copy Markdown
Contributor Author

Ella Rohm-Ensing (@erohmensing) , Yep, it should be covered. see unit tests starting from test_stream_slices_with_state_close_to_now

…cebook-OC-4495-2

# Conflicts:
#	airbyte-integrations/connectors/source-facebook-marketing/metadata.yaml
#	airbyte-integrations/connectors/source-facebook-marketing/pyproject.toml
#	docs/integrations/sources/facebook-marketing.md
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
@artem1205
Artem Inzhyyants (artem1205) deleted the artem1205/source-facebook-OC-4495-2 branch March 20, 2024 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/facebook-marketing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants