Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Source Facebook Marketing: remove testing deleted records from integration test, move it to mock server tests #37095

Merged

Conversation

darynaishchenko
Copy link
Collaborator

resolved: https://github.com/airbytehq/airbyte-internal-issues/issues/6552

What:
removed integration tests that were based on live data and added new test suit for deleted records to mock server tests (unit_tests/integration)

@darynaishchenko darynaishchenko self-assigned this Apr 12, 2024
Copy link

vercel bot commented Apr 12, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Apr 17, 2024 4:39pm

Copy link
Contributor

@maxi297 maxi297 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't check if the coverage between the integration tests and those were equivalent but it seems like a good start. For example, were the integration tests implicitly covering pagination? Assuming the coverage is the same, I'm good with this change

)
filters = (
'[{"field":"ad.effective_status","operator":"IN","value":["ACTIVE","ARCHIVED"]},'
'{"field":"ad.updated_time","operator":"GREATER_THAN","value":1672531200}]'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does this value comes from? It seems like it should be relevant but without a variable, it is hard to know the link between this and where it comes from

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added comment for filters

output = self._read(config().with_ad_statuses(self.statuses), "ads")
assert len(output.records) == 1
account_state = output.most_recent_state.dict()["stream_state"][self.account_id]
assert self.filter_statuses_flag in account_state, f"State should include `filter_statuses` flag"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message is clear but without context about the "why", it is hard to understand why this is asserted here. Can we add more info on that?

Copy link
Collaborator Author

@darynaishchenko darynaishchenko Apr 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated error message with reason why we need filter_statuses

get_account_response(),
)
filters = (
'[{"field":"ad.effective_status","operator":"IN","value":["ACTIVE","ARCHIVED"]},'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use self. statuses here to clarify the link between the config and the API call? The same comment applies for the other tests

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated with self.statuses

assert len(output.records) == 1
account_state = output.most_recent_state.dict()["stream_state"][self.account_id]
assert self.filter_statuses_flag in account_state, f"State should include `filter_statuses` flag"
assert account_state == {"filter_statuses": ["ACTIVE", "ARCHIVED"], "updated_time": "2023-03-21T22:41:46-0700"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use self. statuses here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated with self.statuses

@darynaishchenko
Copy link
Collaborator Author

I haven't check if the coverage between the integration tests and those were equivalent but it seems like a good start. For example, were the integration tests implicitly covering pagination? Assuming the coverage is the same, I'm good with this change

Test logic is the same, we check for deleted records are in response and states have filter statuses. Also parent stream are covered in unit tests already, so we don't need to add test for stream slices etc. Pagination is a part of fb lib, I don't think we should test it.

@darynaishchenko darynaishchenko merged commit 9f2d717 into master Apr 18, 2024
33 checks passed
@darynaishchenko darynaishchenko deleted the daryna/source-facebook-marketing/failing-int-test branch April 18, 2024 09:24
strosek pushed a commit that referenced this pull request Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants