🚨🚨🎉Source Pinterest: migrate to low-code

[askarpets]

What

Resolves https://github.com/airbytehq/airbyte-internal-issues/issues/7435

How

Migrated streams to low-code

Review guide

1. manifest.yaml
2. source.py

User Impact

Format of state has changed for incremental streams

Can this PR be safely reverted and rolled back?

• YES 💚
• NO ❌

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 24, 2024 8:46am

[brianjlai]

a few small things and tweaks but overall looks good w/ the adjustments to the error handling.

Have you done any live test run reports to verify that there are no regressions in the behavior? If not, can you can perform at least one or two to verify the changes, I'll ✅ once we have confirmed that

[askarpets]

@brianjlai here is one of runs: https://github.com/airbytehq/airbyte/actions/runs/9211964690 - the only failed test is related to discover, however, I don't see any issues when comparing original and new schemas

[brianjlai]

@brianjlai here is one of runs: https://github.com/airbytehq/airbyte/actions/runs/9211964690 - the only failed test is related to discover, however, I don't see any issues when comparing original and new schemas

I know this was a breaking change, but was this expected? When you say you don't see any issues, do you mean that the diffs on the control vs. yours do not differ at all? Or that the deviations don't change the behavior?

I think I want to understand why the catalog changed and we should be sure that was intended. If you run discover() against master and your version, what is the difference between the two catalogs?

[askarpets]

@brianjlai here is one of runs: https://github.com/airbytehq/airbyte/actions/runs/9211964690 - the only failed test is related to discover, however, I don't see any issues when comparing original and new schemas

I know this was a breaking change, but was this expected? When you say you don't see any issues, do you mean that the diffs on the control vs. yours do not differ at all? Or that the deviations don't change the behavior?

I think I want to understand why the catalog changed and we should be sure that was intended. If you run discover() against master and your version, what is the difference between the two catalogs?

I double-checked the catalogs manually - the schemas are completely the same except the order of streams in the catalogs. This is because I used different order in the manifest file, however, I don't think this should cause any issues.
Additionally, I've created a connection using our test account since there are more streams with data than I could find in customers' connections and ran the tests with this connection - please see the result.

[brianjlai]

I double-checked the catalogs manually - the schemas are completely the same except the order of streams in the catalogs. This is because I used different order in the manifest file, however, I don't think this should cause any issues.
Additionally, I've created a connection using our test account since there are more streams with data than I could find in customers' connections and ran the tests with this connection - please see the result.

Very interesting, thank you for verifying and double checking this. If ordering of streams in the catalog is the only issue, then that should not be a problem. Approved.

My one ask is reach out to @alafanechere @clnoll about this. I'm curious if this is the desired behavior of the live test framework? In the end checking for existence is what we really care about

[clnoll]

Hey @brianjlai & @askarpets, we definitely don't intend to be checking the stream order. I actually thought we'd fixed this. Will look into it, and sorry for the confusion!

[askarpets]

Hey @brianjlai & @askarpets, we definitely don't intend to be checking the stream order. I actually thought we'd fixed this. Will look into it, and sorry for the confusion!

Thank you @clnoll!