Source Amazon Seller Partner: Add GET_SELLER_FEEDBACK_DATA report #8021
Conversation
|
@lizdeika Hey can you move this PR out of Draft once you are done with changes. Thanks for your contribution. |
|
@lizdeika hey thanks for the contribution. I had a question |
Hi, |
|
@lizdeika thanks for sharing the info that was really helpful. Can you share across output for integrations test and unit tests ? |
|
@harshithmullapudi I have educated myself on incremental syncs and slicing |
github-actions
bot
added
the
area/documentation
|
@harshithmullapudi any idea why data from raw table(data gets synced successfully) does not go to deduped _scd table when Sync mode is "Incremental | Deduped + history" |
|
|
||
|
|
||
| class SellerFeedbackReports(IncrementalReportsAmazonSPStream): | ||
| name = "GET_SELLER_FEEDBACK_DATA" |
There was a problem hiding this comment.
Add doc string as other streams.
There was a problem hiding this comment.
https://sellercentral.amazon.com/help/hub/reference/G202125660 this is the closest I can get
| "Date": { | ||
| "type": ["null", "string"], | ||
| "format": "date" | ||
| }, | ||
| "Rating": { | ||
| "type": ["null", "number"] | ||
| }, | ||
| "Comments": { | ||
| "type": ["null", "string"] | ||
| }, | ||
| "Response": { | ||
| "type": ["null", "string"] | ||
| }, | ||
| "Order ID": { | ||
| "type": ["null", "string"] | ||
| }, | ||
| "Rater Email": { | ||
| "type": ["null", "string"] |
There was a problem hiding this comment.
Is it the name of this fields correct with the raw data/response from the API?
There was a problem hiding this comment.
Yes, fields are named exactly like this
| from Crypto.Cipher import AES | ||
| from source_amazon_seller_partner.auth import AWSSignature | ||
|
|
||
| REPORTS_API_VERSION = "2020-09-04" | ||
| ORDERS_API_VERSION = "v0" | ||
| VENDORS_API_VERSION = "v1" | ||
|
|
||
| REPORTS_MAX_WAIT_SECONDS = 50 | ||
| # 33min. taken from real world experience working with amazon seller partner reports | ||
| REPORTS_MAX_WAIT_SECONDS = 1980 |
There was a problem hiding this comment.
This is based on your amount of data? Maybe transform this to a parameter where people can increase in necessity?
There was a problem hiding this comment.
This is based on experience working with various different report types. They usually are generated faster but 30 minutes was maximum we saw. In any case 50 seconds is way too little.
Parameter is a good idea. Can I do it in another small PR?
|
@marcosmarxm @harshithmullapudi any blocking stuff to get this in? |
…rbytehq#8021) * Source Amazon Seller Partner: Add GET_SELLER_FEEDBACK_DATA report * no field definition here * add dataStartTime, dataEndTime * fixes, version bumps * revert this * cleanup * fixes * real world wait * fix typo * incremental, transformer * fix stream schema * implement incremental with slicing * move * cleanup * update configured catalog * fix tests * changelog * doc * fix doc * fix configured catalog * Field definitions
What
Adds GET_SELLER_FEEDBACK_DATA report to Amazon Seller Partner source connector
🚨 User Impact 🚨
None
Community member or Airbyter
airbyte_secret./gradlew :airbyte-integrations:connectors:<name>:integrationTest.README.mdbootstrap.md. See description and examplesdocs/SUMMARY.mddocs/integrations/<source or destination>/<name>.mdincluding changelog. See changelog exampledocs/integrations/README.mdairbyte-integrations/builds.mdAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>command is passing./publishcommand described here