Tdl 17003 Added back missing field for contact streams #43
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @@ -39,7 +39,7 @@ def transform_conversation_parts(this_json, data_key): | |||
| # Run other transforms, as needed: denest_list_nodes, transform_conversation_parts | |||
| def transform_json(this_json, stream_name, data_key): | |||
| new_json = this_json | |||
| if stream_name in ('users', 'contacts'): # change 'leads' to 'contacts' for API v.2.0 | |||
| if stream_name == 'users': | |||
There was a problem hiding this comment.
Removed contacts stream as the schema is already identical with API response. So no need to perform an extra transformation.
There was a problem hiding this comment.
@umangagrawal-crest Isn't there an existing test case for this line of code?
There was a problem hiding this comment.
No, we don't have an existing test case. Here, we just have reverted the logic that was done as a part of PR31 because we were facing data discrepancy for contacts stream in the new release.
dbshah1212
approved these changes
Dec 29, 2021
karanpanchal-crest
approved these changes
Dec 29, 2021
dmosorast
approved these changes
Dec 31, 2021
KrisPersonal
approved these changes
Dec 31, 2021
Merged
KrisPersonal
pushed a commit
that referenced
this pull request
Jan 5, 2022
* TDL-17000: fix breaking changes and TDL-17017: Fix bookmark for conversation_parts stream. (#45) * Reverted conversation_parts to full_table for parent as V1.1.3 * Updated bookmark test to handle conversation_parts * Updated comments * Added code to select parent_stream if only child stream is selected same as v1.1.3 * Added code comments * Resolved review comment * Resolved review comment * Added code comments * Added bookmark mechanism for conversation_parts same as before but independant of conversations * Added code comments * Added query paramater to get conversations greater and equal to bookmark * Adding changes of PR42 for better unit tests * Added logger messages * Added unit test * Updated unit test * Updated unit test * Updated variables in unit test * Added time_extracted for conversation_parts * Resolved review comments * Added companies to untestable for bookmark test * Reverted changes of last commit * Tdl 17003 Added back missing field for contact streams (#43) * Reverted the logic as per old version * Removed leads as well as it is not supported by tap * Tdl 17006 revert back companies to incremental (#44) * Reverted companies to incremental stream * Debuging integration test * Removed print statement * Added companies to untestable for bookmark test * TDL-17002: Make the logger messages more descriptive (#46) * added logger messages * removed unnecessary loggers * removed unnecessary loggers * updated the code to use stream name instead of parent stream name Co-authored-by: savan-chovatiya <80703490+savan-chovatiya@users.noreply.github.com> Co-authored-by: Umang Agrawal <80704207+umangagrawal-crest@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
JIRA link: https://jira.talendforge.org/browse/TDL-17003
Reverted to the old version
Manual QA steps
Risks
Rollback steps