New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tdl 21520 skip donotprocess records #115
Conversation
tap_pendo/streams.py
Outdated
parent_last_updated = datetime.fromtimestamp(float(record['metadata']['auto'][self.replication_key]) / 1000.0, timezone.utc) | ||
# If there is no last_update key then set 'last_updated=now()' and fetch the records from recent bookmark | ||
parent_last_updated = datetime.fromtimestamp(float( | ||
record['metadata']['auto'][self.replication_key]) / 1000.0, timezone.utc) if record['metadata']['auto'].get(self.replication_key) else now() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
140 characters on one line which is too long. Can you please align it properly. 100 chars per line are preferred.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we know under what scenarios a record will not have a last_update key? Is it linked exclusively to visitors with do_not_process flags set?
.circleci/config.yml
Outdated
@@ -22,7 +22,7 @@ jobs: | |||
command: | | |||
source /usr/local/share/virtualenvs/tap-pendo/bin/activate | |||
# TODO: Adjust the pylint disables | |||
pylint tap_pendo --disable 'broad-except,chained-comparison,empty-docstring,fixme,invalid-name,line-too-long,missing-class-docstring,missing-function-docstring,missing-module-docstring,no-else-raise,no-else-return,too-few-public-methods,too-many-arguments,too-many-branches,too-many-lines,too-many-locals,ungrouped-imports,wrong-spelling-in-comment,wrong-spelling-in-docstring,bad-whitespace,missing-class-docstring' | |||
pylint tap_pendo --disable 'broad-except,chained-comparison,empty-docstring,fixme,invalid-name,line-too-long,missing-class-docstring,missing-function-docstring,missing-module-docstring,no-else-raise,no-else-return,too-few-public-methods,too-many-arguments,too-many-branches,too-many-lines,too-many-locals,ungrouped-imports,wrong-spelling-in-comment,wrong-spelling-in-docstring,bad-whitespace,missing-class-docstring,too-many-statements' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
break into multiple lines for readability purpose.
Description of change
If any visitor is set the Do Not Process flag then Pendo will stop collecting events from that visitor and will stop displaying guides to that visitor. In such scenario we should skip fetching visitor_history.
Manual QA steps
donotprocess
records are getting skipped.Risks
Rollback steps