Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(submitted_landings): support submitted landings without answers #69

Conversation

MuriloKakazu
Copy link
Contributor

Description of change

This fixes a problem in which this tap would fail when at least 1 submitted landing had no answers. This was resulting in a problem in which the process would exit and a bunch of data would not be extracted.

This problem happens for forms in which every field is not required, thus the user is able to submit the response without answering any questions. The following error then would show up when running this tap if at least a single user response fell in such scenario:

CRITICAL 'NoneType' object is not iterable
Traceback (most recent call last):
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/bin/tap-typeform", line 33, in <module>
    sys.exit(load_entry_point('tap-typeform==2.0.0', 'console_scripts', 'tap-typeform')())
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/singer/utils.py", line 229, in wrapped
    return fnc(*args, **kwargs)
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/__init__.py", line 49, in main
    _sync(client, config, args.state, catalog.to_dict())
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/sync.py", line 83, in sync
    stream_obj.sync_obj(client, state, catalog['streams'], form, config["start_date"],
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/streams.py", line 151, in sync_obj
    max_bookmark = self.write_records(records, catalogs, selected_stream_ids,
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/streams.py", line 127, in write_records
    max_bookmark =  self.sync_child_stream(record, catalogs, state, selected_stream_ids,form_id, start_date, max_bookmark)
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/streams.py", line 95, in sync_child_stream
    for rec in record[self.child_data_key]:
TypeError: 'NoneType' object is not iterable

This is bad, as even if the admin updates the form now making the questions required, the existing responses will still fail to be processed by this tap, as old responses without answers will still exist.

I fixed this by simply checking if the stream's child key is blank on the record, then it won't try to process the children data. e.g: when running the submitted_landing stream, if the record's answers attribute is blank, then it will register a submitted_landing, but no answers. As expected 😄

Also, this fixes both reported issues: #22 and #24

Manual QA steps

1 - Create a form, with 1 or more questions, in which none of them have the "required" validation. e.g:
image
2 - Open the form and submit it without answering the questions
3 - Run this tap for that form_id
4 - 💥! CRITICAL 'NoneType' object is not iterable

Risks

  • None I could think of

Rollback steps

  • revert this branch

@MuriloKakazu MuriloKakazu changed the title fix(submitted-landings): support submitted landings without answers fix(submitted_landings): support submitted landings without answers Oct 31, 2022
@rdeshmukh15 rdeshmukh15 requested a review from luandy64 November 2, 2022 13:13
@rdeshmukh15 rdeshmukh15 merged commit cd5147e into singer-io:master Nov 3, 2022
@MuriloKakazu MuriloKakazu deleted the fix-submitted-landings/null-answers branch December 6, 2022 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants