Skip to content

fix(submitted_landings): support submitted landings without answers#69

Merged
rdeshmukh15 merged 1 commit into
singer-io:masterfrom
MuriloKakazu:fix-submitted-landings/null-answers
Nov 3, 2022
Merged

fix(submitted_landings): support submitted landings without answers#69
rdeshmukh15 merged 1 commit into
singer-io:masterfrom
MuriloKakazu:fix-submitted-landings/null-answers

Conversation

@MuriloKakazu

Copy link
Copy Markdown
Contributor

Description of change

This fixes a problem in which this tap would fail when at least 1 submitted landing had no answers. This was resulting in a problem in which the process would exit and a bunch of data would not be extracted.

This problem happens for forms in which every field is not required, thus the user is able to submit the response without answering any questions. The following error then would show up when running this tap if at least a single user response fell in such scenario:

CRITICAL 'NoneType' object is not iterable
Traceback (most recent call last):
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/bin/tap-typeform", line 33, in <module>
    sys.exit(load_entry_point('tap-typeform==2.0.0', 'console_scripts', 'tap-typeform')())
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/singer/utils.py", line 229, in wrapped
    return fnc(*args, **kwargs)
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/__init__.py", line 49, in main
    _sync(client, config, args.state, catalog.to_dict())
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/sync.py", line 83, in sync
    stream_obj.sync_obj(client, state, catalog['streams'], form, config["start_date"],
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/streams.py", line 151, in sync_obj
    max_bookmark = self.write_records(records, catalogs, selected_stream_ids,
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/streams.py", line 127, in write_records
    max_bookmark =  self.sync_child_stream(record, catalogs, state, selected_stream_ids,form_id, start_date, max_bookmark)
  File "/Users/murilo.kakazu/.virtualenvs/tap-typeform/lib/python3.10/site-packages/tap_typeform-2.0.0-py3.10.egg/tap_typeform/streams.py", line 95, in sync_child_stream
    for rec in record[self.child_data_key]:
TypeError: 'NoneType' object is not iterable

This is bad, as even if the admin updates the form now making the questions required, the existing responses will still fail to be processed by this tap, as old responses without answers will still exist.

I fixed this by simply checking if the stream's child key is blank on the record, then it won't try to process the children data. e.g: when running the submitted_landing stream, if the record's answers attribute is blank, then it will register a submitted_landing, but no answers. As expected 😄

Also, this fixes both reported issues: #22 and #24

Manual QA steps

1 - Create a form, with 1 or more questions, in which none of them have the "required" validation. e.g:
image
2 - Open the form and submit it without answering the questions
3 - Run this tap for that form_id
4 - 💥! CRITICAL 'NoneType' object is not iterable

Risks

  • None I could think of

Rollback steps

  • revert this branch

@MuriloKakazu MuriloKakazu changed the title fix(submitted-landings): support submitted landings without answers fix(submitted_landings): support submitted landings without answers Oct 31, 2022
@rdeshmukh15
rdeshmukh15 requested a review from luandy64 November 2, 2022 13:13
@rdeshmukh15
rdeshmukh15 merged commit cd5147e into singer-io:master Nov 3, 2022
@MuriloKakazu
MuriloKakazu deleted the fix-submitted-landings/null-answers branch December 6, 2022 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants