Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(upload-files): handle JSONDecodeError (DEV-2907) #621

Merged
merged 3 commits into from Nov 6, 2023

Conversation

jnussbaum
Copy link
Collaborator

No description provided.

@jnussbaum jnussbaum self-assigned this Nov 6, 2023
@jnussbaum jnussbaum changed the title fix(upload-files): handle JSONDecodeError fix(upload-files): handle JSONDecodeError (DEV-2907) Nov 6, 2023
Copy link

linear bot commented Nov 6, 2023

DEV-2907 upload-files: unhandled JSONDecodeError leads to crash

At the end of the process-files program, when the results from the different workers are collected, the following error happened:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python3.11/3.11.6/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python3.11/3.11.6/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python3.11/3.11.6/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/dsp-tools", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dsp_tools/cli.py", line 577, in main
    run(sys.argv[1:])
  File "/usr/local/lib/python3.11/site-packages/dsp_tools/cli.py", line 610, in run
    success = _call_requested_action(parsed_arguments)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dsp_tools/cli.py", line 499, in _call_requested_action
    success = upload_files(
              ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dsp_tools/fast_xmlupload/upload_files.py", line 409, in upload_files
    result = _upload_files_in_parallel(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dsp_tools/fast_xmlupload/upload_files.py", line 325, in _upload_files_in_parallel
    result.append(uploaded.result())
                  ^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python3.11/3.11.6/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python3.11/3.11.6/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/Cellar/python3.11/3.11.6/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dsp_tools/fast_xmlupload/upload_files.py", line 275, in _upload_file
    res = _upload_without_processing(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dsp_tools/fast_xmlupload/upload_files.py", line 226, in _upload_without_processing
    if response_upload.json().get("message") == "server.fs.mkdir() failed: File exists":
       ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

@jnussbaum jnussbaum merged commit 40000a8 into main Nov 6, 2023
8 of 9 checks passed
@jnussbaum jnussbaum deleted the wip/upload-files-prevent-jsondecodeerror branch November 6, 2023 12:07
@daschbot daschbot mentioned this pull request Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant