Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File.download hangs in debug statement indefinately #386

Closed
gondree opened this issue Jul 5, 2020 · 0 comments · Fixed by #443
Closed

File.download hangs in debug statement indefinately #386

gondree opened this issue Jul 5, 2020 · 0 comments · Fixed by #443
Labels

Comments

@gondree
Copy link

gondree commented Jul 5, 2020

Describe the bug

File.download hangs when downloading a ~230 MB tgz file. I waited 10+ minutes, but never long enough to see it terminate. I have not tried enough files to produce a minimal working example.

Based on traceback when cancelling mid-operation, the library appears hung in this try/except block:

            try:
                logger.debug("Data: {data}".format(data=pformat(response.json())))
            except ValueError:
                logger.debug("Data: {data}".format(data=pformat(response.text)))

Specifically, its hung performing response.json()

  ...
  File "/Users/gondree/anaconda/envs/hob/lib/python3.6/site-packages/canvasapi/file.py", line 32, in download
    response = self._requester.request("GET", _url=self.url)
  File "/Users/gondree/anaconda/envs/hob/lib/python3.6/site-packages/canvasapi/requester.py", line 226, in request
    logger.debug("Data: {data}".format(data=pformat(response.json())))
  File "/Users/gondree/anaconda/envs/hob/lib/python3.6/site-packages/requests/models.py", line 898, in json
    return complexjson.loads(self.text, **kwargs)
  File "/Users/gondree/anaconda/envs/hob/lib/python3.6/site-packages/requests/models.py", line 858, in text
    encoding = self.apparent_encoding
  File "/Users/gondree/anaconda/envs/hob/lib/python3.6/site-packages/requests/models.py", line 728, in apparent_encoding
    return chardet.detect(self.content)['encoding']
  File "/Users/gondree/anaconda/envs/hob/lib/python3.6/site-packages/chardet/__init__.py", line 38, in detect
    detector.feed(byte_str)
  File "/Users/gondree/anaconda/envs/hob/lib/python3.6/site-packages/chardet/universaldetector.py", line 211, in feed
    if prober.feed(byte_str) == ProbingState.FOUND_IT:
  File "/Users/gondree/anaconda/envs/hob/lib/python3.6/site-packages/chardet/charsetgroupprober.py", line 71, in feed
    state = prober.feed(byte_str)
  File "/Users/gondree/anaconda/envs/hob/lib/python3.6/site-packages/chardet/sbcharsetprober.py", line 79, in feed
    byte_str = self.filter_international_words(byte_str)
  File "/Users/gondree/anaconda/envs/hob/lib/python3.6/site-packages/chardet/charsetprober.py", line 87, in filter_international_words
    buf)
  File "/Users/gondree/anaconda/envs/hob/lib/python3.6/re.py", line 222, in findall
    return _compile(pattern, flags).findall(string)

Expected behavior

The expected behavior is that the file gets downloaded, no matter the size and no matter the contents.

Environment information

  • Python version (python --version)
$ python --version
Python 3.6.2 :: Continuum Analytics, Inc.
  • CanvasAPI version (pip show canvasapi)
Name: canvasapi
Version: 0.16.0
Summary: API wrapper for the Canvas LMS
Home-page: https://github.com/ucfopen/canvasapi
Author: University of Central Florida - Center for Distributed Learning
Author-email: techrangers@ucf.edu
License: MIT License
Location: /Users/gondree/anaconda/envs/hob/lib/python3.6/site-packages
Requires: requests, pytz, six
Required-by: 

Additional context

I tried to implement a timeout, like:

        ohandler = signal.getsignal(signal.SIGALRM)
        try:
            signal.signal(signal.SIGALRM, lambda a,b: (_ for _ in ()).throw(ValueError))
            signal.alarm(2)
            logger.debug("Data: {data}".format(data=pformat(response.json())))
        except ValueError:
            logger.debug("Data: {data}".format(data=pformat(response.text)))
        finally:
            signal.signal(signal.SIGALRM, ohandler)
            signal.alarm(0)

But, generally, that did not work correctly.

The simplest fix for me was avoiding the operations when not debugging:

        if logger.isEnabledFor(logging.DEBUG):
            try:
                logger.debug("Data: {data}".format(data=pformat(response.json())))
            except ValueError:
                logger.debug("Data: {data}".format(data=pformat(response.text)))

This solved my problem. It would not be ideal if you ever needed to debug in a situation involving one of these problematic downloads. Reporting in hope that this, or a more clever fix, can be incorporated.

@gondree gondree added the bug label Jul 5, 2020
blepabyte added a commit to blepabyte/canvasapi that referenced this issue Nov 18, 2020
@Thetwam Thetwam added this to the CanvasAPI v2.1.0 milestone Dec 2, 2020
Thetwam added a commit that referenced this issue Dec 2, 2020
* Fix rtype in request method

* Should fix #386 by avoiding call to response.text while logging as response may contain binary data

* Add option for File.get_contents to return bytes instead of a string

* Add test for a response that returns invalid unicode + run formatter

Co-authored-by: Matthew Emond <me@ucf.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants