Re-instantiating Storage Class is SLOW

[jamesmfriedman]

the Storage class located in default.py is causing a lot of slow down. It reinitializes the storage class for every thumbnail that is requested. The problem with that is if you're using something like s3boto, it re-establishes a connection to s3 everytime.

[jamesmfriedman]

A monkey patch for the issue

import sorl.thumbnail.default

from django.utils.functional import LazyObject
from sorl.thumbnail.conf import settings
from sorl.thumbnail.helpers import get_module_class

_storage = None


class Storage(LazyObject):

    def _setup(self):
        global _storage
        if not _storage:
            _storage = get_module_class(settings.THUMBNAIL_STORAGE)()

        self._wrapped = _storage

sorl.thumbnail.default.storage = Storage()

[mariocesar]

@jamesmfriedman have you tested this patch agains the latest rev in the master branch? Also there was an issue with old Boto versions that doesn't reuse connection by default, please confirm that you are using the latest.

Thanks for taking the time, I will look deep into it. Thanks

[jamesmfriedman]

Didn’t try it against master, but it is running and working in 11.12

On Sep 14, 2014, at 6:30 PM, Mario César notifications@github.com wrote:

@jamesmfriedman https://github.com/jamesmfriedman have you tested this patch agains the latest rev in the master branch? Also there was an issue with old Boto versions that doesn't reuse connection by default, please confirm that you are using the latest.

Thanks for taking the time, I will look deep into it. Thanks

—
Reply to this email directly or view it on GitHub #301 (comment).

[mariocesar]

Try pip install sorl-thumbnail==11.12.1b

[pior]

With Boto3 this issue is a real blocker because Boto3 loads some very large JSON files when creating a session object. On Heroku it means 100~150ms per thumbnail, only to generate the secure url (no network calls)

2017-03-25T03:19:40.384783+00:00 app[web.1]: 2017-03-24 23:19:40,384 DEBUG sorl.thumbnail.base:get_thumbnail:68 Getting thumbnail for file [sponsors/YulDev2_1.png] at [200x100]
2017-03-25T03:19:40.391243+00:00 app[web.1]: 2017-03-24 23:19:40,391 DEBUG botocore.loaders:load_file:174 Loading JSON file: /app/.heroku/python/lib/python2.7/site-packages/boto3/data/s3/2006-03-01/resources-1.json
2017-03-25T03:19:40.400819+00:00 app[web.1]: 2017-03-24 23:19:40,400 DEBUG botocore.loaders:load_file:174 Loading JSON file: /app/.heroku/python/lib/python2.7/site-packages/botocore/data/endpoints.json
2017-03-25T03:19:40.424714+00:00 app[web.1]: 2017-03-24 23:19:40,424 DEBUG botocore.loaders:load_file:174 Loading JSON file: /app/.heroku/python/lib/python2.7/site-packages/botocore/data/s3/2006-03-01/service-2.json
2017-03-25T03:19:40.535902+00:00 app[web.1]: 2017-03-24 23:19:40,535 DEBUG botocore.loaders:load_file:174 Loading JSON file: /app/.heroku/python/lib/python2.7/site-packages/botocore/data/_retry.json
...

[claudep]

Should be fixed by #719