Add an option to replace the WSGI file wrapper with ``pyramid.respons…

…e.FileIter`` for problematic environments. Also increase filedepot test coverage

CHANGES.txt

@@ -4,7 +4,8 @@ Change History
 1.3.0 - unreleased
 ------------------
 
-- No changes yet.
+- Add a ``kotti.depot_replace_wsgi_file_wrapper`` option to replace the WSGI
+  file wrapper with ``pyramid.response.FileIter`` for problematic environments.
 
 1.3.0-alpha.3 - 2016-01-11
 --------------------------

docs/developing/advanced/blobs.rst

@@ -38,6 +38,12 @@ The default value for ``kotti.depot_mountpoint`` is ``/depot``::
 
     kotti.depot_mountpoint = /depot
 
+WSGI File Wrapper
+-----------------
+
+In case you have issues serving files with your WSGI server, your can try to set ``kotti.depot_replace_wsgi_file_wrapper = true``.
+This forces Kotti to use :class:`pyramid.response.FileIter` instead of the one provided by your WSGI server.
+
 Storages
 ~~~~~~~~
 

docs/developing/basic/configuration.rst

@@ -43,46 +43,47 @@ The rest has defaults.
 Do take a look at the required settings (in bold) and adjust them in your site's configuration.
 A few of the settings are less important, and sometimes only used by developers, not integrators.
 
-================================  ==============================================
-Setting                           Description
-================================  ==============================================
-**kotti.site_title**              The title of your site
-**kotti.secret**                  Secret token used for the initial admin password
-kotti.secret2                     Secret token used for email password reset token
-**sqlalchemy.url**                `SQLAlchemy database URL`_
-**mail.default_sender**           Sender address for outgoing email
-kotti.asset_overrides             Override Kotti's templates
-kotti.authn_policy_factory        Component used for authentication
-kotti.authz_policy_factory        Component used for authorization
-kotti.available_types             List of active content types
-kotti.base_includes               List of base Python configuration hooks
-kotti.caching_policy_chooser      Component for choosing the cache header policy
-kotti.configurators               List of advanced functions for config
-kotti.date_format                 Date format to use, default: ``medium``
-kotti.datetime_format             Datetime format to use, default: ``medium``
-kotti.depot_mountpoint            Configure the mountpoint for the blob storage.  See :ref:`blobs` for details.
-kotti.depot.*.*                   Configure the blob storage.  See :ref:`blobs` for details.
-kotti.fanstatic.edit_needed       List of static resources used for edit interface
-kotti.fanstatic.view_needed       List of static resources used for public interface
-kotti.login_success_callback      Override Kotti's default ``login_success_callback`` function
-kotti.max_file_size               Max size for file uploads, default: ``10`` (MB)
-kotti.modification_date_excludes  List of attributes in dotted name notation that should not trigger an update of ``modification_date`` on change
-kotti.populators                  List of functions to fill initial database
-kotti.request_factory             Override Kotti's default request factory
-kotti.reset_password_callback     Override Kotti's default ``reset_password_callback`` function
-kotti.root_factory                Override Kotti's default Pyramid *root factory*
-kotti.sanitize_on_write           Configure :ref:`sanitizers` to be used on write access to resource objects
-kotti.sanitizers                  Configure available :ref:`sanitizers`
-kotti.search_content              Override Kotti's default search function
-kotti.session_factory             Component used for sessions
-kotti.templates.api               Override ``api`` object available in templates
-kotti.time_format                 Time format to use, default: ``medium``
-kotti.url_normalizer              Component used for url normalization
-kotti.zcml_includes               List of packages to include the ZCML from
-mail.host                         Email host to send from
-pyramid.default_locale_name       Set the user interface language, default ``en``
-pyramid.includes                  List of Python configuration hooks
-================================  ==============================================
+=====================================  =========================================
+Setting                                Description
+=====================================  =========================================
+**kotti.site_title**                   The title of your site
+**kotti.secret**                       Secret token used for the initial admin password
+kotti.secret2                          Secret token used for email password reset token
+**sqlalchemy.url**                     `SQLAlchemy database URL`_
+**mail.default_sender**                Sender address for outgoing email
+kotti.asset_overrides                  Override Kotti's templates
+kotti.authn_policy_factory             Component used for authentication
+kotti.authz_policy_factory             Component used for authorization
+kotti.available_types                  List of active content types
+kotti.base_includes                    List of base Python configuration hooks
+kotti.caching_policy_chooser           Component for choosing the cache header policy
+kotti.configurators                    List of advanced functions for config
+kotti.date_format                      Date format to use, default: ``medium``
+kotti.datetime_format                  Datetime format to use, default: ``medium``
+kotti.depot_mountpoint                 Configure the mountpoint for the blob storage.  See :ref:`blobs` for details.
+kotti.depot_replace_wsgi_file_wrapper  Replace you WSGI server's file wrapper with :class:`pyramid.response.FileIter`.
+kotti.depot.*.*                        Configure the blob storage.  See :ref:`blobs` for details.
+kotti.fanstatic.edit_needed            List of static resources used for edit interface
+kotti.fanstatic.view_needed            List of static resources used for public interface
+kotti.login_success_callback           Override Kotti's default ``login_success_callback`` function
+kotti.max_file_size                    Max size for file uploads, default: ``10`` (MB)
+kotti.modification_date_excludes       List of attributes in dotted name notation that should not trigger an update of ``modification_date`` on change
+kotti.populators                       List of functions to fill initial database
+kotti.request_factory                  Override Kotti's default request factory
+kotti.reset_password_callback          Override Kotti's default ``reset_password_callback`` function
+kotti.root_factory                     Override Kotti's default Pyramid *root factory*
+kotti.sanitize_on_write                Configure :ref:`sanitizers` to be used on write access to resource objects
+kotti.sanitizers                       Configure available :ref:`sanitizers`
+kotti.search_content                   Override Kotti's default search function
+kotti.session_factory                  Component used for sessions
+kotti.templates.api                    Override ``api`` object available in templates
+kotti.time_format                      Time format to use, default: ``medium``
+kotti.url_normalizer                   Component used for url normalization
+kotti.zcml_includes                    List of packages to include the ZCML from
+mail.host                              Email host to send from
+pyramid.default_locale_name            Set the user interface language, default ``en``
+pyramid.includes                       List of Python configuration hooks
+=====================================  =========================================
 
 kotti.secret and kotti.secret2
 ------------------------------

kotti/__init__.py

@@ -86,6 +86,7 @@ def none_factory(**kwargs):  # pragma: no cover
     'kotti.date_format': 'medium',
     'kotti.datetime_format': 'medium',
     'kotti.depot_mountpoint': '/depot',
+    'kotti.depot_replace_wsgi_file_wrapper': False,
     'kotti.depot.0.backend': 'kotti.filedepot.DBFileStorage',
     'kotti.depot.0.name': 'dbfiles',
     'kotti.fanstatic.edit_needed': 'kotti.fanstatic.edit_needed',

kotti/filedepot.py

@@ -414,7 +414,8 @@ def __init__(self, f, request, disposition='attachment',
             content_encoding=content_encoding)
 
         app_iter = None
-        if request is not None:
+        if request is not None and \
+                not get_settings()['kotti.depot_replace_wsgi_file_wrapper']:
             environ = request.environ
             if 'wsgi.file_wrapper' in environ:
                 app_iter = environ['wsgi.file_wrapper'](f, _BLOCK_SIZE)
@@ -494,9 +495,6 @@ def __init__(self, handler, registry):
         self.handler = handler
         self.registry = registry
 
-        self.cache_max_age = 3600 * 24 * 7
-        self.replace_wsgi_filewrapper = True
-
         DepotManager.set_middleware(self)
 
     def url_for(self, path):

kotti/tests/test_filedepot.py

@@ -99,6 +99,15 @@ def test_create(self, db_session):
         fs = db_session.query(DBStoredFile).filter_by(file_id=file_id).one()
         assert fs.data == "content here"
 
+    def test_list(self):
+        with pytest.raises(NotImplementedError):
+            DBFileStorage().list()
+
+    def test_exists(self, db_session):
+        assert DBFileStorage().exists("1") == False
+        file_id = self.make_one()
+        assert DBFileStorage().exists(file_id) == True
+
     def test_get(self, db_session):
         with pytest.raises(IOError):
             DBFileStorage().get("1")
@@ -275,3 +284,11 @@ def test_tween(self, webtest, filedepot, root, image_asset, db_session):
         assert resp.etag is not None
         assert resp.cache_control.max_age == 604800
         assert resp.body.startswith('\x89PNG')
+
+        # test 404
+        resp = webtest.app.get('/depot/non_existing/fileid', status=404)
+        assert resp.status_code == 404
+
+        resp = webtest.app.get(img.data['thumb_256x256_url'] + 'not',
+                               status=404)
+        assert resp.status_code == 404