Skip to content

Commit

Permalink
[fc] Repository: plone.app.theming
Browse files Browse the repository at this point in the history 
Branch: refs/heads/master
Date: 2021-09-09T16:52:44+02:00
Author: Peter Mathis (petschki) <peter.mathis@kombinat.at>
Commit: plone/plone.app.theming@4f1fb0d

Fix unclosed file

fixes #199

Files changed:
A news/199.bugfix
M src/plone/app/theming/utils.py
Repository: plone.app.theming

Branch: refs/heads/master
Date: 2021-09-09T18:04:36+02:00
Author: Jens W. Klein (jensens) <jk@kleinundpartner.at>
Commit: plone/plone.app.theming@28d4365

Merge pull request #200 from plone/issue-199

Fix unclosed file

Files changed:
A news/199.bugfix
M src/plone/app/theming/utils.py
  • Loading branch information
@jensens
jensens committed Sep 9, 2021
1 parent ed842a5 commit 6527cc0
Showing 1 changed file with 17 additions and 38 deletions.
55 changes: 17 additions & 38 deletions last_commit.txt
View file
Original file line number Diff line number Diff line change
@@ -1,57 +1,36 @@
Repository: plone.formwidget.namedfile
Repository: plone.app.theming


Branch: refs/heads/master
Date: 2021-08-23T17:03:40+02:00
Author: Harald Friessnegger (frisi) <friessnegger@lovelysystems.com>
Commit: https://github.com/plone/plone.formwidget.namedfile/commit/72db277e2ac5c54dcf82a04a034b1ea9560a6463
Date: 2021-09-09T16:52:44+02:00
Author: Peter Mathis (petschki) <peter.mathis@kombinat.at>
Commit: https://github.com/plone/plone.app.theming/commit/4f1fb0d44eb36a9a89d90c002d8b53df36be54fb

fix storage.cleanup()
Fix unclosed file

by not removing items while iterating over the dictionary
fixes #199

Files changed:
A news/52.bugfix
M plone/formwidget/namedfile/utils.py
A news/199.bugfix
M src/plone/app/theming/utils.py

b'diff --git a/news/52.bugfix b/news/52.bugfix\nnew file mode 100644\nindex 0000000..e8e7658\n--- /dev/null\n+++ b/news/52.bugfix\n@@ -0,0 +1 @@\n+Fix issue where expired temporary uploads have not been removed when calling FileUploadTemporaryStorage.cleanup() (https://github.com/plone/plone.formwidget.namedfile/issues/52) [fRiSi]\n\\ No newline at end of file\ndiff --git a/plone/formwidget/namedfile/utils.py b/plone/formwidget/namedfile/utils.py\nindex 1210ce1..de423e9 100644\n--- a/plone/formwidget/namedfile/utils.py\n+++ b/plone/formwidget/namedfile/utils.py\n@@ -43,12 +43,10 @@ def upload_map(self):\n def cleanup(self):\n """Remove obsolete temporary uploads."""\n upload_map = self.upload_map\n- for key, val in upload_map.items():\n- if (\n- val.get("dt", FALLBACK_DATE)\n- < (datetime.now() - timedelta(seconds=FILE_UPLOAD_EXPIRATION_TIME))\n- and randint(0, 5) == 0\n- ): # Avoid conflict errors by deleting only every fifth time # noqa\n+ expiration_limit = datetime.now() - timedelta(seconds=FILE_UPLOAD_EXPIRATION_TIME)\n+ for key in list(upload_map.keys()):\n+ dt = upload_map[key].get(\'dt\', FALLBACK_DATE)\n+ if dt < expiration_limit and randint(0, 5) == 0: # Avoid conflict errors by deleting only every fifth time # noqa\n # Delete expired files or files without timestamp\n del upload_map[key]\n \n'
b'diff --git a/news/199.bugfix b/news/199.bugfix\nnew file mode 100644\nindex 0000000..052c088\n--- /dev/null\n+++ b/news/199.bugfix\n@@ -0,0 +1,2 @@\n+Fix unclosed file when reading manifest.cfg\n+[petschki]\ndiff --git a/src/plone/app/theming/utils.py b/src/plone/app/theming/utils.py\nindex f3ca223..e494e24 100644\n--- a/src/plone/app/theming/utils.py\n+++ b/src/plone/app/theming/utils.py\n@@ -471,9 +471,8 @@ def getThemeFromResourceDirectory(resourceDirectory):\n """\n name = resourceDirectory.__name__\n if resourceDirectory.isFile(MANIFEST_FILENAME):\n- manifest = getManifest(\n- resourceDirectory.openFile(MANIFEST_FILENAME), MANIFEST_FORMAT\n- )\n+ with resourceDirectory.openFile(MANIFEST_FILENAME) as manifest_fp:\n+ manifest = getManifest(manifest_fp, MANIFEST_FORMAT)\n else:\n manifest = {}\n \n'

Repository: plone.formwidget.namedfile
Repository: plone.app.theming


Branch: refs/heads/master
Date: 2021-08-23T17:05:54+02:00
Author: Harald Friessnegger (frisi) <friessnegger@lovelysystems.com>
Commit: https://github.com/plone/plone.formwidget.namedfile/commit/c55667f3f025d2ba65f831d2e93f4fc7f1a477ba

make cleanup interval patchable

reduce risk of conflict errors and allow to force cleanup

fixes #54

Files changed:
A news/54.bugfix
M plone/formwidget/namedfile/utils.py

b'diff --git a/news/54.bugfix b/news/54.bugfix\nnew file mode 100644\nindex 0000000..bb9d919\n--- /dev/null\n+++ b/news/54.bugfix\n@@ -0,0 +1 @@\n+Make CLEANUP_INTERVAL patchable and fix condition (https://github.com/plone/plone.formwidget.namedfile/issues/54)\n\\ No newline at end of file\ndiff --git a/plone/formwidget/namedfile/utils.py b/plone/formwidget/namedfile/utils.py\nindex de423e9..f3a3643 100644\n--- a/plone/formwidget/namedfile/utils.py\n+++ b/plone/formwidget/namedfile/utils.py\n@@ -17,7 +17,7 @@\n FILE_UPLOAD_MAP_KEY = "file_upload_map"\n FILE_UPLOAD_EXPIRATION_TIME = 30 * 60 # seconds\n FALLBACK_DATE = datetime(2000, 2, 2)\n-\n+CLEANUP_INTERVAL = 5\n \n def is_file_upload(item):\n """Check if ``item`` is a file upload."""\n@@ -40,13 +40,19 @@ def upload_map(self):\n upload_map = annotations.setdefault(FILE_UPLOAD_MAP_KEY, OOBTree())\n return upload_map\n \n- def cleanup(self):\n- """Remove obsolete temporary uploads."""\n+ def cleanup(self, force=False):\n+ """Remove obsolete temporary uploads.\n+\n+ To avoid conflict errors, files are deleted on every ~5th method call.\n+ Use force to make sure all expired files are deleted\n+ """\n upload_map = self.upload_map\n expiration_limit = datetime.now() - timedelta(seconds=FILE_UPLOAD_EXPIRATION_TIME)\n+ # Avoid conflict errors by deleting only every fifth time\n+ delete = force or randint(1, CLEANUP_INTERVAL) == 1\n for key in list(upload_map.keys()):\n dt = upload_map[key].get(\'dt\', FALLBACK_DATE)\n- if dt < expiration_limit and randint(0, 5) == 0: # Avoid conflict errors by deleting only every fifth time # noqa\n+ if dt < expiration_limit and delete:\n # Delete expired files or files without timestamp\n del upload_map[key]\n \n'

Repository: plone.formwidget.namedfile


Branch: refs/heads/master
Date: 2021-09-09T13:50:50+02:00
Date: 2021-09-09T18:04:36+02:00
Author: Jens W. Klein (jensens) <jk@kleinundpartner.at>
Commit: https://github.com/plone/plone.formwidget.namedfile/commit/83c24e97f1ed8e23069090299350badf052028ff
Commit: https://github.com/plone/plone.app.theming/commit/28d43655caeef6e4c9e4e7c154bd5d98a567ae15

Merge pull request #55 from plone/fix/temp-storage-master
Merge pull request #200 from plone/issue-199

fix storage.cleanup() for master
Fix unclosed file

Files changed:
A news/52.bugfix
A news/54.bugfix
M plone/formwidget/namedfile/utils.py
A news/199.bugfix
M src/plone/app/theming/utils.py

b'diff --git a/news/52.bugfix b/news/52.bugfix\nnew file mode 100644\nindex 0000000..e8e7658\n--- /dev/null\n+++ b/news/52.bugfix\n@@ -0,0 +1 @@\n+Fix issue where expired temporary uploads have not been removed when calling FileUploadTemporaryStorage.cleanup() (https://github.com/plone/plone.formwidget.namedfile/issues/52) [fRiSi]\n\\ No newline at end of file\ndiff --git a/news/54.bugfix b/news/54.bugfix\nnew file mode 100644\nindex 0000000..bb9d919\n--- /dev/null\n+++ b/news/54.bugfix\n@@ -0,0 +1 @@\n+Make CLEANUP_INTERVAL patchable and fix condition (https://github.com/plone/plone.formwidget.namedfile/issues/54)\n\\ No newline at end of file\ndiff --git a/plone/formwidget/namedfile/utils.py b/plone/formwidget/namedfile/utils.py\nindex 1210ce1..f3a3643 100644\n--- a/plone/formwidget/namedfile/utils.py\n+++ b/plone/formwidget/namedfile/utils.py\n@@ -17,7 +17,7 @@\n FILE_UPLOAD_MAP_KEY = "file_upload_map"\n FILE_UPLOAD_EXPIRATION_TIME = 30 * 60 # seconds\n FALLBACK_DATE = datetime(2000, 2, 2)\n-\n+CLEANUP_INTERVAL = 5\n \n def is_file_upload(item):\n """Check if ``item`` is a file upload."""\n@@ -40,15 +40,19 @@ def upload_map(self):\n upload_map = annotations.setdefault(FILE_UPLOAD_MAP_KEY, OOBTree())\n return upload_map\n \n- def cleanup(self):\n- """Remove obsolete temporary uploads."""\n+ def cleanup(self, force=False):\n+ """Remove obsolete temporary uploads.\n+\n+ To avoid conflict errors, files are deleted on every ~5th method call.\n+ Use force to make sure all expired files are deleted\n+ """\n upload_map = self.upload_map\n- for key, val in upload_map.items():\n- if (\n- val.get("dt", FALLBACK_DATE)\n- < (datetime.now() - timedelta(seconds=FILE_UPLOAD_EXPIRATION_TIME))\n- and randint(0, 5) == 0\n- ): # Avoid conflict errors by deleting only every fifth time # noqa\n+ expiration_limit = datetime.now() - timedelta(seconds=FILE_UPLOAD_EXPIRATION_TIME)\n+ # Avoid conflict errors by deleting only every fifth time\n+ delete = force or randint(1, CLEANUP_INTERVAL) == 1\n+ for key in list(upload_map.keys()):\n+ dt = upload_map[key].get(\'dt\', FALLBACK_DATE)\n+ if dt < expiration_limit and delete:\n # Delete expired files or files without timestamp\n del upload_map[key]\n \n'
b'diff --git a/news/199.bugfix b/news/199.bugfix\nnew file mode 100644\nindex 0000000..052c088\n--- /dev/null\n+++ b/news/199.bugfix\n@@ -0,0 +1,2 @@\n+Fix unclosed file when reading manifest.cfg\n+[petschki]\ndiff --git a/src/plone/app/theming/utils.py b/src/plone/app/theming/utils.py\nindex f3ca223..e494e24 100644\n--- a/src/plone/app/theming/utils.py\n+++ b/src/plone/app/theming/utils.py\n@@ -471,9 +471,8 @@ def getThemeFromResourceDirectory(resourceDirectory):\n """\n name = resourceDirectory.__name__\n if resourceDirectory.isFile(MANIFEST_FILENAME):\n- manifest = getManifest(\n- resourceDirectory.openFile(MANIFEST_FILENAME), MANIFEST_FORMAT\n- )\n+ with resourceDirectory.openFile(MANIFEST_FILENAME) as manifest_fp:\n+ manifest = getManifest(manifest_fp, MANIFEST_FORMAT)\n else:\n manifest = {}\n \n'

0 comments on commit 6527cc0

Please sign in to comment.