New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Versioning creates zillions of empty blob files #532

Closed
hvelarde opened this Issue Jun 18, 2015 · 7 comments

Comments

Projects
None yet
5 participants
@hvelarde
Member

hvelarde commented Jun 18, 2015

When using plone.app.iterate, each version of a content object with persistent tiles with blobs, will create a new empty blob into file system for each found blob and I will also break the CMFEditions Portal ZVC based Histories Storage.

If you're using this feature in your site you have to take special attention to the number of free inodes in your file system, as you can run out of them; use the df -i command to check it:

$ df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
rootfs                    10485760 8929045 1556715   86% /
udev                       2057472     272 2057200    1% /dev
tmpfs                      2058962     204 2058758    1% /run
/dev/disk/by-label/DOROOT 10485760 8929045 1556715   86% /
tmpfs                      2058962       1 2058961    1% /run/lock
tmpfs                      2058962       5 2058957    1% /run/shm

@hvelarde hvelarde added the bug label Jun 18, 2015

@hvelarde

This comment has been minimized.

Member

hvelarde commented Jun 18, 2015

This is related to plone/plone.app.mosaic#80

@hvelarde

This comment has been minimized.

Member

hvelarde commented Jun 18, 2015

This is also related to plone/Products.CMFEditions#28

hvelarde added a commit that referenced this issue Jun 19, 2015

@jpgimenez jpgimenez self-assigned this Sep 9, 2015

jpgimenez added a commit that referenced this issue Sep 17, 2015

@runyaga

This comment has been minimized.

runyaga commented Jan 14, 2016

@hvelarde does this commit fix the zillion of files? dont see any activty post 9/17 commit

@hvelarde

This comment has been minimized.

Member

hvelarde commented Jan 14, 2016

@runyaga I don't know, but I don't want to fix/patch that here because then we will have to maintain it.

according to @datakurre the actual issue is different: deepcopying a blob results on a new blob with zero data and this is what CMFEditions does by default; check the issues referenced above.

please keep communication channels open before working on issues to avoid rework.

BTW, happy new year!

@datakurre

This comment has been minimized.

Contributor

datakurre commented Jan 14, 2016

In short: When attribute is not handled by a dedicated plugin in CMFEditions, it gets deep copied using pickle + unpickle. By Googling around I've understood that it's a "known" limitation of ZODB that cloning blobs that way results in empty blobs. Plone only has support for versioning blobs defined in Archetypes or Dexterity (or its behaviors') schemas.

Unfortunately tiles are designed in a way that it's not trivial to figure out tile's schema from its annotation object. To fix this, we need some more generic way version blobs in annotation (not cause conflict with behaviors using annotation storage while doing that)..

@fredvd

This comment has been minimized.

Member

fredvd commented Feb 10, 2017

@hvelarde I'm about to upgrade a Plone 4 site with an older collective.cover where this might be an issue, I have a blobstorage with 2/3 of the 15000 files being size zero.

The only weird thing is that plone.app.iterate was for some reason installed, but never available to end users (for some reason the actions were never available because the condition check fails).

For me and others who find this issue later, did you do anything to clean up the many empty blob files in your site after you upgraded cover with the plone.app.tiles 1.1.0 dependency that solves creating new blob files?

@hvelarde

This comment has been minimized.

Member

hvelarde commented Feb 10, 2017

@fredvd you have to pack the database; BTW, for some reason I still have empty files on the sites that had this problem; is not the same high number, but this is still an issue.

I highly recommend to install collective.revisionmanager and run it from time to time to get rid of orphan versions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment