PFFile Directories Need Clean Up

[joshbeal]

Users of my app have complained about it taking up too much space on disk. Inspecting the app bundle reveals two large directories maintained by Parse.

• Private Documents / Parse / PFFileStaging (600+ MB)
• Caches / Parse / PFFileCache (40+ MB)

I find it odd that Parse is not cleaning those up. I also don't understand why it is loading files in PFFileStaging at all since I'm using a regular networking library to load the files from URL.

I've observed these issues on V1.7.4 but I have no reason yet to believe they've been fixed in later releases. Any thoughts?

[richardjrossiii]

PFFileStaging is done so that we can ensure that no concurrent modification happens to the file after you request uploading. However, we should be moving, after upload, all files from PFFileStaging to PFFileCache, unless the upload is interrupted/fails. See PFFileController.m.

Can we get a small repro case?

[nlutsenko]

What I think should happen is:

• we should have an explicit way to cleanup all the caches for PFFiles (aka delete Caches/Parse/PFFileCache)
• automatically trim the staging directory when the file isn't references anymore, like after an app restart.

[richardjrossiii]

Agreed on clearing the staging directory upon parse initialization. Not so sure a public API for clearing the cache is useful, but we can add it.

[joshbeal]

+1 for automatically trimming the staging directory. it would be nice to have that done in background to avoid blocking app launch.

[nlutsenko]

Hey guys, a quick update... we are tracking this for version 1.8.2 and will get there very soon.

Our current plan is to implement automatic nuking for Private Documents/PFFileStaging directory on every app restart. That directory is not automatically cleared by the system, so this is quite relevant to do.

In regards to File Cache under Caches/Parse/PFFileCache - that one is guaranteed to be cleared automatically by the system whenever the device is low on disk space, so manually clearing cache there is not necessary.

[xjirman]

My app-in-development had 2.4GB of data in PFFileStaging which led me here. I just updated the app with 1.8.2 Parse SDK version, cleaned build and relaunched the app. I was expecting this directory to be wiped clean but unfortunately nothing happened. Are there any specific steps to take or is this process fully transparent to the developer?

[nlutsenko]

The first usage of PFFile APIs - will clear the directory for you, since it's lazily loaded.
Taking into account that applications that have a lot of files in PFFileStaging will utilize PFFile APIs extensively - FileStagingController and clearing of that cache is made lazily when you access PFFile first time after relaunch of the application. Hope it helps. :)

[xjirman]

Awesome! Worked like a charm, thank you!

[MobileVet]

Please reopen this issue:
We are using 1.10.0 and still see both of these folders grow without bounds. The Private Documents folder is NOT nuked on every restart of the application.

Additionally, I have n=20 of PFFileCache NOT being deleted by the system when low on disk space. This where low = < 200MB of space.

Bottom line, I think we / Parse needs to assume the worst and manage these directories accordingly. Our application runs for months on end and can build up GBs of data in the current incarnation.

As an aside, I wonder if part of the reason we have trouble is bad network. Would files be left around if the uploading failed?

[nlutsenko]

There was a slight change - PFFileStaging under Private Documents is unused at this time at all, so it's safe to assume for you that it's safe to nuke it if you are using 1.11.0+ version of the SDK.
Both staged and cached files go to app caches (which is managed by the system and should be evicted if you are running low on space) and we have an enhancement issue #488 to give you an ability to clear that cache away manually.

The only thing that comes to my mind as a workaround for now - you can safely delete the cache yourself by using something like this on every app launch before you interact with file APIs on Parse.
Please also keep in mind, this example clears away query cache (if you are not using Local Datastore):

    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSCachesDirectory, NSUserDomainMask, YES);
    NSString *folderPath = [paths.firstObject stringByAppendingPathComponent:@"Parse"];
    [[NSFileManager defaultManager] removeItemAtPath:folderPath error:nil];

[nlutsenko]

@MobileVet, I've also just realized that taking into account that you have an app running for months non-stop - the system indeed doesn't get a chance to remove this cache.

Keep an eye open on #488, and if you have an idea for an API that would work across platforms - that would be amazing.

[MobileVet]

@nlutsenko thanks for the note. Actually, I misspoke a bit. The app should be able to run for months but we shut it down each night from 3am - 8am. Despite this, we aren't seeing these caches get cleared. I am not sure if it related but we shut the app down programmatically with an exit(0).

[parse-github-assistant]

The label type:feature cannot be used in combination with type:improvement.