Cleaning up the cache

[reinink]

Currently Glide does not do any cache cleanup, meaning it generates manipulated images and places them into the cache file system, but it never deletes them. It's possible that over time this cache gets large and contains many unused image manipulations.

The simple solution here is to simply delete the entire cache periodically, and on small projects this is probably fine. But on larger projects it could be problematic since all image manipulations will have to be regenerated.

It would probably be good if Glide came with a CacheCleanup helper. This cleanup tool could be run every time a request is received, or it could be setup with a scheduler (probably better). Something like:

$cache = new Filesystem(new LocalAdapter('cache-folder'));
$cleanup = new CacheCleanup($cache)
$cleanup->expire(2592000); // 30 days

This approach would delete any manipulations older than 30 days. This can be determined easily use the $filesystem->getTimestamp() method.

However, this technique will not take into consideration whether they are being actively accessed or not. Using the last accessed date would be much more complicated since this information is not readily available. To do this you would need to create a data store, and save this information every time a manipulation is created. I feel this falls outside of the scope of this project. If this sort of cache cleanup is needed, developers are welcome to implement this themselves.

[h4cc]

If there are multiple ways of implementing the CacheCleanup, it could be integrated by using the Strategy Pattern.
Some default strategy like OlderThanCleanup or NoCleanup can then be provided by glide.

[reinink]

@h4cc That's probably a good idea actually.

I've thought of another way of doing this. Glide could keep an access log in the cache file system, which could be then used by a cleanup job later. This access log would provide access dates in addition to just creation dates.

[barryvdh]

@reinink and what about when the source image changes?

For example, I have a filebrowser where I can resize/rotate etc some files. But changes aren't shown because the file is cached.
A solution could be checking the time the file has changed and invalidate the cache if needed, but this causes extra calls..

[reinink]

I'm actually like the idea I'm seeing in PR #60.

Basically, we could allow folks to override how the cached filename is generated. This would allow you to group all manipulations for a specific source image into a specific folder. Then, if you delete that source folder, you can easily delete all the cached variations. Something like this:

$server->setCachePathFilter(function (Request $request, $path) {
    // modify $path
    return $path;
});

At this point I'm just not sure a closure is the best way (it seems a little complicated), or if we just abstract the cache path generation to a new class, with a interface.

Whatever the case, I think I'm going to do something here. I just need to work out the simplest way, and do some testing.

[reinink]

Or, another idea, why don't we just automatically organized the cached images into folders, and the folders will simply be the source name. For example:

// Source:
/source/kayaks.jpg
/source/logos/logo.png

// Cache:
/cache/kayaks.jpg/md5-cache-filename
/cache/kayaks.jpg/md5-cache-filename
/cache/kayaks.jpg/md5-cache-filename
/cache/logos/logo.png/md5-cache-filename
/cache/logos/logo.png/md5-cache-filename
/cache/logos/logo.png/md5-cache-filename

That would make it really easy to delete the cache for any particular source image. For example:

$cache->deleteDir('kayaks.jpg');

Thoughts?

[schmidex]

I like that ;-)
This storage structure and a delteCache($source) method on the Server class.

[ingro]

That would be great, a way to clear the cache only for a specific image it's the missing piece for us to start using Glide in production!

[reinink]

@schmidex @ingro Glad you like! Coming soon to the next release of Glide. Hopefully within the next few weeks.

[ADmad]

why don't we just automatically organized the cached images into folders, and the folders will simply be the source name.

👍 💯

[joshrhykerd]

I really like the idea of being able to clean up the cache files using folders by source name. Has there been any progress on this enhancement yet? Thanks for the great library!

[reinink]

@joshrhykerd Glad you like! And yes, this feature has been built and will be available as soon as I release version 1.0, which is really any day now. You can see this feature mentioned in the WIP docs, here.

[joshrhykerd]

I hate to ask, but there isn't any development branch or anything anywhere with this functionality already built in is there? I'm in a slight predicament now where I neeeeeed this ability. lol Again thanks again for this great library!

[reinink]

@joshrhykerd For sure, just use dev-master. In your Composer file:

{
    "require": {
        "league/glide": "dev-master"
    }
}

Note that you'll likely also want to use a framework adapter, to make getting responses back easier. I originally had these built right in, but I recent split them all into separate packages. Add the framework adapter of your choice in the same way:

{
    "require": {
        "league/glide-symfony": "dev-master"
    }
}

And then just follow the docs for how to setup the response factory. It will look something like this:

$server = League\Glide\ServerFactory::create([
    'response' => new League\Glide\Responses\SymfonyResponseFactory()
]);

Just be aware that you're using development code. Be sure to switch from dev-master to a tagged release as soon as one is available. Good luck!

[joshrhykerd]

@reinink Thanks!

[joshrhykerd]

@reinink I'm not sure if this is a bug or not, but when I call

$server->deleteCache($filename);

with just the source filename it doesn't seem to work. I have to also include the server cache_path_prefix.

$server->deleteCache($server->getCachePathPrefix() . DS . $filename);

Should it automatically be including the cache_path_prefix for me? or am I doing something wrong?

[reinink]

Hmm, maybe it's broken. It was working, but I still need to write tests for this. Honestly, for now you can just use Flysystem directly. For example:

$cache->deleteDir('path/to/directory/to/delete');

[reinink]

This feature should now be fixed. (5ededae)

[reinink]

Closing this ticket. These changes have been merged into the master branch and will be tagged shortly when I release v1.0! Hopefully not long now.

[argb]

@reinink Looking forward to the release of version 1.0,and when ? :D

[papalardo]

The method deleteCache not works with me.

I did need to do this way:

$server = app(GlideServer::class);

// $server->deleteCache($imagePath); // not works
// $server->deleteCache($server->getCachePathPrefix() . "/" . $imagePath); // not works
 
Storage::disk('images')->deleteDirectory($server->getCachePathPrefix() . "/" . $imagePath); // works