clean-temp on deleting too much?

[igorescobar]

Yesterday I decided to give a go on a flag called clean-temp on. My cache was around 50+ GiB and everything was going fine.

This morning the cache was half of it (!) and the number of cache MISS increased a lot. Before only like 10% of the request was actually a MISS and now only ~60-65% is a MISS.

I'm using the latest stable docker image.

[jiangwenyuan]

@igorescobar If you have multiple instances pointing to same directory, you should avoid use clean-temp.

If one instance is creating cache(writing to temp dir), and if another instance starts, it will delete all temp files if clean-temp is on. So the creating by previous instance will fail.

But it has nothing to do with real cache files and requests, as it only happens on start and will stop before receive requests.

Previous versions delete temp files on start, I just add an option to disable that. As I did not expect the use cases like yours: multiple instances pointing to same disk directory.

[igorescobar]

Is there a way to disable the usage of these temp files?

Nginx has this use_temp_path=on|off that basically work around temp file issue but I'm not sure how they implemented it.

I hope I'm not annoying you by mentioning Nginx all the time. I'm just using it as a reference so you can compare it with some other tool that does something similar. 😅

[jiangwenyuan]

Not at all:)

Nginx uses temp files also, use_temp_path on|off just controls where nginx puts the temp files, the one defined by proxy_temp_path or the default one.

[igorescobar]

NGINX first writes files that are destined for the cache to a temporary storage area, and the use_temp_path=off directive instructs NGINX to write them to the same directories where they will be cached. We recommend that you set this parameter to off to avoid unnecessary copying of data between file systems. use_temp_path was introduced in NGINX version 1.7.10 and NGINX Plus R6.

[igorescobar]

So, If I got it right... what this does is... instead of saving a temp file somewhere and then move it afterwards to the actual cache directory it will save the temp file directly on the actual official destination.

Would this avoid the creation of temp files that would be there forever if no one cleans it?

[jiangwenyuan]

No, nginx creates temp files anyway. If it's in same directory, the move is cheap. If the temp directory is on a different file system, the move is expensive(copy).

[igorescobar]

So, how do I get rid of these temp files running multiple containers? 🤔

[jiangwenyuan]

No need. If nuster quits normally, there's no temp file left. And if there are too many temp files, just delete that. And the temp files are harmless, you can leave them there as long as the disk space is not your concern. Besides, even if the temp files which are used by some instance are deleted, there's no harm. Next request will create that again.

So as of now, there's no need to worry about that.

[igorescobar]

Ok, So the recommendation is to live is off if you have this 1+ containers/instances pointing to the same volume. 👍

Thanks!