Huge disk-cache directories #9870
Comments
|
@HeavyHorst , will discuss internally and get back to you on making this configurable. |
|
Thank you! That would be great. |
|
Is there an easy way to disable range-caching in the source code for testing? |
poornas
pushed a commit
to poornas/minio
that referenced
this issue
Jun 24, 2020
The default behavior is to cache each range requested to cache drive. Add a environment variable `MINIO_RANGE_CACHE` - when set to off, it disables range caching and instead downloads entire object in the background. Fixes minio#9870
7 tasks
poornas
pushed a commit
to poornas/minio
that referenced
this issue
Jun 24, 2020
The default behavior is to cache each range requested to cache drive. Add a environment variable `MINIO_RANGE_CACHE` - when set to off, it disables range caching and instead downloads entire object in the background. Fixes minio#9870
@HeavyHorst , you can use the patch referenced here. |
|
Works great by the way! |
poornas
pushed a commit
to poornas/minio
that referenced
this issue
Jun 27, 2020
The default behavior is to cache each range requested to cache drive. Add a environment variable `MINIO_RANGE_CACHE` - when set to off, it disables range caching and instead downloads entire object in the background. Fixes minio#9870
poornas
pushed a commit
to poornas/minio
that referenced
this issue
Jun 29, 2020
The default behavior is to cache each range requested to cache drive. Add a environment variable `MINIO_RANGE_CACHE` - when set to off, it disables range caching and instead downloads entire object in the background. Fixes minio#9870
harshavardhana
pushed a commit
that referenced
this issue
Jun 29, 2020
The default behavior is to cache each range requested to cache drive. Add an environment variable `MINIO_RANGE_CACHE` - when set to off, it disables range caching and instead downloads entire object in the background. Fixes #9870
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Is your feature request related to a problem? Please describe.
We currently host about 70TB of media in Amazon S3 and and are trying to use minio in gateway mode to save some bandwith costs.
The problem is that every http-range is cached again, which leads to constantly high traffic from s3 and requires huge amounts of cache storage (people seek around in videos all the time).
For some files I end up with massive cache-directories of 350GB+ for a single 1.6GB video file.
Here is a cache.json file of such a case:
cache.txt
Describe the solution you'd like
I would like to be able to simply cache the complete files.
I think that would be clearly more efficient in our case (only 1.6GB transfer from amazon instead of 350GB) for just this one file.
The text was updated successfully, but these errors were encountered: