Dynamic default action and quality for thumbnail information #813
Comments
Isn't this covered by the Thumbnailer configuration? I agree that we need a better way to enter these configurations. https://docs.directus.io/guides/thumbnailer.html#thumbnail-actions |
I'm not really following regarding the "Default thumbnail action". Doesn't it allow you to get any source based on the URL structure of the file (where the thumbnail settings act as a whitelist/blacklist)? |
I am talking about the thumbnails array that is provided in an image.data.thumbnails. That array is generated by the "get_thumbnails" function ( https://github.com/directus/api/blob/master/src/helpers/file.php#L181 ) and only fetches the thumbnail dimensions from settings and generate a thumbnail URL with action crop and quality better. Doesnt matter what you configure. |
Ahhh gotcha |
As I am fairly new to directus I would be interested to know if this array is considered to be a good place to start in regards to generating thumbnail URLs. Sure, I could get the filename and build the thumbnail URL by myself but it feels more error resistant and maintainable to use that thumbnail array. |
I'm not entirely sure what a good solution would be here. If that array would return all the possible options, we'd end up with tens or even hundreds of links 🤔 The path structure of the thumbnail stays the same, so maybe we can return something like {
"sizes": ["200x200", "250x300"],
"qualities": ["good", "poor"],
"fit": ["contain"]
} and you could use that information to dynamically generate the URL? |
It seems the the ideal solution is to create a custom interface for managing this array. Then we can have a proper GUI for editing the config and still save things in the current format. |
I think you're confused @benhaynes. We're discussing the output of the API. Right now, the API returns an array of URLs to the various thumbnails, however its incomplete: It will return urls for the different sizes, but not the different qualities. |
Yup! Haha... didn't pay attention to the repo. The thumbnail whitelist should set each set of params, which it might not be doing now. That would give explicit control over what's allowed and avoid the issue of too many combinations being returned. We use a named quality tag, but could switch to a percentage based one to keep things easier. So a whitelisted thumb value would look like:
Then it would be super easy to return all possible thumbs since each is whitelisted. Am I on the right track now? |
That sounds like a proper solution to me 👍 |
To keep this a CSV and since this is already for URLs, this seems cleaner:
We could also have quality last so it's separate from the dimensions and maybe even optional (need to have a single default, like
|
To achieve better clarity/visibility, we are now tracking feature requests within the Feature Request project board. This issue being closed does not mean it's not being considered. |
Just for the suggestion. Can we use the This gives more flexibility for adding additional parameters without making any major change. The above syntax is used by many image delivery services like Cloudinary, ImageKit For more info |
Random thought of the day: should we use a POST request instead and rely on JSON for all the options? |
There are two things to consider here, how we save the settings for this and how we access thumbnails in the URL. Ideally both are the same. I like the idea of your syntax recommendation @hemratna — but I also don't want to make things overly complex if we can avoid it. Things like zoom, x, y, format, etc are all good options, but would we completely change the way we store thumbnail files? Right now we have a set directory architecture based on the width, height, action, and quality. With the way I recommended we can have a CSV or whitelisted configurations... How would we store multiple whitelisted thumbnail configs with this? Use a different character? Here is another way of doing it: And another with query strings: |
If you are suggesting let's create a |
Store thumbnailStore all the thumbnail under one directory
Store configurationStore multiple whiltelisted thumbnail configs as below.
Simply we are looking for 2 different delimiters here. Pros of this approach is
|
Thumbnail StorageStoring all thumbnails in one directory might have issues when we support folders in the future. When we support folders, you could potentially have files with the same name in different folders:
Note: Our current solution doesn't work for this either. This also changes the original file name, which might not be a huge deal, but it's nice that as of now we serve thumbnails with the same name as the original. It also makes it more difficult to "clear" or remove all of a specific thumbnail type (you'd really need a script to do this). Whereas now you can delete a sub-directory to remove a specific thumbnail group. Not a big deal, but worth noting. Just wondering, couldn't we make directories with the whitelisted params and then keep the original filename within it? For example: Thumbnail ConfigThe main reason for the whitelisting is to avoid potential file upload vulnerabilities where malicious users can generate many different thumbnails on the server simply by requesting different permutations. So we'd need to really understand how the Thumbnailer works if fewer config params are set. For example, if we whitelist I think one of these below might work (@rijkvanzanten ?), but depending on which we choose we might need to update the tags interface (the one that will manage this Setting) to allow for custom delimiters (a bar in this case).
Lastly, I'm not sure about the "named" quality params — since those would need to be defined/set somewhere. I think a 1-100 quality is more foolproof, and we could make this optional with a single global quality default (eg: |
Full disclosure: haven't read everything thoroughly / studied it yet I like the flexibility, but
reads like an alien language. Docs are going to be absolutely crucial |
I agree. That's why it took me hours to write this... I'm just not happy with how this looks raw. We'll be using the
But still, I agree that this will need very clear docs. |
I've only skimmed this thread, there's a lot of discussion haha. How about considering them like CSS properties? So it "sort of" follows a standard, with the added bonus, that imo, the following feels less "alien" |
It does "break" with 'normal' URL key-value pairs though eg That could also be an option though, query parameters:
|
What different flags do we have @hemratna? Maybe
|
First we should decide if we want "wildcards" with
I personally think this leaves us open to the same issues where malicious users can flood the system with unwanted files... but I wanted to discuss before totally dismissing it. If we're planning on allowing wildcards then we should have nested directories (similar to what we have now):
Or, if we will NOT allow wildcards, we can group them into single folders:
|
@rijkvanzanten — one thing we're considering here is that there may be other operations/filters added in the future (thus moving to JSON). So we should make sure the folder structure allows for that too. https://docs.cloudimage.io/go/cloudimage-documentation/en/operations |
Additionally, I am also thinking like all fields are not required or each field is individually supported too. There are some possibilities like below.
P.S. Optional fields may be part of future enhancement. But I want to make sure we don't have to make major changes in the future. |
Wildcards are very dangerous yet very useful. I suppose once we have authentication for thumbnails/files it's less of a problem though |
Agree. I guess as long as our defaults don't include wildcards, and there's a way to I think our only default will be the main one used in File Library:
So we need to make sure that this only allows for ONE thumbnail, and not hundreds by exploiting the undefined parameters (now or in the future). For example: the crop positioning, filters, etc. Maybe we can add a |
We make all the four arguments In the future, we may support optional arguments like |
Perfect. I guess we can figure out how to handle Anything else we need to figure out for this one? |
All set for now. 💪 |
1 final thing to consider: is this a breaking change? |
If the directory structure is the same, and potentially the URL structure is the same (depends if we want to add the key names), all were changing is how we store the settings... which we should be able to write migration scripts for. |
Okay so let me conclude this conversation here. we will use the old URL only, and will not pass any query string or will not update the directory name. Am I right? If we are going with this flow then below are some points which we should think once before this change.
So let's move to the summing-up;
Thus for the development of this functionality, we should think about the logic of migration. |
Thank you Binal. A few questions: Creating perfect migrations for this seems impossible, since we're trying to combine different values into one combined value (too many permutations). If you had the following settings:
What if you also wanted a thumbnail with a different Problem & RequirementsTaking a step back, I wonder if we should keep the whitelist configuration more simple... and allow for it to also be more permissive/flexible (if desired). Here is what we want/need:
Possible SolutionFour thumbnail settings. And for those legacy users migrating, we can just turn on "allow all thumbnails" so nothing breaks in their system (no migrations needed).
|
Hey @benhaynes , After checking your suggestion it seems like we are removing 3 variable from settings and adding new 4. So dont you think so we are doing something more difficult? |
The The main reason for this switch is to allow proper limiting of thumbnails, or generation of any (if needed). These new settings do that but the old ones don't. Thoughts? |
Yeah, agree. So can I start the development? As all the doubts are clear now. |
Great! Let's get the App a bit more stable first (since this is a new feature). Hopefully we can come back to this soon and get it implemented. |
Hi, any updates on this? I ask because I'm in an urgent need of a solution to #1080 which appears to be dependent on this (I'm surprised other users aren't complaining about #1080 since for me it seems a rather serious impairment to the overall utility of file manager) Is there anything I or others can do to work around this, or help bring about a solution? |
No updates, but anyone can either submit a PR or sponsor the work to speed up the development. Our team has been super busy with client projects, but hopefully we'll get to this in the next month or so. |
Example Asset Data
Original Absolute Path
Generated Asset Path
Original/Generated Asset Middleware
Middleware would return with original |
We can also add a name per whitelist name: before:
after:
This would also give us a solution for the folder of the Generated Asset Path: before:
after:
|
Just adding that only
And below is an example of a more complex thumbnail whitelist:
|
Files can be 2 types.
For the private files; one secret will be passed as a query string. The secret string will contain the expiry time, the user token (For whom this file will belong). Though we can consider it as a feature enhancement. |
Let me brief this whole flow again. Please review it and let me know if anything is missing. JSON Format
Here, URL structure The
Though, the storage will DB Variables
I didn't add Folder Structure
Thus, the Question🤔 Will the other [ |
Some updates after discussing internally:
|
Feature Request
Make action and quality used in generating thumbnail information in file helper configurable or dynamic in some way.
What problem does this feature solve?
Currently the thumbnail information added to the image data is by default "crop" with quality "good".
See:
https://github.com/directus/api/blob/master/src/helpers/file.php#L210
How do you think this should be implemented?
A simple approach would be to add setting options "Default thumbnail action" same for quality. That could be used as a 4th/5th parameter in the file helper.
A more sophisticated solution would be to read thumbnail settings and create thumbnails for all configured thumbnail actions and qualities. But maybe the current default would result in too many entries ( 2 actions * 4 qualities per size ).
Would you be willing to work on this?
Don't know if this would be a good place for a first contribution but I would be willing to give it a shot.
The text was updated successfully, but these errors were encountered: