Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Advanced Image Cropping Using Focus Point (X%, Y%) #3240

Closed
jeremy-farrance opened this issue Jan 5, 2024 · 9 comments
Closed

Comments

@jeremy-farrance
Copy link
Sponsor

jeremy-farrance commented Jan 5, 2024

I'm submitting a

[x] feature request

...about

[x] edit experience / UI
[x] admin experience UI
[x] Content Types or data management
[x] APIs like REST
[x] other / unknown

Summary, in brief

Allow another image cropping option that takes two values, an X%, Y% to specify a target, and use the provided (enhanced or other) math function to create the 4 values needed to use ImageFlow's crop x1, y1, x2, y2 (rectangle).

Current Behavior

The fantastic progress on Image/Picture rendering is amazing, but still leaves edge cases that can be difficult for a normal editor-user to overcome. Image Cropping current supports "Corner or Side Crop":

image

Images come in all shapes and sizes and fairly often with large images, getting the right part of the image framed fails when you only have the ability to specify an edge/corner to crop to does not get you what you need. It also leaves out the concepts of scale (keeping aspect ration and zoom.

Though its quite a bit of code/math to handle this in a logical and user-intuitive way, its also a pretty high level of ROI (for the content managers).

Here is an older issue which covers part of this: #1692 Improvements on Image Handling.

Here is an example:

  1. Original (is an 8 mexapixel image)
    image

  2. In use with defaults, no cropping; the boy in the air has his head cut-off and you cannot see the driving board
    image

  3. Using existing Cropping set to Top-Center it loses context and framing is very poor
    image

  4. Desired result (not achievable in modern 2sxc v14-v17) is the red outlined area
    image

So my suggested feature addition is a 4th dropdown option for Crop Behavior, possibly named "Advanced Image Cropping." Prior to switch to using .Picture() and the related modern image output APIs in 2sxc (now using ImageFlow), I had implemented parts of this solution in custom situations and this is documented here with (possibly still useful code):

Advanced Image Cropping Using Focus Point (X%, Y%) << code sample

The above link simply sets a focus point or target, then gets the 4 corners to crop the image to without going outside the image. It retains aspect ratio, but does not account for Zoom. Note that the Desired Result (4 in the example above) would simply be to set Y to 42% and leave X at the default, 50%; nicely framing the mid-air kid and diving board while still cropping the image in a contextually pleasing way.

A slightly more advanced version would allow you specify 2 points which would be the upper-left and lower-right corner of the desired sub-image. This would be enough info to maintain aspect ratio and zoom+crop the image and then resize to the desired width.

I'll try to come back and re-read this in a day or two to spot and fill-in any gaps I've left. Thank you for your consideration!!

@jeremy-farrance
Copy link
Sponsor Author

jeremy-farrance commented Jan 5, 2024

I forgot to add. Prior to the modern Img() and Picture() methods, I would do this using ImageResizer building crazy URL params. But the modern APIs do not appear to allow me to get to or affect the params you are building for ImageFlow. So I checked and it does look like ImageFlow supports cropping the same as (similar to) ImageResizer did. And for fun, I generated my x1,y1 and x2,y2 and tried it, as you can see below, its VERY possible and not crazy hard... check this out!

&crop=0,16.825,100,66.667&cropxunits=100&cropyunits=100

Produces:
image

So I guess my backup question is, is there a way using .Picture() to allow me to add to the URL you are creating?

@skarpik
Copy link
Sponsor

skarpik commented Jan 5, 2024

@iJungleboy - the feature that @jeremy-farrance is recommending would be a extremely helpful. The addition of Image Cropping (default crop, corner or side crop, no cropping) to 2sxc was a big step forward in usability and flexibility. Adding the ability to crop images in the manner that @jeremy-farrance is suggesting would be particularly useful for users who don't regularly use graphics software. Even for me, it would simplify my workflow when assembling large webpages with a lot of images.

@iJungleboy
Copy link
Contributor

We've been wanting to do something like this for a while, but there are still a few problems.

Typically a user has a template which is responsible for cropping the image - and making it a size which is in line with the design. Eg. all images may need to be 16:9.

In this scenario, an crop should be smarter than just exact-crop. It would actually need to create a different crop which matches the expected proportions, but keeps the desired area.

You can see this failing here:

https://pdpr2023.accuraty.us/Portals/0/adam/Content/_Q9ekacF7UCnbDV3VXUeCg/Image/centennial-aquatic-center-boy-jumping-off-high-dive-0935.jpg?w=900&h=300&crop=0,8,100,66.667&cropxunits=100&cropyunits=100

Basically this mode will assume that's exactly the area you want to show, and will pad the rest.

Adding &mode=crop improves it a bit, but actually trims of more than the user probably wanted:

https://pdpr2023.accuraty.us/Portals/0/adam/Content/_Q9ekacF7UCnbDV3VXUeCg/Image/centennial-aquatic-center-boy-jumping-off-high-dive-0935.jpg?w=900&h=200&crop=0,8,100,66.667&cropxunits=100&cropyunits=100&mode=crop

I believe these are the current options: https://docs.imageflow.io/querystring/transforms.html

So I think we're kind of close to a solution, but not there yet.

I have two long running issue open on ImageFlow

maybe adding more requests / notes would help 😉

@jeremy-farrance
Copy link
Sponsor Author

jeremy-farrance commented Jan 10, 2024

@iJungleboy, could you take a closer look at the code I supplied; the link above to Adv Cropping w Focus? I believe I've already included the solution to the problem you are highlighting. The code (CalculateCropCoordinates()) does the work to keep the corners within the image edges. So your examples above, do not happen.

@iJungleboy
Copy link
Contributor

@jeremy-farrance I hadn't noticed your code, thanks for the hint.

I can't evaluate if it's complete, but in general I also believe that the logic would be fairly simple, 30-100 lines of code + ideally some tests to verify the numbers.

The problem is that this must be in the ImageFlow processing pipeline to work well and also be performant. So quite some heavy lifting unless you're very familiar with the ImageFlow code base.

So I'm sure that if you would write all the code in ImageFlow, it would either be included and/or it could serve as a plugin which we could bundle.

In the core team we don't have the resources ATM, so we can't do it. Would you like to contribute?

@jeremy-farrance
Copy link
Sponsor Author

Since I have demonstrated that ImageFlow can generate the desired image if we pre-generate the x1/y1, x2/y2 as needed. I am not sure I understand why it needs to be in ImageFlow, but I defer to your judgement. I took a quick look at ImageFlow and its mostly Rust and C++, so way outside my wheelhouse. I am barely an intermediate procedural-thinking C# guy that thinks everything can be done in LINQ. ;)

So, as an alternative, is there a way (now or in the future) for me to call .Picture() and modify (or pass in a function that will modify) the final URL for the image?

As I mentioned above, in the old days this was easy because we were doing the work ourselves and generating the URL that ImageResizer would process/cache the image to. But now, I don't see any way to do that. You've got your own processing pipeline here. And as I said above, there is far too much ROI in using .Picture() for me to duplicate the effort and also get my advanced cropping in place. You know what you've built, do you see any way to give me a way to pass something in to .Picture() that would let me do any of the things I need/want?

@iJungleboy
Copy link
Contributor

ATM I don't see a quick solution. The 2sxc-API doesn't know anything about the image, so the initial size etc. isn't known, which is required to do the calculations.

This could of course be programmed in, but there are quite a few hoops to jump through - various formats, performance (because you should probably cache the sizes and not look them up on each HTML-render) etc.

The current model is built on the idea that a URL is created which says what is wanted - like placing an order. The resizer provides what was requested. That's why a streamlined setup would also require that the final optimal sizes are calculated by ImageFlow, not by the Razor-Template, which in some cases wouldn't even be able to access the raw image.

AFAIK it's not too hard to create plugins (I think they are called filters or something) for ImageFlow though - in C#.... wanna give it a go?

@jeremy-farrance
Copy link
Sponsor Author

I don't believe I am capable of it. Seems like a higher level code-base and outside of being an intermediate level C# person, my programming skills in other languages remain beginner. In addition, I can't reliably contribute as I am very active in the business so I rarely get more than a few weekends at a time where I am able to learn and contribute in a sustained way no matter how much I enjoy this type of thing. Anyhow, I greatly appreciate the response and I've added this to my "wanna do" list.

@jeremy-farrance
Copy link
Sponsor Author

I did a quick search of ImageFlow (GitHub, docs, etc). Do you know how to find the info on creating a C# plugin or filter?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants