Apply updates to XY and Anchor positions based on EXIF metadata #221

JimBobSquarePants · 2022-02-07T04:32:00Z

Prerequisites

I have written a descriptive pull-request title
I have verified that there are no overlapping pull-requests open
I have verified that I am following matches the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules 👮.
I have provided test coverage for my change (where applicable)

Description

Following on from #217 This PR updates ResizeWebProcessor to correctly handle XY and Anchor commands when processing EXIF orientated images.

There's a lot of work involved in handling this with most of the logic based upon some of the work upstream in ImageSharp when normalizing orientation values. The following image will help explain the affect on an image each rotation value has.

cc @ronaldbarendse

.github/CONTRIBUTING.md

JimBobSquarePants · 2022-02-07T04:33:53Z

src/ImageSharp.Web/FormattedImage.cs

+        /// </summary>
+        /// <param name="orientation">The decoded orientation. Use <see cref="ExifOrientationMode"/> for comparison.</param>
+        /// <returns>The <see cref="bool"/> indicating whether the image contains EXIF metadata indicating that the image is rotated.</returns>
+        public bool IsExifRotated(out ushort orientation)


I've made this public as it's useful for other processor implementations. The EXIF parsing logic is identical to AutoOrientProcessor<T>.GetExifOrientation()

Why is this only checking for rotated images and not flipped ones? I think that made sense when only considering width/height, but because the ResizeWebProcessor also needs to handle the correct X/Y and anchor position, this should also return true when it's flipped.

I'd also rename it to TryGetExifOrientation(out ushort orientation) and return whether the image contains a valid EXIF orientation value or not. Then just check if it's false or the orientation is set to 1 (top/left) before doing any of the transforms.

Why is this only checking for rotated images and not flipped ones?

Because rotated is an important metric as that affects Size. Flipped does not affect that property and I wanted a quick way to tell, writing the method to reflect that desire. Can easily refactor though to suit your design and move rotate the check inside the processor itself.

There you go. Refactored. Much cleaner now.

JimBobSquarePants · 2022-02-07T04:35:40Z

src/ImageSharp.Web/Processors/ResizeWebProcessor.cs

+                return anchor;
+            }
+
+            /*


I had to use a notepad for this bit. I think my logic is sound but could do with a second opinion.

codecov · 2022-02-07T04:39:05Z

Codecov Report

Merging #221 (eb4f34f) into master (39c9a8c) will increase coverage by 0%.
The diff coverage is 92%.

@@          Coverage Diff           @@
##           master   #221    +/-   ##
======================================
  Coverage      86%    87%            
======================================
  Files          70     70            
  Lines        1828   1928   +100     
  Branches      268    283    +15     
======================================
+ Hits         1588   1680    +92     
- Misses        170    174     +4     
- Partials       70     74     +4

Flag	Coverage Δ
unittests	`87% <92%> (+<1%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/ImageSharp.Web/FormattedImage.cs	`80% <54%> (-7%)`	⬇️
...rc/ImageSharp.Web/Processors/ResizeWebProcessor.cs	`94% <96%> (+1%)`	⬆️
.../ImageSharp.Web/Middleware/ImageSharpMiddleware.cs	`85% <100%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 39c9a8c...eb4f34f. Read the comment docs.

Jeavon · 2022-02-07T16:04:21Z

I have discovered this PR additionally fixes a null exception for images lacking orientation metadata 👍

JimBobSquarePants · 2022-02-08T06:27:02Z

Just added a bunch of manual testing to confirm my approach. Seems to work very well.

deanmarcussen · 2022-02-08T12:21:00Z

src/ImageSharp.Web/Middleware/ImageSharpMiddleware.cs


        /// <summary>
        /// Used to temporarily store cache resolver reads to reduce the overhead of cache lookups.
        /// </summary>
        private static readonly ConcurrentTLruCache<string, (IImageCacheResolver, ImageCacheMetadata)> CacheResolverLru
-            = new(1024, TimeSpan.FromMinutes(5));
+            = new(1024, TimeSpan.FromSeconds(30));


are you going to pop these back up? or make them configurable?

I guess we could make them configurable but I've no idea what to name the properties (InMemoryCacheDuration?)

I found that if I deleted the cache folder when something was stored in there it prompted a null reference exception in the cache resolver. I dropped the times to make that less likely.

Ah... They're private static. We can't pass options.

Ah... They're private static. We can't pass options.

True, there might be an argument for making them non static, for performance.
In our scenario, we have multiple instances of the middleware (per tenant), so multiple instances of the cache per tenant, is perhaps desirable, as the items count will be less, so the lookup a little faster.
(then becomes configurable)

Something we can think about later though :)

Yeah that might well be an idea then.

ronaldbarendse

I've added my suggestions in PR #224, as the current implementation doesn't correctly parse/transform the center coordinates and isn't easily re-usable in custom processors.

ronaldbarendse · 2022-02-08T13:59:01Z

src/ImageSharp.Web/Processors/ResizeWebProcessor.cs

+            }
+
+            AffineTransformBuilder builder = new();
+            Size size = image.Image.Size();


The image size shouldn't be required when dealing with coordinates ranging from 0 to 1.

I'd actually forgotten that those coordinates were 0-1. They're not documented well on the underlying type. Suggested fixes are much more useful here that in a separate PR.

I don't want the other changes. I don't want to have to create yet another public type, especially one that is mostly filled with methods specific to ResizeWebProcessor. The PointF Transform method for example requires the X/Y properties to be within a 0-1 range. Not useful for anything that currently exists. You're creating solutions to problems I don't and won't have. Processor implementation outside of this library are the responsibility of whoever develops them not me. I'm building a framework, not a utility belt.

I've implemented a proper fix for XY transforms now anyway, took me longer than I'm proud of to figure out. If you look at the image above and compare it to the code the process actually becomes pretty clear.

Most of those methods are specific to the ResizeOptions indeed, which can be re-used on different web processors that require resizing (e.g. when trying to crop to a specific target rectangle, which Umbraco does in the CropWebProcessor based on crop coordinates), so it's not really specific to ResizeWebProcessor.

Making the PointF transform use a different range would be as simple as moving the Size variable to a parameter, but I get the point you're making here 😉

I've quickly tested the latest changes and it all seems to work as expected 👍🏻

Thanks for testing, that's very useful 👍

JimBobSquarePants added 2 commits February 7, 2022 12:59

Apply updates to XY and Anchor positions based on EXIF metadata

b4fd913

Fix XY position.

45c4352

JimBobSquarePants added enhancement breaking api labels Feb 7, 2022

JimBobSquarePants requested review from deanmarcussen and a team February 7, 2022 04:32

JimBobSquarePants commented Feb 7, 2022

View reviewed changes

.github/CONTRIBUTING.md Show resolved Hide resolved

JimBobSquarePants commented Feb 7, 2022

View reviewed changes

JimBobSquarePants added this to the v2.0.0 milestone Feb 7, 2022

JimBobSquarePants added 4 commits February 8, 2022 10:27

Merge branch 'master' into js/exif-resize-fixes

15dfa53

Update FormattedImage.cs

9bc9532

Update Directory.Build.targets

7c41792

Add manual tests and fixes

f2b3594

deanmarcussen reviewed Feb 8, 2022

View reviewed changes

Use TryGetExifOrientation

17ef18c

ronaldbarendse suggested changes Feb 8, 2022

View reviewed changes

JimBobSquarePants added 2 commits February 9, 2022 03:29

Fix XY transforms

72d128b

Fix Anchor position for TopRight

eb4f34f

JimBobSquarePants merged commit 40a5075 into master Feb 9, 2022

JimBobSquarePants deleted the js/exif-resize-fixes branch February 9, 2022 00:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Apply updates to XY and Anchor positions based on EXIF metadata #221

Apply updates to XY and Anchor positions based on EXIF metadata #221

JimBobSquarePants commented Feb 7, 2022

JimBobSquarePants Feb 7, 2022

ronaldbarendse Feb 8, 2022

JimBobSquarePants Feb 8, 2022 •

edited

Loading

JimBobSquarePants Feb 8, 2022

JimBobSquarePants Feb 7, 2022

codecov bot commented Feb 7, 2022 •

edited

Loading

Jeavon commented Feb 7, 2022

JimBobSquarePants commented Feb 8, 2022

deanmarcussen Feb 8, 2022

JimBobSquarePants Feb 8, 2022

JimBobSquarePants Feb 8, 2022

deanmarcussen Feb 8, 2022

JimBobSquarePants Feb 8, 2022

ronaldbarendse left a comment

ronaldbarendse Feb 8, 2022

JimBobSquarePants Feb 8, 2022

JimBobSquarePants Feb 8, 2022

ronaldbarendse Feb 8, 2022

JimBobSquarePants Feb 8, 2022

Apply updates to XY and Anchor positions based on EXIF metadata #221

Apply updates to XY and Anchor positions based on EXIF metadata #221

Conversation

JimBobSquarePants commented Feb 7, 2022

Prerequisites

Description

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JimBobSquarePants Feb 8, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Feb 7, 2022 • edited Loading

Codecov Report

Jeavon commented Feb 7, 2022

JimBobSquarePants commented Feb 8, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ronaldbarendse left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JimBobSquarePants Feb 8, 2022 •

edited

Loading

codecov bot commented Feb 7, 2022 •

edited

Loading