[Bug] [CameraView] [Android] Taken photos are rotated incorrectly

[lassana]

Description

When taking a photo with CameraView on Android, photos are rotated incorrectly.

Previously reported in #297, #751, #779. PRs #307, #886 claim to fix this but they don't.

Stack Trace

Link to Reproduction Sample

CameraBug.zip

Steps to Reproduce

Confirmed on Pixel 3a XL Android 12 and Xiaomi Redmi 3s Android 6.0.

1. Open the attached project and run.

2. Press the Capture button which invokes the Shutter() method:

3. The app saves the captured image data into a new file and sets that file as a source of the ImageView (right below the camera view):

Expected Behavior

ImageData of MediaCapturedEventArgs contains a JPEG image with correct orientation applied.

Actual Behavior

ImageData is an incorrectly rotated JPEG and additional manual transformations are required.

Basic Information

• Version with issue: 1.3.0-pre2
• Last known good version: None
• IDE:
• Platform Target Frameworks:
  • iOS:
  • Android:
  • UWP:
• Android Support Library Version:
• Nuget Packages:
• Affected Devices: Pixel 3a XL (Android 12), Xiaomi Redmi 3s (Android 6)

Workaround

A workaround is to use ImageSharp where you can manually rotate the image according to MediaCapturedEventArgs.Rotation property. It works pathetically slow though.

public static class MediaCapturedEventArgsExtensions
{
    public static byte[] GetRotatedData(this MediaCapturedEventArgs self)
        => TransformOrientation(self.ImageData, self.Rotation);

    private static byte[] TransformOrientation(byte[] imageInBytes, double rotation)
    {
        using var image = Image.Load(imageInBytes, out IImageFormat imageFormat);
        RotateMode mode = rotation switch
        {
            90d => RotateMode.Rotate90,
            180d => RotateMode.Rotate180,
            270d => RotateMode.Rotate270,
            _ => RotateMode.None,
        };
        image.Mutate(x => x.Rotate(mode));
        return ImageToByteArray(image, imageFormat);
    }

    private static byte[] ImageToByteArray(Image image, IImageFormat imageFormat)
    {
        using var ms = new MemoryStream();
        image.Save(ms, imageFormat);
        return ms.ToArray();
    }
}

Reproduction imagery

[lassana]

A better workaround is to write EXIF data into the image using AndroidX ExifInterface:

public void SetExifOrientation(string filePath, double rotation)
{
    int orientationValue = rotation switch
    {
        90d => ExifInterface.OrientationRotate90,
        180d => ExifInterface.OrientationRotate180,
        270d => ExifInterface.OrientationRotate270,
        _ => ExifInterface.OrientationNormal,
    };
    var exif = new ExifInterface(filePath);
    exif.SetAttribute(
        tag: ExifInterface.TagOrientation,
        value: orientationValue.ToString());
    exif.SaveAttributes();
}

[GUOLDEV]

Hi @lassana, could you please confirm if it is still happening in the latest XCT version?

[lassana]

Hi @lassana, could you please confirm if it is still happening in the latest XCT version?

Hi @GUOLDEV , it's still happening in 1.3.0:

[GUOLDEV]

Hi @lassana thanks for testing! If possible, could you please test it using XCT sample app as well?

[GUOLDEV]

@lassana I think I understand what you mean. You are talking about how the CameraView currently works. And yes, you have to handle the rotation by yourself based on the Rotation argument. So it's not a bug. It's more a behaviour change request, right? @jfversluis what do you think?

[lassana]

And yes, you have to handle the rotation by yourself based on the Rotation argument.

Yep that's what my workaround does, but the question is why would you have to to deal with additional properties at all? Isn't it the library's job to generate a properly rotated image so developers wouldn't have to copy-paste the same workaround? I guess rotating the whole image might be quite time-consuming, but writing the EXIF data works rather quickly. BTW it seems it's been planned at some point but left as a to-do:

XamarinCommunityToolkit/src/CommunityToolkit/Xamarin.CommunityToolkit/Views/CameraView/Android/CameraFragment.android.cs

Lines 380 to 388 in 02884f5

	// See TODO on CameraView.SavePhotoToFile
	// Insert Exif information to jpeg file
	/*if (Element.SavePhotoToFile)
	{
	filePath = ConstructMediaFilename(null, extension: "jpg");
	File.WriteAllBytes(filePath, bytes);
	}
	Sound(MediaActionSoundType.ShutterClick);
	OnPhoto(this, (filePath, Element.SavePhotoToFile ? null : bytes);*/

It's more a behaviour change request, right?

It certainly can be treated as the following behavior change: make the CameraView behave on Android just like it does on iOS, because the described issue does not happen on iOS. However, the image has to be stored as a file in order to use AndroidX ExifInterface, so CameraView.SavePhotoToFile might have to be implemented first.

[bbhxwl]

And yes, you have to handle the rotation by yourself based on the Rotation argument.

Yep that's what my workaround does, but the question is why would you have to to deal with additional properties at all? Isn't it the library's job to generate a properly rotated image so developers wouldn't have to copy-paste the same workaround? I guess rotating the whole image might be quite time-consuming, but writing the EXIF data works rather quickly. BTW it seems it's been planned at some point but left as a to-do:

XamarinCommunityToolkit/src/CommunityToolkit/Xamarin.CommunityToolkit/Views/CameraView/Android/CameraFragment.android.cs

Lines 380 to 388 in 02884f5

	// See TODO on CameraView.SavePhotoToFile
	// Insert Exif information to jpeg file
	/*if (Element.SavePhotoToFile)
	{
	filePath = ConstructMediaFilename(null, extension: "jpg");
	File.WriteAllBytes(filePath, bytes);
	}
	Sound(MediaActionSoundType.ShutterClick);
	OnPhoto(this, (filePath, Element.SavePhotoToFile ? null : bytes);*/

It's more a behaviour change request, right?

It certainly can be treated as the following behavior change: make the CameraView behave on Android just like it does on iOS, because the described issue does not happen on iOS. However, the image has to be stored as a file in order to use AndroidX ExifInterface, so CameraView.SavePhotoToFile might have to be implemented first.

So do I. So do I when I use 1.2.0.

[jfversluis]

Hey everyone! Thank you for your patience here! Would you maybe be able to test version 2.1.0-build-64069.1871 from this NuGet feed: https://pkgs.dev.azure.com/xamarin/public/_packaging/XamarinCommunityToolkit-PullRequests/nuget/v3/index.json

That is the result of a PR I just made to fix this rotation thing. Let me know if this works!

[brettnguyen]

Hey everyone! Thank you for your patience here! Would you maybe be able to test version 2.1.0-build-64069.1871 from this NuGet feed: https://pkgs.dev.azure.com/xamarin/public/_packaging/XamarinCommunityToolkit-PullRequests/nuget/v3/index.json

That is the result of a PR I just made to fix this rotation thing. Let me know if this works!
@jfversluis can you post screenshots from you end of this working? I will test as soon as I can. Thank you so much for all the help

[brettnguyen]

@jfversluis also does this fix videos too?

[brettnguyen]

A better workaround is to write EXIF data into the image using AndroidX ExifInterface:

public void SetExifOrientation(string filePath, double rotation)
{
    int orientationValue = rotation switch
    {
        90d => ExifInterface.OrientationRotate90,
        180d => ExifInterface.OrientationRotate180,
        270d => ExifInterface.OrientationRotate270,
        _ => ExifInterface.OrientationNormal,
    };
    var exif = new ExifInterface(filePath);
    exif.SetAttribute(
        tag: ExifInterface.TagOrientation,
        value: orientationValue.ToString());
    exif.SaveAttributes();
}

@lassana
Can you provide an example with in a camera app project? also does this stop preview videos being rotated too?

[boris-df]

V2.0.5 (the latest available right now) still has this rotation problem on Android (works fine on iOS!)

Problem: not only photos are rotated wrong - but videos too!
AFAIK we could write a file with different EXIF-Values (which in turn might cause problems with other app that open such files - because not every app is looking into the exif data to rotate correctly!) - but this won't work with Video, right?

If i rotate my Pixel4a 90° to the right (to get a landscape video) the video is head down or so.. very weird behaviour

[boris-df]

Hey everyone! Thank you for your patience here! Would you maybe be able to test version 2.1.0-build-64069.1871 from this NuGet feed: https://pkgs.dev.azure.com/xamarin/public/_packaging/XamarinCommunityToolkit-PullRequests/nuget/v3/index.json

I cannot find any newer version than 2.0.5 with this link ... where can i find the Android fix to apply it myself?

[DaviBittencourt]

@jfversluis I tested version 2.1.0-build-64069.1871 and it is working.

[michaldivis]

@DaviBittencourt nice! Would have been nice to have this fix about 2 years ago though.

[harsh9592]

Any idea when the version with the solution will be release for production or available in nuget.org? As of now I can update upto 2.0.5 version of xamarin.community.toolkit.
Any timeline or idea on availability of version 2.1.0 . Also not found update in pre-release.
Thanks.