Switching From GPUImageStillCamera to GPUImagePicture produces orientation issues

[powellmark]

Been playing around with the latest GPUImage and came across and interesting problem. My test app starts immediately with a live camera via GPUImageStillCamera, I have a button to select a image from the photo library which then switches the source to GPUImagePicture. If the select photo is portrait (i.e. the source image was taken with the iPhone in portrait mode), the display is rotated incorrectly. However, it appears that if the source image is landscape it is fine. The results are somewhat inconsistent, but this seems to be the general case.

Here is the test App's single View Controller:

@implementation ViewController {
GPUImageStillCamera *stillCamera;
GPUImagePicture *picture;
GPUImageFilter *filter;
}
@synthesize outputView;

- (void)viewDidLoad
{
[super viewDidLoad];
stillCamera = [[GPUImageStillCamera alloc] init];
stillCamera.outputImageOrientation = UIInterfaceOrientationPortrait;

filter = [[GPUImageGammaFilter alloc] init];
[stillCamera addTarget:filter];
[filter addTarget:self.outputView];

[stillCamera startCameraCapture];
}

- (void)viewDidUnload
{
[self setOutputView:nil];
[super viewDidUnload];

}

- (IBAction)selectPhotoFromLibrary:(id)sender {
UIImagePickerController *controller = [[UIImagePickerController alloc] init];
controller.editing = NO;
controller.sourceType = UIImagePickerControllerSourceTypePhotoLibrary;
controller.delegate = self;
[stillCamera pauseCameraCapture];
[self presentModalViewController:controller animated:YES];
}

- (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingMediaWithInfo:(NSDictionary *)info {

UIImage *original = [info objectForKey:UIImagePickerControllerOriginalImage];

picture = [[GPUImagePicture alloc] initWithImage:original];

[stillCamera removeAllTargets];
[picture addTarget:filter];
[picture processImage];

[self dismissModalViewControllerAnimated:YES];
}

[iamcam]

This may help: http://stackoverflow.com/a/9447468/694080

[powellmark]

Thanks for the link, but I think it's a different issue related to the OpenGL context. My belief is due to the fact that I can load the same image fine if the filter chain didn't have a GPUImageStillCamera as a source prior to the GPUImagePicture.

I.e.

Load Image -> Set Image Targets -> Correct
Show Live Feed -> Load Image -> Stop Feed removing its targets -> Set Image targets -> Incorrect

[iamcam]

Ah, I see. It may be one of your filters. I'm doing something similar but haven't seen any similar issues, except when I load images from the library (hence the reference above).

[powellmark]

In the test case I generated above there is just a single filter (Gamma). I am also using UIImagePickerController not ALAssets, not sure if there is some gotcha there or not.

[iamcam]

I noticed the same thing with picker, but needed a higher resolution image than the "original" photo it links to, hence ALAsset. You can force filters to process at a given size, which could be a work-around for you if you don't find a straightforward solution here

[powellmark]

I'll convert my test to use ALAsset and see if UIPickerController is the contributing factor.

[powellmark]

Just added the following to my Test App:

- (IBAction)selectPhotoUsingALAssets:(id)sender {
ALAssetsLibrary *library = [[ALAssetsLibrary alloc] init];

// Enumerate just the photos and videos group by using ALAssetsGroupSavedPhotos.
[library enumerateGroupsWithTypes:ALAssetsGroupSavedPhotos usingBlock:^(ALAssetsGroup *group, BOOL *stop) {

    // Within the group enumeration block, filter to enumerate just videos.
    [group setAssetsFilter:[ALAssetsFilter allPhotos]];

    // For this example, we're only interested in the first item.
    [group enumerateAssetsAtIndexes:[NSIndexSet indexSetWithIndex:[group numberOfAssets]-1]
                            options:0
                         usingBlock:^(ALAsset *alAsset, NSUInteger index, BOOL *innerStop) {

                             ALAssetRepresentation *rep = [alAsset defaultRepresentation];
                             CGImageRef iref = [rep fullResolutionImage];
                             if (iref) {
                                 UIImage *largeimage = [UIImage imageWithCGImage:iref];
                                 [picture removeAllTargets];
                                 picture = [[GPUImagePicture alloc] initWithImage:largeimage];
                                 [stillCamera removeAllTargets];
                                 [picture addTarget:filter];
                                 [picture processImage];
                             }
                         }];
}
                     failureBlock: ^(NSError *error) {
                         // Typically you should handle an error more gracefully than this.
                         NSLog(@"No groups");
                     }];
}

It just grabs the last image in the Saved Photos group and uses it (I made sure the last photo is a problematic photo).

After loading this, it too has orientation issues. Interestingly enough it's aspect ratio is correct, it is just side ways. While if I load via UIImagePickerController it is side ways and stretched.

Stumped here, and really creating issues for me.

[powellmark]

Summary: With the above code, and taking a picture using the iPhone camera in portrait mode, that picture loads incorrectly from both ALAssets and UIImagePickerController.

[iamcam]

You know what, I didn't notice I was just pulling up a fullScreen version of the image. Doing the full-size gives me issues

What I'm also seeing is that it differs whether or not you choose a photo from the Library vs the Photo stream. Weird.

On Jul 26, 2012, at 5:26 PM, Mark Powell wrote:

Summary: With the above code, and taking a picture using the iPhone camera in portrait mode, that picture loads incorrectly from both ALAssets and UIImagePickerController.

---

Reply to this email directly or view it on GitHub:
#298 (comment)

[iamcam]

@BradLarson Is this a GPUImage thing, or something we have to deal with before sending it to the framework? It seems that orientation is stored in the UIImage meta data, but is ignored by GPUImagePicture, or it's just doing what it's told...

[iamcam]

I have one more thing to offer, that I think might fix this problem. Fingers crossed - it's working on a couple images I'm testing with.

Use the code here:
http://stackoverflow.com/a/538064/694080

In the first line, change the max resolution: int kMaxResolution = MAX( image.size.width, image.size.height);

So back to imagePicker: didFinishPicking... you can do something like this:

UIImage *originalImg = [self scaleAndRotateImage:[info objectForKey:UIImagePickerControllerOriginalImage]];
// Do something fun with the correctly - oriented originalImg, no thanks to Cocoa ;)

[powellmark]

Wow... well, in my tests it fixes the issue from UIImagePickerController, but the ALAssets is still rotated. That's fine for my case because I'd rather now have to build a UI to selecting an image. I do worry about what sort of overhead doing this is going to cause. But... It works... it's just... gross. Heheh.

[iamcam]

I got it working with ALAssets, so maybe it's simply a matter of implementation. Take a look at my gist to see if maybe that helps you. The full resolution image is nice to have, but it is slow to load for obvious reasons. I suppose it all depends on your needs

[powellmark]

Ok, well the issue seems to be gone, the fact that we have to use the scaleAndRotate just to fix it doesn't sit well with me, but it at least works. Would be really nice to know what's going on here and fix without having the sort of overhead of "redrawing" an image. It immediately raises concerns about the unneeded load and memory usage. Oh, well, at least I can move on. :) But I wouldn't call this issue closed.

[iamcam]

That UIImage shows it correctly oriented may suggest the same things are happening in the background. I'm curious to see @BradLarson's take on it.

[powellmark]

Well, this solution, while working added more time than I would have liked. However, I was able to do a little digging and at least do it ONLY when it's required. By checking the image.imageOrientation property I only call the scale and rotate method if it's not equal to UIImageOrientationUp.

One possibility is to lap in a GPUImageRotationFilter to just rotate the filter on the GPU if it's not up.

[iamcam]

That's not a bad idea. For now I'm just going to inject some code into the scaleAndRotateImage: method for now. Did you find that rotation is inherent to GPUImageFilter, or do you think we need a discreet filter specifically for rotation?