Forced orientation sometimes not reflected in the image stream

[3ph]

Describe the bug

I'm using following code to force change the orientation of the interface (and the stream):

if (orientations.count == 0) {
    mask |= UIInterfaceOrientationMaskAll;
  } else {
    for (NSString* orientation in orientations) {
      if ([orientation isEqualToString:@"DeviceOrientation.portraitUp"]) {
        mask |= UIInterfaceOrientationMaskPortrait;
      } else if ([orientation isEqualToString:@"DeviceOrientation.portraitDown"]) {
        mask |= UIInterfaceOrientationMaskPortraitUpsideDown;
      } else if ([orientation isEqualToString:@"DeviceOrientation.landscapeLeft"]) {
        mask |= UIInterfaceOrientationMaskLandscapeLeft;
      } else if ([orientation isEqualToString:@"DeviceOrientation.landscapeRight"]) {
        mask |= UIInterfaceOrientationMaskLandscapeRight;
      }
    }
  }

  if (!mask) {
    return;
  }
  [[NSNotificationCenter defaultCenter]
      postNotificationName:@(kOrientationUpdateNotificationName)
                    object:nil
                  userInfo:@{@(kOrientationUpdateNotificationKey) : @(mask)}];

Most of the time this seems to be working well and the stream changes to the correct orientation. However, sometimes it doesn't get updated and the stream stays in the old orientation (the UI would correctly update to the new orientation).

To Reproduce

There doesn't seem to be a way to consistently reproduce it but seems to happen more when the device is in landscape and portrait is forced.

When I keep rotating the phone and force flip between portrait and landscape it would eventually (usually quite soon) happen.

Interestingly the stream fixes itself when the device is rotated more than 45deg or even more interestingly is put flat without orientation change.

Expected behavior

I would expect the stream to adjust to set orientation.

Version

1.3.0

Smartphone info.

Tested on various iPhones, iOS versions ranging from iOS14 - iOS16.

Additional context

No response

Screenshots

No response

Relevant log output

No response

[shogo4405]

Did you set rtmpStream.orientation = value?

if let orientation = DeviceUtil.videoOrientation(by: UIApplication.shared.statusBarOrientation) {
    rtmpStream.orientation = orientation
}

[ThibaultBee]

Hi,

Yes, we got this code:

if let orientation = DeviceUtil.videoOrientation(by: UIApplication.shared.statusBarOrientation) {
      rtmpStream.orientation = orientation
 }

See https://github.com/apivideo/api.video-ios-live-stream/blob/2aab29979327ec71ce6710f74d439fe12b28ebcb/Sources/ApiVideoLiveStream.swift#L113.

And we are also calling this code when an UIDevice.orientationDidChangeNotification is triggered. See https://github.com/apivideo/api.video-ios-live-stream/blob/2aab29979327ec71ce6710f74d439fe12b28ebcb/Sources/ApiVideoLiveStream.swift#L286.

[leo150]

I have the same issue and it's quite hard to reproduce. Sometimes the stream is started in portrait orientation and after the rotation to landscape, the output keeps being portrait. The stream looks like landscape video after aspect fit into portrait stream. After few minutes staying in this "portrait" mode, the stream becomes distorted. Also didn't have this before 1.3.0 or so.

stream.videoOrientation = isInterfaceOrientationLandscape ?
					AVCaptureVideoOrientation.landscapeLeft : AVCaptureVideoOrientation.portrait
stream.videoSettings = [ // 1280x720 for landscape, 720x1280 for portrait
	.width: targetResolution.width,
	.height: targetResolution.height,
]

So looks like the issue isn't about videoOrientation because the video has correct orientation. More looks like it's about videoSettings and some race condition related to invalidateSession flag. Before the refactor, changing settings would instantly trigger the session to invalidate. Right now it's waiting until the next inputBuffer call.

Images of the issue

[shogo4405]

@leo150
Thank you for report. Side effects 411cf02. Not released yet. I will fix it.

[shogo4405]

I can't reproduce this issue with native iOS.
I think caused by Flutter.