Rotation on iOS does not look right #16322
Comments
DaveShuckerow
added
platform-ios
|
|
tvolkert
added
customer: splashbulb
engine
|
Cross-posting from #13300 and consolidating on this issue:
|
|
/cc @chickenblood |
|
the stretching of the aspect ratio is what looks worst here. We could potentially also cross-fade from a fixed rotated background so that we don't incur the black edges. |
I would love some control here. Specifically it would be nice to allow for the device to rotate without utilizing SystemChrome.setPreferredOrientations but not forcing the main widget to repaint/size. I could see instead passing a callback to the WidgetsApp with an orientation change. From a developer's perspective this callback could be passed to something like a ChangeNotifierProvider and allow us to have very granule control of this process. Similar to how the Camera app functions on iOS where the main view doesn't actually switch between states but instead the icons rotate to their appropriate view. This skewing is really really annoying from a UX point of view. |
|
/cc @goderbauer |
goderbauer
added
the
framework
|
@dnfield Any update on this? We are running into this as well, and on iPads it looks pretty bad. |
|
I'm not aware of anyone working on this at the moment - it might take a little bit between holidays coming up and Flutter Interact, but it is being tracked. @xster may know more. |
|
The way this rotation is implemented on iOS, is that it generates the final frame by applying an animated size change (not transform) to the entire window content (the root view controller's view), so the layout changes dynamically, while applying a 90-degree rotation transform at the same time. At the end of the rotation, the applied transform and layout match what is should look like in landscape. I'm not familiar with the Flutter codebase yet, but would be willing to take a look. Would what I describe or something like it be possible with Flutter's layout engine? |
|
We want to implement -viewWillTransitionToSize:withTransitionCordinator: in FlutterViewController With that, we should be able to pipe the relevant information down to the framework so it can know to run an animation for rotation and when that animation should end. One thing unclear to me is how long that animation should be, and whether we'll be able to communicate with the framework quickly enough to actually get the animation going in time. Another option would be to just try to do it all in FlutterViewController.mm and basically just do a size animation on whatever our last frame was unti lwe finish rotating. |
|
@dnfield in the mean time, is there a known method to hide the transition during the 200 (or so) milliseconds it takes to rotate? I made an attempt by extending WidgetsFlutterBinding, made my own runApp method to pass my own WidgetsBinding, overrode |
|
I believe it will require engine modifications to be able to start drawing on time - not 100% sure though. @chinmaygarde may know about that? |
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as duplicate.
|
Reproducible on the latest versions of flutter recordingScreen.Recording.2023-01-03.at.10.12.36.movupdated sampleimport 'package:flutter/material.dart';
import 'package:flutter/services.dart';
void main() {
SystemChrome.setPreferredOrientations([
DeviceOrientation.portraitUp,
DeviceOrientation.landscapeLeft,
DeviceOrientation.landscapeRight,
]);
runApp(const MyApp());
}
class MyApp extends StatelessWidget {
const MyApp({Key? key}) : super(key: key);
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Flutter Demo',
theme: ThemeData(
primarySwatch: Colors.blue,
),
home: const MyHomePage(),
);
}
}
class MyHomePage extends StatelessWidget {
const MyHomePage({Key? key}) : super(key: key);
@override
Widget build(BuildContext context) {
return const Scaffold(
body: Center(
child: Text('hello world'),
),
);
}
}flutter doctor -v |
danagbemava-nc
added
found in release: 3.3
This comment was marked as duplicate.
This comment was marked as duplicate.
|
Also internally tracked with b/266184881. |
|
This is a summary of our research so far, in case it can be helpful for a potential better solution in the future. There are 4 approaches discussed below: Approach 1. Snapshot trick This was initially attempted by Chinmay. On device, we add a snapshot during the rotation, and then fade it out after the rotation is complete. This approach simply moves the distortion from the beginning of the rotation to the end of the rotation, so the same amount of distortion is still there, just moved to the end (video here). Approach 2. Improved snapshot trick We improved this approach by swapping snapshot in the middle of the rotation (rather than at the end of rotation). The benefits are: However, there are still 2 drawbacks: Slight pause with dynamic content: 2.snapshot.video.mp4Approach 3. Size interpolation The original idea was from Dan's comment. Basically the engine sends interpolated sizes during the rotation. (PR here) This approach worked pretty well for simple UI - there are almost no distortions at all during the rotation. However, on complex UI (e.g. flutter gallery), we experienced significant frame drops. Reducing the number of interpolations helps with frame drops, but it also resulted in a very un-smooth transition. Works well with simple UI: 3.size.interpolation.mp4Frame drop on complex UI: interpolation.perf.issue.mp4Approach 4. Delayed swap This is actually the simplest approach - just delay the swap of width and height until the middle of rotation, rather than at the beginning of the rotation. (PR here) This approach gives us the same benefit as approach 2, without the 2 drawbacks discussed above. Note that there is still a swapping (or "jumping effect") in the middle of rotation, but it's significantly less noticeable than the existing behavior, which "swaps/jumps" at the beginning of rotation, with larger amount of distortion. Delayed swap on complex UI: delayed.swap.mp4So in conclusion, with the pros and cons discussed above, we will go with approach 4. |
|
@hellohuanlin Couldn't we take a screenshot at the start of the rotation where height and width have already changed and then fade it in? |
When you take a screenshot where height and width already changed, the screenshot itself has the max distortion. So the result won't look good.
Nope. The native iOS actually does size interpolation. |
|
@hellohuanlin in a native iOS app, it doesn't look to me like the size is interpolated.
This screenshot was taken at the beginning of a screen rotation. You can see 2 "screenshots" one of the state before the rotation, and one of the state after the rotation, with the exception that the end state didn't just change width / height, but the height remains (so that the complete height can be filled during the fade in).
I hope you understand what I mean. Here is the full video of the rotation RPReplay_Final1680721048.MP4 |
|
@b3nni97 I don't think apple is taking a screenshot (otherwise there would be distortion of the screenshot when the window's aspect ratio changes). It's impossible to take a screenshot of what the screen will look like after the rotation is complete, at the time before the rotation even starts (if that's what you want) - the screenshot shows exactly what the current screen is when the screenshot is taken. Your particular example is interesting - it changes the entire layout to a split view, in which case I'm not sure what apple is doing under the hood. Without layout change, apple's behavior is size interpolation, as shown below: swiftui.mp4 |
|
@hellohuanlin
Hmm, I know too little about the engine, I thought it would be possible to do a complete rebuild with the new width and height, and then display this as a "screenshot". |
The screenshot is not flutter-related tho. It's just UIKit API to take a screenshot of the current screen. So once the end state is displayed, you can't "blend it in", as it's already on the screen. |
|
I think @b3nni97 may be suggesting we could build a new tree with the new width & height and paint it to a buffer rather than the screen. |
|
@tvolkert Gotcha. That likely requires big changes in both framework and engine. However, a new idea came to me that fading in end-state snapshot should have similar result to fading out begin-state snapshot. The latter is much easier to do, so I did a quick experiment (and I call this "approach 5"): Approach 5 Fade out begin-state snapshot during the rotation. Here's the result: 100.fade.mp4This doesn't look at good as the original approach 2 when I tried it on device, because:
There's another approach that combines approach 2 and 5, and I call this approach 6. Approach 6: Show the begin-state snapshot for 40% of rotation, then fade out for 20% of rotation, then show end-state real screen for 40% of rotation (code here, number adjustable, I just find 40%-20%-40% result in the smoothest transition). Here's the result: 20.fade.mp4Approach 6 is smoother than approach 2. I think it has the best result among approaches 1, 2, 5 and 6. However, I still think approach 4 is better than any snapshot tricks, since it works for dynamic contents (videos & animations). |
|
Yeah I think approach 4 still looks the best. |
…distortion (#40730) The "size interpolation" solution didn't go well (more context [here](#40412 (comment))). Then a new solution came to my mind, and I call it **"delayed swap"**: In the originally behavior, we swap the width/height immediately before the rotation, resulting in roughly ~4x distortion in the beginning. With "delayed swap" solution, we **swap the width/height right in the middle of the rotation** (i.e. delay the swap for half of the transition duration). This new "delayed swap" solution gives us the same benefit as the "snapshot" solution: - reducing ~4x distortion to ~2x - most distorted frames occur in the middle of rotation when it's moving the fastest, making it hard to notice And it fixes the drawback of "snapshot" solution: - it works well with dynamic content like animation or video - it doesn't have a ~0.5 second penalty when taking the snapshot Looks pretty good on flutter gallery: https://user-images.githubusercontent.com/41930132/228383137-7cd09982-89a9-4c83-bf55-9431de708278.mp4 *List which issues are fixed by this PR. You must list at least one issue.* Fixes flutter/flutter#16322 *If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].* [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
danagbemava-nc
added
the
r: fixed
|
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of |
and others
Internal: b/77275812
From an internal user trying out Flutter on the external flow for iOS:
The text was updated successfully, but these errors were encountered: