Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make iOS 15 SDK and runtime available to engine builders #85555

Closed
jmagman opened this issue Jun 29, 2021 · 46 comments · Fixed by flutter/engine#31915
Closed

Make iOS 15 SDK and runtime available to engine builders #85555

jmagman opened this issue Jun 29, 2021 · 46 comments · Fixed by flutter/engine#31915
Assignees
Labels
engine flutter/engine repository. See also e: labels. P1 High-priority issues at the top of the work list platform-ios iOS applications specifically team Infra upgrades, team productivity, code health, technical debt. See also team: labels. team-infra Owned by Infrastructure team

Comments

@jmagman
Copy link
Member

jmagman commented Jun 29, 2021

Xcode 13 beta 2 is now available on go/xcode. Add the iOS 15 SDK and simulator runtime to the Xcode 12a7209 cipd package and reupload.
This will allow the engine to start implementing new iOS features, and to test them on the simulator.

See go/xcode-at-flutter.

@jmagman jmagman added team Infra upgrades, team productivity, code health, technical debt. See also team: labels. platform-ios iOS applications specifically team-infra Owned by Infrastructure team labels Jun 29, 2021
@jmagman jmagman changed the title Add iOS 15 SDK and runtime to engine Xcode cipd package Add iOS 15 and macOS 12 SDK and runtime to engine Xcode cipd package Jun 29, 2021
@jmagman
Copy link
Member Author

jmagman commented Jun 30, 2021

Having trouble uploading a second Xcode cipd package with the same version. Filed https://bugs.chromium.org/p/chromium/issues/detail?id=1225115.

@jmagman jmagman changed the title Add iOS 15 and macOS 12 SDK and runtime to engine Xcode cipd package Make iOS 15 beta 3 and macOS 12 beta 3 SDK and runtime available to engine builders Jul 1, 2021
@jmagman jmagman changed the title Make iOS 15 beta 3 and macOS 12 beta 3 SDK and runtime available to engine builders Make iOS 15 beta 2 and macOS 12 beta 3 SDK and runtime available to engine builders Jul 1, 2021
@jmagman jmagman changed the title Make iOS 15 beta 2 and macOS 12 beta 3 SDK and runtime available to engine builders Make iOS 15 beta 2 and macOS 12 beta 2 SDK and runtime available to engine builders Jul 1, 2021
@jmagman
Copy link
Member Author

jmagman commented Jul 1, 2021

That package didn't work, it had universal-darwin21/ruby/config.h instead of the expected universal-darwin19/ruby/config.h

@jmagman jmagman changed the title Make iOS 15 beta 2 and macOS 12 beta 2 SDK and runtime available to engine builders Make iOS 15 beta 2 SDK and runtime available to engine builders Jul 1, 2021
@jmagman
Copy link
Member Author

jmagman commented Jul 1, 2021

Hmm, the framework is trying to be built with the latest SDK, I guess that isn't pinned to iOS 14 somewhere:
https://logs.chromium.org/logs/flutter/led/magder_google.com/0b941017b4e4bba1a371aeac687eb25a17e9355d321c625defd28e38a95aff3b/+/u/build_ios_debug_sim/stdout

clang++ -shared -isysroot /opt/s/w/ir/cache/osx_sdk/XCode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator15.0.sdk ...

Which is causing it to hit dart-lang/sdk#46499.

@jmagman
Copy link
Member Author

jmagman commented Jul 14, 2021

dart-lang/sdk#46499 is fixed, let's see if this passes:
https://luci-milo.appspot.com/raw/build/logs.chromium.org/flutter/led/magder_google.com/04c064813ea2ef0203ec50b74086e113c07fbb53a84ab5fb36548f875e21f0b9/+/build.proto

Next step is to investigate why felt_ios-safari_test isn't finding the expected simulator:

  �[31mFailed to load "test/canvaskit/color_filter_golden_test.dart":�[0m
  Failed to run Safari: Exception: Safari failed with exit code 148.
  Standard output:
  No devices are booted.
  .
  dev/browser.dart 104:9  new Browser.<fn>

https://luci-milo.appspot.com/raw/build/logs.chromium.org/flutter/led/magder_google.com/5eefc89e563aabdf6e520837b25dce91dccf894274b2ca88de767fc1d6a18e92/+/build.proto

@jmagman
Copy link
Member Author

jmagman commented Jul 14, 2021

Hmm, they both passed this time.

@jmagman
Copy link
Member Author

jmagman commented Jul 14, 2021

@jmagman
Copy link
Member Author

jmagman commented Jul 14, 2021

@jmagman
Copy link
Member Author

jmagman commented Jul 15, 2021

  1. Old version of clang can't parse the iOS 15 headers with new _Nullable_result nullability annotation https://reviews.llvm.org/D92495
/opt/s/w/ir/cache/osx_sdk/XCode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator15.0.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSURLSession.h:500:168: error: expected ')'
- (void)readDataOfMinLength:(NSUInteger)minBytes maxLength:(NSUInteger)maxBytes timeout:(NSTimeInterval)timeout completionHandler:(void (^) (NSData * _Nullable_result data, BOOL atEOF, NSError * _Nullable error))completionHandler;

https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket.appspot.com/8841691005180545936/+/u/build_ios_debug_sim_arm64/stdout

/opt/s/w/ir/cache/osx_sdk/XCode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator15.0.sdk/System/Library/Frameworks/Metal.framework/Headers/MTLDevice.h:266:74: note: to match this '('
typedef void (^MTLNewComputePipelineStateWithReflectionCompletionHandler)(id <MTLComputePipelineState> __nullable computePipelineState, MTLComputePipelineReflection * _Nullable_result reflection, NSError * __nullable error);

https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket.appspot.com/8841691005230098080/+/u/build_ios_debug_sim_arm64/stdout

  1. New HarfBuzz warning (I wonder why, it should be on the same toolchain as before?) @jason-simmons says

HarfBuzz worked around this by disabling the Clang range-loop-analysis check (see harfbuzz/harfbuzz@86993c0)
We can cherry pick that to our branch of HarfBuzz or disable it in the GN build rules

[1610/3970] CXX obj/third_party/harfbuzz/src/harfbuzz_sources.hb-subset.o
In file included from ../../third_party/harfbuzz/src/hb-subset.cc:35:
../../third_party/harfbuzz/src/hb-ot-hdmx-table.hh:110:40: warning: loop variable '_' is always a copy because the range of type 'hb_map_iter_t<hb_range_iter_t<unsigned int, unsigned int>, (lambda at ../../third_party/harfbuzz/src/hb-ot-hdmx-table.hh:126:15), hb_function_sortedness_t::NOT_SORTED, nullptr>' does not return a reference [-Wrange-loop-analysis]
    for (const hb_item_type<Iterator>& _ : +it)
                                       ^
../../third_party/harfbuzz/src/hb-ot-hdmx-table.hh:145:17: note: in instantiation of function template specialization 'OT::hdmx::serialize<hb_map_iter_t<hb_range_iter_t<unsigned int, unsigned int>, (lambda at ../../third_party/harfbuzz/src/hb-ot-hdmx-table.hh:126:15), hb_function_sortedness_t::NOT_SORTED, nullptr>, nullptr>' requested here
    hdmx_prime->serialize (c->serializer, version, it);
                ^
../../third_party/harfbuzz/src/hb-ot-hdmx-table.hh:110:10: note: use non-reference type 'hb_item_type<hb_map_iter_t<hb_range_iter_t<unsigned int, unsigned int>, (lambda at ../../third_party/harfbuzz/src/hb-ot-hdmx-table.hh:126:15), hb_function_sortedness_t::NOT_SORTED, nullptr> >' (aka 'hb_pair_t<unsigned int, hb_map_iter_t<hb_map_iter_t<hb_range_iter_t<unsigned int, unsigned int>, hb_map_t *&, hb_function_sortedness_t::NOT_SORTED, nullptr>, (lambda at ../../third_party/harfbuzz/src/hb-ot-hdmx-table.hh:134:16), hb_function_sortedness_t::NOT_SORTED, nullptr> >')
    for (const hb_item_type<Iterator>& _ : +it)
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.

https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket.appspot.com/8841691005886643360/+/u/build_ios_debug_sim_arm64/stdout

  1. Scenario app needs to be pegged to older OS?
Running command "xcodebuild -sdk iphonesimulator -scheme IosUnitTests -destination platform='iOS Simulator,name=iPhone 8' test FLUTTER_ENGINE=ios_debug_sim"
2021-07-14 16:37:15.326 xcodebuild[53204:932846] [MT] DVTSDK: Skipped SDK /opt/s/w/ir/cache/osx_sdk/XCode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.6.sdk; its version (13.6) is below required minimum (14.0) for the iphonesimulator platform.
2021-07-14 16:37:15.328 xcodebuild[53204:932846] [MT] DVTSDK: Skipped SDK /opt/s/w/ir/cache/osx_sdk/XCode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.6.sdk; its version (13.6) is below required minimum (14.0) for the iphoneos platform.
Command line invocation:
    /opt/s/w/ir/cache/osx_sdk/XCode.app/Contents/Developer/usr/bin/xcodebuild -sdk iphonesimulator -scheme IosUnitTests -destination "platform=iOS Simulator,name=iPhone 8" test FLUTTER_ENGINE=ios_debug_sim

Build settings from command line:
    FLUTTER_ENGINE = ios_debug_sim
    SDKROOT = iphonesimulator15.0

Fatal error: invalid Xcode.app 'version.plist': invalid content in '/opt/s/w/ir/cache/osx_sdk/XCode.app/Contents/version.plist' - ProductBuildVersion '13A5155e' does not match CFBundleShortVersionString '12.100.3' because their major version numbers differ (13 vs 12).: file XCBCore/Core.swift, line 198
xcodebuild: error: Unable to find a destination matching the provided destination specifier:
		{ platform:iOS Simulator, OS:latest, name:iPhone 8 }

https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket.appspot.com/8841691004871486816/+/u/Host_Tests_for_ios_debug_sim/stdout

  1. felt ios-safari test can't find the expected simulator:
  �[31mFailed to load "test/canvaskit/backdrop_filter_golden_test.dart":�[0m
  Failed to run Safari iOS: Exception: Safari iOS failed with exit code 148.
  Standard output:
  No devices are booted.
  .
  dev/browser.dart 143:9  new Browser.<fn>

https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket.appspot.com/8841691005349843248/+/u/felt_ios-safari_test/stdout

@chinmaygarde
Copy link
Member

The Xcode 13 toolchain is available on CI now we should definitely proceed on unblock this as it will hinder use of newer APIs.

@jmagman Do you believe we should file bugs for the individual work issues you identified or are they all minor enough that we someone can attempt to track and fix these in one go?

@chinmaygarde chinmaygarde added the engine flutter/engine repository. See also e: labels. label Nov 4, 2021
@jmagman
Copy link
Member Author

jmagman commented Nov 5, 2021

The Xcode 13 toolchain is available on CI

Do you mean the cipd package I uploaded? It doesn't work yet, it's missing some simulator runtimes, and more importantly, the version of clang enforced by flutter_tool can't even be used with the iOS 15 headers with new nullability annotation. #85555 (comment)

We need to keep the builders on a version of clang older than or equal to the version enforced by the flutter_tool, or users will have App Store submission failures related to the bitcode version used to build the engine frameworks. So this is blocked on #91634. Which is blocked on getting the cipd package to also include all the simulator runtimes needed to pass the framework devicelab tests. I just need to keep banging on it.

This will become much easier once #91689 is done, right now it takes many hours to validate.

@jmagman
Copy link
Member Author

jmagman commented Nov 5, 2021

Old version of clang can't parse the iOS 15 headers with new _Nullable_result nullability annotation

This one might go away when I upgrade the builders to the version of clang in Xcode 12.3 #84453 (comment), I need to check though.

@chinmaygarde chinmaygarde added the P2 Important issues not at the top of the work list label Nov 8, 2021
@jmagman
Copy link
Member Author

jmagman commented Dec 6, 2021

For some reason, the Mac Web Engine builder can't find any iOS simulator runtimes (note unavailable, runtime profile not found):

== Device Types ==
iPhone 4s (com.apple.CoreSimulator.SimDeviceType.iPhone-4s)
...
iPad mini (6th generation) (com.apple.CoreSimulator.SimDeviceType.iPad-mini-6th-generation)
== Runtimes ==
== Devices ==
-- Unavailable: com.apple.CoreSimulator.SimRuntime.iOS-13-0 --
    iPhone 8 (19B472BA-C0FF-45EE-8CEF-9C6DDA134F18) (Shutdown) (unavailable, runtime profile not found)
    iPhone 8 Plus (5828F22B-BA3B-443D-B554-76FD75C67AF4) (Shutdown) (unavailable, runtime profile not found)
    iPhone 11 (DF95828D-5A62-4198-9214-67F1E3CE7A07) (Shutdown) (unavailable, runtime profile not found)

https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/magder_google.com/b6b2af5e756fd2f481aad07ffaa7d630973d972532a1139e128429a681287595/+/build.proto?server=chromium-swarm.appspot.com

Mac Unopt also cannot find available simulators: https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/magder_google.com/330b67ac0733ed4e15440038e105b2621336d5f14c27a136641b9ed8f67409ce/+/build.proto?server=chromium-swarm.appspot.com

But the devicelab staging builders using the same Xcode cipd bundle show available runtimes, and tests are passing on simulators:

== Device Types ==
iPhone 4s (com.apple.CoreSimulator.SimDeviceType.iPhone-4s)
...
iPad mini (6th generation) (com.apple.CoreSimulator.SimDeviceType.iPad-mini-6th-generation)
== Runtimes ==
iOS 13.0 (13.0 - 17A577) - com.apple.CoreSimulator.SimRuntime.iOS-13-0
iOS 13.5 (13.5 - 17F61) - com.apple.CoreSimulator.SimRuntime.iOS-13-5
iOS 15.0 (15.0 - 19A339) - com.apple.CoreSimulator.SimRuntime.iOS-15-0
watchOS 5.0 (5.0 - 16R363) - com.apple.CoreSimulator.SimRuntime.watchOS-5-0 (unavailable, The watchOS 5.0 simulator runtime is not supported on hosts after macOS 11.99.0.)
watchOS 6.1 (6.1.1 - 17S445) - com.apple.CoreSimulator.SimRuntime.watchOS-6-1
== Devices ==
-- iOS 13.0 --
    iPhone 8 (B8A23A9E-4189-421B-8AFA-0A5E907840CA) (Shutdown) 
    iPhone 8 Plus (2CF97D94-C0F9-42E1-89FB-E1ECEC1674DF) (Shutdown) 
    iPhone 11 (79A9EB55-F6BB-44EC-9CCB-45CD7F9F965B) (Shutdown)

https://luci-milo.appspot.com/p/flutter/builders/staging/Mac_ios_staging%20flavors_test_ios/679

@godofredoc are you aware of any additional Xcode/simulator/caching priming the devicelab recipes do that the engine do not?

@jmagman
Copy link
Member Author

jmagman commented Dec 6, 2021

The failing engine builders do not even show iOS 15.0 devices at all, even as unavailable. It does show iPhone 13.

@godofredoc
Copy link
Contributor

The linked task ran on a devicelab machine. Devicelab downloads xcode once and caches them indefinitely. Does running the task on a different devicelab machine works properly?

@jmagman
Copy link
Member Author

jmagman commented Dec 6, 2021

@godofredoc When I download the cipd bundle I only see the iOS 15 simulator runtime in Contents/Developer/Platforms/iPhoneOS.platform/Library/Developer/CoreSimulator/Profiles/Runtimes/ even though the one I uploaded contains iOS 13, 13.6, 14, and 15.

I bet XcodeIOSSimulatorRuntimeRelPath needs to be excluded in the custom go script?
go/flutter-infra-xcode#creating-new-versions-of-the-mac-toolchain

Or we need to do #91689 to get the necessary runtimes installed.

@jmagman
Copy link
Member Author

jmagman commented Dec 7, 2021

It's passed on flutter-devicelab-mac-10, flutter-devicelab-mac-26, and flutter-devicelab-mac-30.

Actually that particular test is only building the app, not running tests on the simulator.
module_test_ios is running and passing running tests on a simulator, and only shows the iOS 15 runtime available, which makes sense based on the downloaded cipd package.

Executing: xcrun simctl list runtimes in /opt/s/w/ir/x/w/recipe_cleanup/tmp742h195q/flutter sdk with environment {BOT: true, LANG: en_US.UTF-8}
== Runtimes ==
iOS 15.0 (15.0 - 19A339) - com.apple.CoreSimulator.SimRuntime.iOS-15-0
"xcrun" exit code: 0

https://logs.chromium.org/logs/flutter/led/magder_google.com/ed6964ff8e4b372114f572353080765155c66c27ef0527e7d99454b7b83e55c8/+/u/run_module_test_ios/test_stdout

I don't understand why IosUnitTests can't find a simulator then for -destination "platform=iOS Simulator,name=iPhone 11" since that resolves to iOS 15 SDKROOT = iphonesimulator15.0, and iPhone 11 is available in the downloaded cipd package, and Contents/Developer/Platforms/iPhoneOS.platform/Library/Developer/CoreSimulator/Profiles/DeviceTypes/ contains iPhone 11.simdevicetype. The module_test_ios log shows an available iOS 15, iPhone 11 simulator:

{ platform:iOS Simulator, id:608E7473-09A7-4F41-AC12-0E673AEEF710, OS:15.0, name:iPhone 11 }

@jmagman
Copy link
Member Author

jmagman commented Dec 7, 2021

Filed #94780

@jmagman
Copy link
Member Author

jmagman commented Dec 15, 2021

I believe #91689 is the last piece to getting the tests to pass. Mac Unopt and Mac Web Engine need to be able to run tests on simulators versions other than the default one (iOS 15) shipped in Xcode 13.

@juliavi
Copy link

juliavi commented Dec 29, 2021

any update? ☺️

@jmagman
Copy link
Member Author

jmagman commented Jan 5, 2022

I believe #91689 is the last piece to getting the tests to pass. Mac Unopt and Mac Web Engine need to be able to run tests on simulators versions other than the default one (iOS 15) shipped in Xcode 13.

🙂

@jmagman
Copy link
Member Author

jmagman commented Jan 22, 2022

@jmagman jmagman added P1 High-priority issues at the top of the work list and removed P2 Important issues not at the top of the work list labels Jan 26, 2022
@anuccio15
Copy link

Any update on this? Reaaalllllyyyyy need flutter/engine#29797 to be able to be merged sometime soon here

@jmagman
Copy link
Member Author

jmagman commented Feb 18, 2022

We hit an issue running web tests on the version of macOS needed by the newer version of Xcode, working on upgrading the version of Chrome here #98278.

@eric8810
Copy link

eric8810 commented Mar 7, 2022

We hit an issue running web tests on the version of macOS needed by the newer version of Xcode, working on upgrading the version of Chrome here #98278.

looks like it was closed

@jmagman
Copy link
Member Author

jmagman commented Mar 7, 2022

We hit an issue running web tests on the version of macOS needed by the newer version of Xcode, working on upgrading the version of Chrome here #98278.

looks like it was closed

Yes, we're chipping away at all these issues. The Chrome package issues was blocking #94078, we're working on that next.

@jmagman
Copy link
Member Author

jmagman commented Mar 9, 2022

The framework builders have been updated to Xcode 13 as of #98297
The flutter doctor enforced tool minimum was updated in #98297 (a dependency to this change to avoid App Store submission failures related to the bitcode version used to build the engine framework)

Found a new clang error introduced since the last time I tested this in January #99795

We're working on getting the builders to be able to cache multiple versions of the iOS simulator runtimes #99698

Once all the tests in flutter/engine#31915 pass we can merge it, and that will make the iOS 15 SDK available and will close this issue.

@jmagman
Copy link
Member Author

jmagman commented Mar 10, 2022

Clang issue resolved. Caching multiple versions of the simulator also resolved.
Now hitting an issue running the web tests in Safari on the simulator #99885

@jmagman
Copy link
Member Author

jmagman commented Mar 29, 2022

iOS 15 SDK and simulators are available in the engine as of flutter/engine#31915

@github-actions
Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 12, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
engine flutter/engine repository. See also e: labels. P1 High-priority issues at the top of the work list platform-ios iOS applications specifically team Infra upgrades, team productivity, code health, technical debt. See also team: labels. team-infra Owned by Infrastructure team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants