Handle resilient stored properties in objcImpl #73128

beccadax · 2024-04-19T00:59:40Z

This PR temporarily bans, and lays the groundwork for eventually supporting, resilient stored properties in @objc @implementation classes.

When a stored property contains a resilient value type, its size cannot be determined at compile time. That means the size and exact layout of the instance variables is not known, either. The layout must be computed, and the instance variables and class size data updated, when the class is realized (i.e. at runtime, but before it is used).

For Swift classes (including ordinary @objc classes), the mechanism used to do this works like so:

When the Objective-C runtime realizes a class, it checks a few flags and notices that the class has an "ObjC metadata update function", which it invokes.
The ObjC metadata update function retrieves the Swift type metadata for the class.
If the Swift type metadata is not yet complete, this will cause a "Swift type metadata completion function" to be run.
The Swift type metadata completion function gathers information about the sizes of the types in the stored properties (which are now known since it's runtime) and calls the swift_updateClassMetadata2() function to update the instance variable layout and tell the ObjC runtime to finish realizing the class.

This happens before the Objective-C runtime does its own ivar sliding. In effect, the ivars are slid twice—once to account for changes in the sizes of other ivars ahead of them in the class layout, then again to account for changes in the size of the superclass.

Unfortunately, every single step in this chain is designed for full-fledged Swift classes with Swift metadata; all of them break horribly if you try to use them with a "pure ObjC" class like the ones created by @objc @implementation:

The Objective-C runtime doesn't check for or call the ObjC metadata update function unless the "Swift bit" is set on the class. The Swift bit is normally used to indicate the presence of Swift metadata, so it's not supposed to be set on pure ObjC classes.
Retrieving the Swift type metadata requires use of the type metadata symbol, which is not present for pure ObjC classes.
The Swift type metadata completion function is specified in Swift metadata and there's no way to attach one to a pure ObjC class.
The swift_updateClassMetadata2() function assumes it has been passed a pointer to Swift class metadata and both reads and writes fields that are not present in pure ObjC classes.

This PR lays the groundwork for an alternate mechanism to update the layout of a pure ObjC class during class realization:

The compiler will generate a different ObjC metadata update function which gathers type size information and directly passes it to the runtime, rather than using Swift metadata access mechanisms. This sidesteps problems 2 and 3.
It will call a new runtime entry point, swift_updatePureObjCClassMetadata(), which uses only the Objective-C metadata. This sidesteps problem 4.

However, problem 1 will require changes to the Objective-C runtime, so the new code generation is gated behind an experimental feature flag, ObjCImplementationWithResilientStorage. Without that flag, Swift will unconditionally reject properties that would require this new mechanism; with it, Swift will enforce a (for now) 99.99 minimum deployment target for it.

Fixes rdar://126948464.

Because this test is disabled, a previous commit failed to add an experimental feature flag that’s required to run it.

Rather than having individual methods return without doing anything, arrange for ClassMetadataVisitor to never call them in the first place. This makes it so ClassMetadataScanner also knows which fields are omitted and it can compute offsets correctly. The old logic is still present for field offsets to make this change NFC.

beccadax · 2024-04-24T23:48:40Z

@swift-ci please test

beccadax · 2024-04-26T01:21:54Z

@swift-ci please test

beccadax · 2024-04-26T17:34:20Z

@swift-ci please test

beccadax · 2024-04-26T20:49:48Z

@swift-ci please test

tshortli · 2024-04-30T15:20:06Z

lib/IRGen/GenMeta.cpp

  // Collect the stored properties of the type.
  unsigned numFields = getNumFields(target);

  // Fill out an array with the field type metadata records.
-  Address fields = IGF.createAlloca(
+  Address fields = createAlloca(


Minor: it looks like some code in this file hasn't been reformatted after changes

Yeah, I initially wasn't sure if this change would stick. I should go over this again.

tshortli · 2024-04-30T15:27:32Z

lib/IRGen/GenMeta.cpp

+    // Update and FixedOrUpdate require support in both the Swift and ObjC
+    // runtimes.
+    auto requiredAvailability=ctx.getUpdatePureObjCClassMetadataAvailability();
+    // FIXME: Take the class's availability into account


Out of curiosity, does class metadata layout happen lazily when the first instance is created or something? I'm wondering how gating this on the availability of the class itself would be safe in practice.

The first time the ObjC runtime tries to access any of the metadata associated with a class—basically, use it as anything but a completely opaque pointer—it performs an operation called "realization". The metadata update function that Swift generates will be called during realization, and it will call the new swift_updatePureObjCClassMetadata() function at that time. So realization might happen when you try to look up the +alloc method to allocate an instance of the class, or it could happen even earlier, depending on how the code uses the class.

tshortli · 2024-04-30T15:34:39Z

test/IRGen/objc_implementation.swift

@@ -1,34 +1,34 @@
 // Test doesn't pass on all platforms (rdar://101420862)
 // REQUIRES: OS=macosx

-// RUN: %target-swift-frontend(mock-sdk: %clang-importer-sdk) -enable-experimental-feature CImplementation -I %S/Inputs/abi -F %clang-importer-sdk-path/frameworks %s -import-objc-header %S/Inputs/objc_implementation.h -emit-ir > %t.ir
+// RUN: %target-swift-frontend(mock-sdk: %clang-importer-sdk) -enable-experimental-feature CImplementation -I %S/Inputs/abi -F %clang-importer-sdk-path/frameworks %s -import-objc-header %S/Inputs/objc_implementation.h -emit-ir -target %target-future-triple > %t.ir


Not sure if it would make sense here, but we now have lit substitutions like %target-swift-abi-6.0-triple if you wanted to specify a specific, supported deployment target.

I don't think we technically know yet when this will land, so future availability seems most appropriate for the moment.

@objc

When an @objc @implementation class requires the use of `ClassMetadataStrategy::Update` because some of its stored properties do not have fixed sizes, we adjust the direct field offsets during class realization by emitting a custom metadata update function which calls a new entry point in the Swift runtime. That entry point adjusts field offsets like `swift_updateClassMetadata2()`, but it only assumes that the class has Objective-C metadata, not Swift metadata. This commit introduces an alternative mechanism which does the same thing without using any Swift-only metadata. It’s a rough implementation with important limitations: • We’re currently using the field offset vector, which means that field offsets are being emitted into @objc @implementation classes; these will be removed. • The new Swift runtime entry point duplicates a lot of `swift_updateClassMetadata2()`’s implementation; it will be refactored into something much smaller and more compact. • Availability bounds for this feature have not yet been implemented. Future commits in this PR will correct these issues.

We really don’t need ‘em; we can just adjust the direct field offsets. The runtime entry point currently uses a weird little hack that we will refactor away shortly.

Merge the three-stage operation originally designed for field vectors into a single unified loop that acts directly on the ivar offsets instead of using a faux field offset vector.

@objc

We’re not committing to @objc @implementation back-deploying to pre-stable Apple platforms.

…when the deployment target is not high enough to support them.

Resilence support will require changes to the Objective-C runtime to expand support for metadata initialization functions. Add a separate experimental feature flag to help with staging that support in, and modify diagnostics to not suggest increasing the minimum deployment target for now.

beccadax · 2024-04-30T19:06:11Z

Made some formatting improvements at @tshortli's request. No functional changes.

beccadax · 2024-04-30T19:06:18Z

@swift-ci please smoke test

beccadax · 2024-05-01T00:59:25Z

Windows build timed out.

beccadax · 2024-05-01T00:59:34Z

@swift-ci please smoke test Windows platform

beccadax force-pushed the objcimpl-resilient branch from 10009b0 to b0fe8d4 Compare April 20, 2024 00:26

beccadax added 2 commits April 24, 2024 11:39

Fix objc_implementation_objc_client test

35f05cd

Because this test is disabled, a previous commit failed to add an experimental feature flag that’s required to run it.

beccadax force-pushed the objcimpl-resilient branch from b0fe8d4 to 6a44e8f Compare April 24, 2024 19:22

beccadax changed the title ~~Support resilient stored properties in objcImpl~~ Handle resilient stored properties in objcImpl Apr 24, 2024

beccadax force-pushed the objcimpl-resilient branch 3 times, most recently from d1e9c47 to 9f4f051 Compare April 24, 2024 23:47

beccadax marked this pull request as ready for review April 24, 2024 23:48

beccadax requested review from hborla, slavapestov, a team, mikeash, al45tair, xedin and tshortli as code owners April 24, 2024 23:48

beccadax requested review from ahoppen, bnbarham, hamishknight and rintaro as code owners April 26, 2024 17:26

beccadax force-pushed the objcimpl-resilient branch from 175c244 to fb60673 Compare April 26, 2024 17:33

beccadax force-pushed the objcimpl-resilient branch from fb60673 to 64974f4 Compare April 26, 2024 20:49

ahoppen removed their request for review April 26, 2024 22:04

beccadax mentioned this pull request Apr 27, 2024

Change objcImpl syntax to @objc @implementation #73309

Merged

tshortli approved these changes Apr 30, 2024

View reviewed changes

beccadax added 7 commits April 30, 2024 12:03

Eliminate objcImpl field offset vectors

d1f14e5

We really don’t need ‘em; we can just adjust the direct field offsets. The runtime entry point currently uses a weird little hack that we will refactor away shortly.

[NFC] Refactor swift_updatePureObjCClassMetadata()

915a49a

Merge the three-stage operation originally designed for field vectors into a single unified loop that acts directly on the ivar offsets instead of using a faux field offset vector.

Set a minimum deployment target for objcImpl

4578be2

We’re not committing to @objc @implementation back-deploying to pre-stable Apple platforms.

Diagnose resilient properties in objcImpl

1dceb24

…when the deployment target is not high enough to support them.

Add target to failing objcImpl test

2947e6c

Fix -target in several objcImpl tests

6d3e1ad

beccadax force-pushed the objcimpl-resilient branch from 64974f4 to 6d3e1ad Compare April 30, 2024 19:04

beccadax enabled auto-merge April 30, 2024 19:06

beccadax merged commit 981233a into apple:main May 1, 2024
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handle resilient stored properties in objcImpl #73128

Handle resilient stored properties in objcImpl #73128

beccadax commented Apr 19, 2024 •

edited

beccadax commented Apr 24, 2024

beccadax commented Apr 26, 2024

beccadax commented Apr 26, 2024

beccadax commented Apr 26, 2024

tshortli Apr 30, 2024

beccadax Apr 30, 2024

tshortli Apr 30, 2024

beccadax Apr 30, 2024 •

edited

tshortli Apr 30, 2024

beccadax Apr 30, 2024

beccadax commented Apr 30, 2024

beccadax commented Apr 30, 2024

beccadax commented May 1, 2024

beccadax commented May 1, 2024

Handle resilient stored properties in objcImpl #73128

Handle resilient stored properties in objcImpl #73128

Conversation

beccadax commented Apr 19, 2024 • edited

beccadax commented Apr 24, 2024

beccadax commented Apr 26, 2024

beccadax commented Apr 26, 2024

beccadax commented Apr 26, 2024

tshortli Apr 30, 2024

Choose a reason for hiding this comment

beccadax Apr 30, 2024

Choose a reason for hiding this comment

tshortli Apr 30, 2024

Choose a reason for hiding this comment

beccadax Apr 30, 2024 • edited

Choose a reason for hiding this comment

tshortli Apr 30, 2024

Choose a reason for hiding this comment

beccadax Apr 30, 2024

Choose a reason for hiding this comment

beccadax commented Apr 30, 2024

beccadax commented Apr 30, 2024

beccadax commented May 1, 2024

beccadax commented May 1, 2024

beccadax commented Apr 19, 2024 •

edited

beccadax Apr 30, 2024 •

edited