[POC] Extensible protocol conformances attempt III

[johnno1962]

Hi Apple,

This is a third proof of concept PR looking at the possibility of allowing the adding of conformances to protocol extensions as mentioned in the Generics Manifesto (after two previous attempts #29272, #25786). This version now works reliably across module boundaries subject to the known limitation that while you can now do pretty much anything extending conformances in whatever module you like, the conformances to potentially extended protocols of a specific nominal must be in the outermost module. If this limitation is not respected the compiler now reliably flags this as an error as the program could potentially crash at run time. It may be possible to lift this limitation later.

Edit: It’s been possible to lift this limitation at the cost of there now being instances where you could contrive a situation where a program compiles but crashes at run time if you return an existential created in a module that doesn’t have knowledge of the extended conformances to one that does. (See below)

To work across modules, the entries in a witness table for a nominal/protocol pair are extended in a manner that is compatible with the existing ordering to be:

• associated value entries
• pointers to witness tables of directly inherited protocols
• pointer to “FTW” member function thunks of the original protocol for the nominal
• witness tables of extended conformances in order of the module that added the conformance.

This means functions taking an argument of a protocol in a module defining the protocol will continue to operate if an existential from a module that has extended the protocol’s conformances should call that function with an existential.

As adding a conformance to a protocol affects any nominal that has an existing conformance to the protocol being extended, witness tables with the additional entries have to be generated for the nominals when the new conformances are used. As these ad-hoc witness tables can be generated in more than one source file, these witness tables are given internal linker scope. Protocol descriptors are also generated as are entries in the __swift5_proto section of the binary, and as a result dynamically finding these conformances at run time seems to "just work”. Adding a synthesised conformance such as Codable to another protocol also seems to work.

While I don’t imagine this will be merged, it could be as all the advantage of this feature is that it is an additive change not currently permitted in the language and only takes effect when it is used which is only possible when the flag -enable-conforming-protocol-extensions has been provided. Hence, existing tests pass. Having to an extent proved the concept I’ll attempt to re-engage the evolution forums to discuss the idea on the existing thread with a view to a proposal if there is any interest.

Edit: Raised a proposal PR on swift-evolution REPO.

Toolchain for evaluation available here. Test SPM project here.

Addresses: SR-11013

[johnno1962]

The last commit removed the limitation that conformances can not be added to a protocol that a nominal already conforms to in another module by serialising in a way that is backward compatible the "requirement synthetic substitution map" information needed to emit new witness tables. Proposal updated, and new toolchain available here:

http://johnholdsworth.com/swift-LOCAL-2020-06-09-a-osx.tar.gz or from google drive here.

[shahmishal]

Please update the base branch to main by Oct 5th otherwise the pull request will be closed automatically.

• How to change the base branch: (Link)
• More detail about the branch update: (Link)