- Proposal: SE-NNNN
- Author(s): Matthew Johnson
- Status: Review
- Review manager: TBD
Automatic protocol forwarding introduces the ability to use delegation without the need write forwarding member implementations manually.
A preliminary mailing list thread on this topic had the subject protocol based invocation forwarding
Swift-evolution thread: Proposal Draft: automatic protocol forwarding
Delegation is a robust, composition oriented design technique that keeps interface and implementation inheritance separate. The primary drawback to this technique is that it requires a lot of manual boilerplate to forward implemenation to the implementing member. This proposal eliminates the need to write such boilerplate manually, thus making delegation-based designs much more convenient and attractive.
This proposal may also serve as the foundation for a future enhancement allowing a very concise "newtype" declaration. In the meantime, it facilitates similar functionality, although in a slightly more verbose manner.
Several examples follow.
The first two show how this proposal could improve how forwarding is implemented by the lazy collection subsystem of the standard library. This makes an interesting case study as each example employs a different forwarding mechanism.
The relevant portion of the current implementation of LazySequence
looks like this (with comments removed and formatting tweaks):
// in SequenceWrapper.swift:
public protocol _SequenceWrapperType {
typealias Base : SequenceType
typealias Generator : GeneratorType = Base.Generator
var _base: Base {get}
}
extension SequenceType
where Self : _SequenceWrapperType, Self.Generator == Self.Base.Generator {
public func generate() -> Base.Generator {
return self._base.generate()
}
public func underestimateCount() -> Int {
return _base.underestimateCount()
}
@warn_unused_result
public func map<T>(
@noescape transform: (Base.Generator.Element) throws -> T
) rethrows -> [T] {
return try _base.map(transform)
}
@warn_unused_result
public func filter(
@noescape includeElement: (Base.Generator.Element) throws -> Bool
) rethrows -> [Base.Generator.Element] {
return try _base.filter(includeElement)
}
public func _customContainsEquatableElement(
element: Base.Generator.Element
) -> Bool? {
return _base._customContainsEquatableElement(element)
}
public func _preprocessingPass<R>(@noescape preprocess: (Self) -> R) -> R? {
return _base._preprocessingPass { _ in preprocess(self) }
}
public func _copyToNativeArrayBuffer()
-> _ContiguousArrayBuffer<Base.Generator.Element> {
return _base._copyToNativeArrayBuffer()
}
public func _initializeTo(ptr: UnsafeMutablePointer<Base.Generator.Element>)
-> UnsafeMutablePointer<Base.Generator.Element> {
return _base._initializeTo(ptr)
}
}
// in LazySequence.swift:
public struct LazySequence<Base : SequenceType>
: LazySequenceType, _SequenceWrapperType {
public init(_ base: Base) {
self._base = base
}
public var _base: Base
public var elements: Base { return _base }
}
LazySequence
is using the approach to forwarding mentioned by Kevin Ballard on the mailing list in response to this proposal. This approach has several deficiencies that directly impact LazySequence
:
-
LazySequence
must publicly expose implementation details. Both its_base
property as well as its conformance to_SequenceWrapperType
. -
The forwarding members must be manually implemented. They are trivial, but mistakes are still possible. In this case,
@warn_unused_result
is missing in some places where it should probably be specified (and would be synthesized using the approach in this proposal due to its presence in the protocol member declarations). -
It is not immediately apparent that
_SequenceWrapperType
and the corresponding extension only provide forwarding members. Even if the name clearly indicates that it is possible that the code does something different. It is possible for somebody to come along after the initial implementation and add a new method that does something other than simple forwarding. -
Because the forwarding is implemented via a protocol extension as default methods it can be overriden by an extension on
LazySequence
.
Here is an alternative implemented using the current proposal:
// _LazySequenceForwarding redeclares the subset of the members of SequenceType we wish to forward.
// The protocol is an implementation detail and is marked private.
private protocol _LazySequenceForwarding {
typealias Generator : GeneratorType
@warn_unused_result
func generate() -> Generator
@warn_unused_result
func underestimateCount() -> Int
@warn_unused_result
func map<T>(
@noescape transform: (Generator.Element) throws -> T
) rethrows -> [T]
@warn_unused_result
func filter(
@noescape includeElement: (Generator.Element) throws -> Bool
) rethrows -> [Generator.Element]
@warn_unused_result
func _customContainsEquatableElement(
element: Generator.Element
) -> Bool?
func _copyToNativeArrayBuffer() -> _ContiguousArrayBuffer<Generator.Element>
func _initializeTo(ptr: UnsafeMutablePointer<Generator.Element>)
-> UnsafeMutablePointer<Generator.Element>
}
public struct LazySequence<Base : SequenceType> : LazySequenceType {
public init(_ base: Base) {
self._base = base
}
// NOTE: _base is now internal
internal var _base: Base
public var elements: Base { return _base }
public forward _LazySequenceForwarding to _base
// The current proposal does not currently support forwarding
// of members with nontrivial Self requirements.
// Because of this _preprocessingPass is forwarded manually.
// A future enhancement may be able to support automatic
// forwarding of protocols with some or all kinds of
// nontrivial Self requirements.
public func _preprocessingPass<R>(@noescape preprocess: (Self) -> R) -> R? {
return _base._preprocessingPass { _ in preprocess(self) }
}
}
This example takes advantage of a very important aspect of the design of this proposal. Neither Base
nor LazySequence
are required to conform to _LazySequenceForwarding
. The only requirement is that Base
contains the members specified in _LazySequenceForwarding
as they will be used in the synthesized forwarding implementations.
The relaxed requirement is crucial to the application of the protocol forwarding feature in this implementation. We cannot conform Base
to _LazySequenceForwarding
. If it were possible to conform one protocol to another we could conform SequenceType
to _LazySequenceForwarding
, however it is doubtful that we would want that conformance. Despite this, it is clear to the compiler that Base
does contain the necessary members for forwarding as it conforms to LazySequence
which also declares all of the necessary members.
This implementation is more robust and more clear:
-
We no longer leak any implementation details.
-
There is no chance of making a mistake in the implementation of the forwarded members. It is possible that a mistake could be made in the member declarations in
_LazySequenceForwarding
. However, if a mistake is made there a compiler error will result. -
The set of forwarded methods is immediately clear, with the exception of
_preprocessingPass
because of its nontrivialSelf
requirement. Removing the limitation on nontrivialSelf
requirements is a highly desired improvement to this proposal or future enhancement to this feature. -
The forwarded members cannot be overriden in an extension on
LazySequence
. If somebody attempts to do so it will result in an ambiguous use error at call sites.
The relevant portion of the current implementation of LazyCollection
looks like this (with comments removed and formatting tweaks):
// in LazyCollection.swift:
public struct LazyCollection<Base : CollectionType>
: LazyCollectionType {
public typealias Elements = Base
public var elements: Elements { return _base }
public typealias Index = Base.Index
public init(_ base: Base) {
self._base = base
}
internal var _base: Base
}
extension LazyCollection : SequenceType {
public func generate() -> Base.Generator { return _base.generate() }
public func underestimateCount() -> Int { return _base.underestimateCount() }
public func _copyToNativeArrayBuffer()
-> _ContiguousArrayBuffer<Base.Generator.Element> {
return _base._copyToNativeArrayBuffer()
}
public func _initializeTo(
ptr: UnsafeMutablePointer<Base.Generator.Element>
) -> UnsafeMutablePointer<Base.Generator.Element> {
return _base._initializeTo(ptr)
}
public func _customContainsEquatableElement(
element: Base.Generator.Element
) -> Bool? {
return _base._customContainsEquatableElement(element)
}
}
extension LazyCollection : CollectionType {
public var startIndex: Base.Index {
return _base.startIndex
}
public var endIndex: Base.Index {
return _base.endIndex
}
public subscript(position: Base.Index) -> Base.Generator.Element {
return _base[position]
}
public subscript(bounds: Range<Index>) -> LazyCollection<Slice<Base>> {
return Slice(base: _base, bounds: bounds).lazy
}
public var isEmpty: Bool {
return _base.isEmpty
}
public var count: Index.Distance {
return _base.count
}
public func _customIndexOfEquatableElement(
element: Base.Generator.Element
) -> Index?? {
return _base._customIndexOfEquatableElement(element)
}
public var first: Base.Generator.Element? {
return _base.first
}
}
LazyCollection
is using direct manual implementations of forwarding methods. It corresponds exactly to implementations that would be synthesized by the compiler under this proposal. This approach avoids some of the problems with the first approach:
-
It does not leak implementation details. This is good!
-
The forwarded members cannot be overriden.
Unfortunately it still has some drawbacks:
-
It is still possible to make mistakes in the manual forwarding implementations.
-
The set of forwarded methods is even less clear than under the first approach as they are now potentially interspersed with custom, nontrivial member implementations, such as
subscript(bounds: Range<Index>) -> LazyCollection<Slice<Base>>
in this example. -
This approach requires reimplementing the forwarded members in every type which forwards them and is therefore less scalable than the first approach and this proposal. This may not matter for
LazyCollection
but it may well matter in other cases.
One intersting difference to note between LazySequence
and LazyCollection
is that LazySequence
forwards three members which LazyCollection
does not: map
, filter
, and _preprocessingPass
. It is unclear whether this difference is intentional or not.
This difference is particularly interesting in the case of _preprocessingPass
. LazyCollection
appears to be using the default implementation for CollectionType
in Collection.swift, which results in _base._preprocessingPass
not getting called. It is not apparent why this behavior would be correct for LazyCollection
and not for LazySequence
.
I wonder if the difference in forwarded members is partly due to the fact that the set of forwarded members is not as clear as it could be.
Here is an alternate approach implemented using the current proposal. It assumes that the same SequenceType
members that are forwarded by LazySequence
should also be forwarded by LazyCollection
, allowing us to reuse the _LazySequenceForwarding
protocol declared in the first example.
// _LazyCollectionForwarding redeclares the subset of the members of Indexable and CollectionType we wish to forward.
// The protocol is an implementation detail and is marked private.
private protocol _LazyCollectionForwarding: _LazySequenceForwarding {
typealias Index : ForwardIndexType
var startIndex: Index {get}
var endIndex: Index {get}
typealias _Element
subscript(position: Index) -> _Element {get}
var isEmpty: Bool { get }
var count: Index.Distance { get }
var first: Generator.Element? { get }
@warn_unused_result
func _customIndexOfEquatableElement(element: Generator.Element) -> Index??
}
public struct LazyCollection<Base : CollectionType>
: LazyCollectionType {
public typealias Elements = Base
public var elements: Elements { return _base }
public init(_ base: Base) {
self._base = base
}
internal var _base: Base
public forward _LazyCollectionForwarding to _base
// It may be the case that LazyCollection should forward _preprocessingPass
// in the same fashion that LazySequence uses, which cannot yet be automated
// under the current proposal.
}
extension LazyCollection : CollectionType {
// This implementation is nontrivial and thus not forwarded
public subscript(bounds: Range<Index>) -> LazyCollection<Slice<Base>> {
return Slice(base: _base, bounds: bounds).lazy
}
}
This approach to forwarding does not exhibit any of the issues with the manual approach and only takes about half as much code now that we are able to reuse the previous declaration of _LazySequenceForwarding
.
NOTE: LazyMapCollection
in Map.swift uses the same manual forwarding approach as LazyCollection
to forward a handful of members and would therefore also be a candidate for adopting the new forwarding mechanism as well.
I propose introducing a forward
declaration be allowed within a type declaration or type extension. The forward
declaration will cause the compiler to synthesize implementations of the members required by the forwarded protocols. The synthesized implementations will simply forward the method call to the specified member.
The basic declaration looks like this:
forward Protocol, OtherProtocol to memberIdentifier
The first clause contains a list of protocols to forward.
The second clause specifies the identifier of the property to which the protocol members will be forwarded. Any visible property that implements the members required by the protocol is eligible for forwarding. It does not matter whether it is stored, computed, lazy, etc.
It is also possible to include an access control declaration modifier to specify the visibility of the synthesized members.
When a protocol member includes a Self
parameter forwarding implementations must accept the forwarding type but supply an argument of the forwardee type when making the forwarding call. The most straightforward way to do this is to simply use the same property getter that is used when forwarding. This is the proposed solution.
When a protocol member includes a Self
return type forwarding implementations must return the forwarding type. However, the forwardee implmentation will return a value of the forwardee type. This result must be used to produce a value of the forwarding type in some way.
The solution in this proposal is based on an ad-hoc overloading convention. A protocol-based solution would probably be desirable if it were possible, however it is not. This proposal supports forwarding to more than one member, possibly with different types. A protocol-based solution would require the forwarding type to conform to the "Self
return value conversion" protocol once for each forwardee type.
When a forwardee value is returned from a static member an initializer will be used to produce a final return value. The initializer must be visible at the source location of the forward
declaration and must look like this:
struct Forwarder {
let forwardee: Forwardee
forward P to forwardee
init(_ forwardeeReturnValue: Forwardee) { //... }
}
When a forwardee value is returned from an instance member an instance method will be used to transform the return value into a value of the correct type. An instance method is necessary in order to allow the forwarding type to access the state of the instance upon which the method was called when performing the transformation.
If the instance method is not implemented the initializer used for static members will be used instead.
The transformation has the form:
struct Forwarder {
let forwardee: Forwardee
forward P to forwardee
func transformedForwardingReturnValue(forwardeeReturnValue: Forwardee) -> Forwarder { //... }
}
NOTE: This method should have a better name. Suggestions are appreciated!
NOTE: Forwardee
does not actually conform to P
itself. Conformance is not required to synthesize the forwarding member implementations. It is only required that members necessary for forwarding exist. This is particularly important to the second example.
public protocol P {
typealias TA
var i: Int
func foo() -> Bool
}
private struct Forwardee {
typealias TA = String
var i: Int = 42
func foo() -> Bool { return true }
}
public struct Forwarder {
private let forwardee: Forwardee
}
extension Forwarder: P {
// user declares
public forward P to forwardee
// compiler synthesizes
// TA must be synthesized as it cannot be inferred for this protocol
public typealias TA = String
public var i: Int {
get { return forwardee.i }
set { forwardee.i = newValue }
}
public func foo() -> Bool {
return forwardee.foo()
}
}
NOTE: Existentials of type P
do not actually conform to P
itself. Conformance is not required to synthesize the forwarding member implementations. It is only required that members necessary for forwarding exist.
public protocol P {
func foo() -> Bool
}
struct S: P {
private let p: P
// user declares:
forward P to p
// compiler synthesizes:
func foo() -> Bool {
return p.foo()
}
}
public protocol P {
func foo(value: Self) -> Bool
}
extension Int: P {
func foo(value: Int) -> Bool {
return value != self
}
}
struct S: P {
private let i: Int
// user declares:
forward P to i
// compiler synthesizes:
func foo(value: S) -> Bool {
return i.foo(value.i)
}
}
Using the instance method:
public protocol P {
func foo() -> Self
}
extension Int: P {
func foo() -> Int {
return self + 1
}
}
struct S: P {
private let i: Int
func transformedForwardingReturnValue(forwardeeReturnValue: Int) -> S {
return S(i: forwardeeReturnValue)
}
// user declares:
forward P to i
// compiler synthesizes:
func foo() -> S {
return self.transformedForwardingReturnValue(i.foo())
}
}
Using the initializer:
public protocol P {
func foo() -> Self
}
extension Int: P {
func foo() -> Int {
return self + 1
}
}
struct S: P {
private let i: Int
init(_ value: Int) {
i = value
}
// user declares:
forward P to i
// compiler synthesizes:
func foo() -> S {
return S(i.foo())
}
}
public protocol P {
func foo() -> Bool
}
public protocol Q {
func bar() -> Bool
}
extension Int: P, Q {
func foo() -> Bool {
return true
}
func bar() -> Bool {
return false
}
}
struct S: P, Q {
private let i: Int
// user declares:
forward P, Q to i
// compiler synthesizes:
func foo() -> Bool {
return i.foo()
}
func bar() -> Bool {
return i.bar()
}
}
public protocol P {
func foo() -> Bool
}
public protocol Q {
func bar() -> Bool
}
extension Int: P {
func foo() -> Bool {
return true
}
}
extension Double: Q {
func bar() -> Bool {
return false
}
}
struct S: P, Q {
private let i: Int
private let d: Double
// user declares:
forward P to i
forward Q to d
// compiler synthesizes:
func foo() -> Bool {
return i.foo()
}
func bar() -> Bool {
return d.bar()
}
}
NOTE: C
cannot declare conformance to the protocol due to the Self
return value requirement. However, the compiler still synthesizes the forwarding methods and allows them to be used directly by users of C
.
public protocol P {
func foo() -> Self
}
extension Int: P {
func foo() -> Int {
return self + 1
}
}
// C does not and cannot declare conformance to P
class C {
private let i: Int
init(_ value: Int) {
i = value
}
// user declares:
forward P to i
// compiler synthesizes:
func foo() -> C {
return C(i.foo())
}
}
TODO: grammar modification to add the forward
declaration
-
Automatic forwarding only synthesizes member implementations. It does not automatically conform the forwarding type to the protocol(s) that are forwarded. If actual conformance is desired (as it usually will be) it must be explicitly stated.
-
The forwardee type need not actually conform to the protocol forwarded to it. It only needs to implement the members the forwarder must access in the synthesized forwarding methods. This is particularly important as long as protocol existentials do not conform to the protocol itself.
-
While it will not be possible to conform non-final classes to protocols containing a
Self
return type forwarding should still be allowed. The synthesized methods will have a return type of the non-final class which in which the forwarding declaration occured. The synthesized methods may still be useful in cases where actual protocol conformance is not necessary. -
All synthesized members recieve access control modifiers matching the access control modifier applied to the
forward
declaration. -
TODO: How should other annotations on the forwardee implementations of forwarded members (such as @warn_unused_result) be handled?
-
It is possible that the member implementations synthesized by forwarding will conflict with existing members or with each other (when forwarding more than one protocol). All such conflicts, with one exception, should produce a compiler error at the site of the forwarding declaration which resulted in conflicting members.
One specific case that should not be considered a conflict is when forwarding more than one protocol with identical member declarations to the same member of the forwarding type. In this case the synthesized implementation required to forward all of the protocols is identical. The compiler should not synthesize multiple copies of the implementation and then report a redeclaration error. -
It is likely that any attempt to forward different protocols with
Self
return types to more than one member of the same type will result in sensible behavior. This should probable be a compiler error. For example:
protocol P {
func foo() -> Self
}
protocol Q {
func bar() -> Self
}
struct Forwarder: P, Q {
let d1: Double
let d2: Double
forward P to d1
forward Q to d2
func transformedForwardingReturnValue(_ forwardeeReturnValue: Double) -> Forwarder {
// What do we do here?
// We don't know if the return value resulted from forwarding foo to d1 or bar to d2.
// It is unlikely that the same behavior is correct in both cases.
}
}
This is a strictly additive change. It has no impact on existing code.
In the spirit of incremental change, this proposal focuses on core functionality. Several enhancements to the core functionality are possible and are likely to be explored in the future.
The current proposal makes automatic forwarding an "all or nothing" feature. In cases where you want to forward most of the implementation of a set of members but would need to "override" one or more specific members the current proposal will not help. You will still be required to forward the entire protocol manually. Attempting to implement some specific members manually will result in a redeclaration error.
This proposal does not allow partial forwarding synthesis in order to focus on the basic forwarding mechanism and allow us to gain some experience with that first, before considering the best way to make partial forwarding possible without introducing unintended potential for error. One example of a consideration that may apply is whether or not forwardee types should be able to mark members as "final for forwarding" in some way that prevents them from being "overriden" by a forwarder.
While the current proposal provides the basic behavior desired for newtype
, it is not as concise as it could be. Adding syntactic sugar to make this common case more concise would be straightforward:
// user declares
newtype Weight = Double forwarding P, Q
// compiler synthesizes
struct Weight: P, Q {
var value: Double
forward P, Q to value
init(_ value: Double) { self.value = value }
}
However, there are additional nuances related to associated types that should be considered and addressed by a newtype
proposal.
It may be possible to allow the forward
declaration in protocol extensions by forwarding to a required property of the protocol. This may have implementation complexities and other implications which would hold back the current proposal if it required the forward
declaration to be allowed in protocol extensions.
I originally thought it would make the most sense to specify forwarding alongside the forwardee member declaration. This proposal does not do so for the following reasons:
-
We must be able to specify access control for the forwarded members that are synthesized. Introducing a forwarding declaration is the most clear way to allow this.
-
It will sometimes be necessary to forward different protocols to the same forwardee with different access control levels. It would be very clunky to do this as part of the member declaration.
-
It should be possible to synthesize forwarding retroactively as part of an extension. This would not be possible if forwarding had to be specified in the original member declaration.
There is not a compelling reason to require this. It is not necessary to synthesize and compile the forwarding methods and it would prevent the use of protocol existentials as the forwardee.
It may seem reasonable to automatically synthesize conformance to the protocol in addition to the member implementations. This proposal does not do so for the following reasons:
-
Forwarding is considered an implementation detail that is not necessarily visible in the public interface of the type. The forwardee may be a private member of the type.
-
Type authors may wish to control where the actual conformance is declared, especially if protocol conformances are allowed to have access control in the future.
-
There may be use cases where it is desirable to have the forwarded members synthesized without actually conforming to the protocol. This is somewhat speculative, but there is not a compelling reason to disallow it.
It may seem reasonable to have a *
placeholder which will forward all visible protocol conformances of the forwardee type. This proposal does not include such a placeholder for the following reasons:
-
A placeholder like this could lead to unintended operations being synthesized if additional conformances are declared in the future. The new conformances could even lead to conflicts during synthesis which cause the code to fail to compile. The potential for such breakage is not acceptable.
-
A placeholder like this would not necessarily cause all desired forwarding methods to be synthesized. This would be the case when the members necessary to conform exist but actual conformance does not exist. This would be the case when the forwardee type is an existential. This could lead to programmer confusion.
-
An explicit list of protocols to forward is not unduely burdensome. It is straightforward to declare a new protocol that inherits from a group of protocols which are commonly forwarded together and use the new protocol in the forwarding declaration.
-
This is easily added as a future enhancement to the current proposal if we later decide it is necessary.
It is impossible to synthesize forwarding of methods which contain the forwardee type as a parameter or return type that are not declared as part of a protocol interface in a correct and safe manner. This is because it may or may not be correct to promote the forwardee type in the signature to the forwarder.
As an example, consider the following extension to Double
. Imagine trying to synthesize a forwarding method in a Pixel
type that forwards to Double
. Should the return type be Pixel
or Double
? It is impossible to tell for sure.
extension Double {
func foo() -> Double {
return self
}
}
When the method is declared in a protocol it becomes obvious what the signature of the forwarding method must be. If the protocol declares the return type as Self
, the forwarding method must have a return type of Pixel
. If the protocol declares the return type as Double
the forwarding method will continue to have a return type of Double
.
It may seem like a good idea to allow synthesized forwarding to Optional
members where a no-op results when the Optional
is nil
. There is no way to make this work in general as it would be impossible to forward any member requiring a return value. If use cases for forwarding to Optional
members emerege that are restricted to protocols with no members requiring return values the automatic protocol forwarding feature could be enhanced in the future to support these use cases.
As with forwarding to Optional
members, forwarding the same protocol to more than one member is not possible in general. However it is possible in cases where no protocol members have a return value. If compelling use cases emerge to motivate automatic forwarding of such protocols to more than one member an enhancement could be proposed in the future.
Some types may be designed to be used as components that are always forwarded to by other types. Such types may wish to be able to communicate with the forwarding type in some way. This can be accomplished manually.
If general patterns emerge in practice it may be possible to add support for them to the language. However, it would be preliminary to consider support for such a feature until we have significant experience with the basic forwarding mechanism itself.