diff --git a/docs/SIL.rst b/docs/SIL.rst index c40f0fc2d14ea..53ed7e97209fb 100644 --- a/docs/SIL.rst +++ b/docs/SIL.rst @@ -2107,6 +2107,94 @@ make the use of such types more convenient; it does not shift the ultimate responsibility for assuring the safety of unsafe language/library features away from the user. +Copy-on-Write Representation +---------------------------- + +Copy-on-Write (COW) data structures are implemented by a reference to an object +which is copied on mutation in case it's not uniquely referenced. + +A COW mutation sequence in SIL typically looks like:: + + (%uniq, %buffer) = begin_cow_mutation %immutable_buffer : $BufferClass + cond_br %uniq, bb_uniq, bb_not_unique + bb_uniq: + br bb_mutate(%buffer : $BufferClass) + bb_not_unique: + %copied_buffer = apply %copy_buffer_function(%buffer) : ... + br bb_mutate(%copied_buffer : $BufferClass) + bb_mutate(%mutable_buffer : $BufferClass): + %field = ref_element_addr %mutable_buffer : $BufferClass, #BufferClass.Field + store %value to %field : $ValueType + %new_immutable_buffer = end_cow_mutation %buffer : $BufferClass + +Loading from a COW data structure looks like:: + + %field1 = ref_element_addr [immutable] %immutable_buffer : $BufferClass, #BufferClass.Field + %value1 = load %field1 : $*FieldType + ... + %field2 = ref_element_addr [immutable] %immutable_buffer : $BufferClass, #BufferClass.Field + %value2 = load %field2 : $*FieldType + +The ``immutable`` attribute means that loading values from ``ref_element_addr`` +and ``ref_tail_addr`` instructions, which have the *same* operand, are +equivalent. +In other words, it's guaranteed that a buffer's properties are not mutated +between two ``ref_element/tail_addr [immutable]`` as long as they have the +same buffer reference as operand. +This is even true if e.g. the buffer 'escapes' to an unknown function. + + +In the example above, ``%value2`` is equal to ``%value1`` because the operand +of both ``ref_element_addr`` instructions is the same ``%immutable_buffer``. +Conceptually, the content of a COW buffer object can be seen as part of +the same *static* (immutable) SSA value as the buffer reference. + +The lifetime of a COW value is strictly separated into *mutable* and +*immutable* regions by ``begin_cow_mutation`` and +``end_cow_mutation`` instructions:: + + %b1 = alloc_ref $BufferClass + // The buffer %b1 is mutable + %b2 = end_cow_mutation %b1 : $BufferClass + // The buffer %b2 is immutable + (%u1, %b3) = begin_cow_mutation %b1 : $BufferClass + // The buffer %b3 is mutable + %b4 = end_cow_mutation %b3 : $BufferClass + // The buffer %b4 is immutable + ... + +Both, ``begin_cow_mutation`` and ``end_cow_mutation``, consume their operand +and return the new buffer as an *owned* value. +The ``begin_cow_mutation`` will compile down to a uniqueness check and +``end_cow_mutation`` will compile to a no-op. + +Although the physical pointer value of the returned buffer reference is the +same as the operand, it's important to generate a *new* buffer reference in +SIL. It prevents the optimizer from moving buffer accesses from a *mutable* into +a *immutable* region and vice versa. + +Because the buffer *content* is conceptually part of the +buffer *reference* SSA value, there must be a new buffer reference every time +the buffer content is mutated. + +To illustrate this, let's look at an example, where a COW value is mutated in +a loop. As with a scalar SSA value, also mutating a COW buffer will enforce a +phi-argument in the loop header block (for simplicity the code for copying a +non-unique buffer is not shown):: + + header_block(%b_phi : $BufferClass): + (%u, %b_mutate) = begin_cow_mutation %b_phi : $BufferClass + // Store something to %b_mutate + %b_immutable = end_cow_mutation %b_mutate : $BufferClass + cond_br %loop_cond, exit_block, backedge_block + backedge_block: + br header_block(b_immutable : $BufferClass) + exit_block: + +Two adjacent ``begin_cow_mutation`` and ``end_cow_mutation`` instructions +don't need to be in the same function. + + Instruction Set --------------- @@ -3199,6 +3287,56 @@ strong reference count is greater than 1. A discussion of the semantics can be found here: :ref:`arcopts.is_unique`. +begin_cow_mutation +`````````````````` + +:: + + sil-instruction ::= 'begin_cow_mutation' '[native]'? sil-operand + + (%1, %2) = begin_cow_mutation %0 : $C + // $C must be a reference-counted type + // %1 will be of type Builtin.Int1 + // %2 will be of type C + +Checks whether %0 is a unique reference to a memory object. Returns 1 in the +first result if the strong reference count is 1, and 0 if the strong reference +count is greater than 1. + +Returns the reference operand in the second result. The returned reference can +be used to mutate the object. Technically, the returned reference is the same +as the operand. But it's important that optimizations see the result as a +different SSA value than the operand. This is important to ensure the +correctness of ``ref_element_addr [immutable]``. + +The operand is consumed and the second result is returned as owned. + +The optional ``native`` attribute specifies that the operand has native Swift +reference counting. + +end_cow_mutation +```````````````` + +:: + + sil-instruction ::= 'end_cow_mutation' '[keep_unique]'? sil-operand + + %1 = end_cow_mutation %0 : $C + // $C must be a reference-counted type + // %1 will be of type C + +Marks the end of the mutation of a reference counted object. +Returns the reference operand. Technically, the returned reference is the same +as the operand. But it's important that optimizations see the result as a +different SSA value than the operand. This is important to ensure the +correctness of ``ref_element_addr [immutable]``. + +The operand is consumed and the result is returned as owned. The result is +guaranteed to be uniquely referenced. + +The optional ``keep_unique`` attribute indicates that the optimizer must not +replace this reference with a not uniquely reference object. + is_escaping_closure ``````````````````` @@ -4193,7 +4331,7 @@ ref_element_addr ```````````````` :: - sil-instruction ::= 'ref_element_addr' sil-operand ',' sil-decl-ref + sil-instruction ::= 'ref_element_addr' '[immutable]'? sil-operand ',' sil-decl-ref %1 = ref_element_addr %0 : $C, #C.field // %0 must be a value of class type $C @@ -4205,11 +4343,15 @@ Given an instance of a class, derives the address of a physical instance variable inside the instance. It is undefined behavior if the class value is null. +The ``immutable`` attribute specifies that all loads of the same instance +variable from the same class reference operand are guaranteed to yield the +same value. + ref_tail_addr ````````````` :: - sil-instruction ::= 'ref_tail_addr' sil-operand ',' sil-type + sil-instruction ::= 'ref_tail_addr' '[immutable]'? sil-operand ',' sil-type %1 = ref_tail_addr %0 : $C, $E // %0 must be a value of class type $C with tail-allocated elements $E @@ -4222,6 +4364,10 @@ object which is created by an ``alloc_ref`` with ``tail_elems``. It is undefined behavior if the class instance does not have tail-allocated arrays or if the element-types do not match. +The ``immutable`` attribute specifies that all loads of the same instance +variable from the same class reference operand are guaranteed to yield the +same value. + Enums ~~~~~ diff --git a/include/swift/AST/Builtins.def b/include/swift/AST/Builtins.def index 1510688d37e3d..cd4aa5b46a5b4 100644 --- a/include/swift/AST/Builtins.def +++ b/include/swift/AST/Builtins.def @@ -429,6 +429,29 @@ BUILTIN_SIL_OPERATION(IsUnique, "isUnique", Special) /// BridgeObject to be treated as a native object by the runtime. BUILTIN_SIL_OPERATION(IsUnique_native, "isUnique_native", Special) +/// beginCOWMutation(inout T) -> Int1 +/// +/// Begins a copy-on-write mutation for a buffer reference which is passed as +/// inout argument. It returns a true if the buffer is uniquely referenced. +/// In this case the buffer may be mutated after calling this builtin. +/// +/// The beginCOWMutation builtin is very similar to isUnique. It just translates +/// to a different SIL instruction (begin_cow_mutation), which is the preferred +/// representation of COW in SIL. +BUILTIN_SIL_OPERATION(BeginCOWMutation, "beginCOWMutation", Special) + +/// beginCOWMutation_native(inout T) -> Int1 +/// +/// Like beginCOWMutation, but it's assumed that T has native Swift reference +/// counting. +BUILTIN_SIL_OPERATION(BeginCOWMutation_native, "beginCOWMutation_native", Special) + +/// endCOWMutation(inout T) +/// +/// Ends a copy-on-write mutation for a buffer reference which is passed as +/// inout argument. After calling this builtin, the buffer must not be mutated. +BUILTIN_SIL_OPERATION(EndCOWMutation, "endCOWMutation", Special) + /// bindMemory : (Builtin.RawPointer, Builtin.Word, T.Type) -> () BUILTIN_SIL_OPERATION(BindMemory, "bindMemory", Special) @@ -651,6 +674,12 @@ BUILTIN_MISC_OPERATION(AssignCopyArrayFrontToBack, "assignCopyArrayFrontToBack", BUILTIN_MISC_OPERATION(AssignCopyArrayBackToFront, "assignCopyArrayBackToFront", "", Special) BUILTIN_MISC_OPERATION(AssignTakeArray, "assignTakeArray", "", Special) +/// COWBufferForReading has type T -> T +/// +/// Returns the buffer reference which is passed as argument. +/// This builtin indicates to the optimizer that the buffer is not mutable. +BUILTIN_MISC_OPERATION(COWBufferForReading, "COWBufferForReading", "n", Special) + // unsafeGuaranteed has type T -> (T, Builtin.Int8) BUILTIN_MISC_OPERATION(UnsafeGuaranteed, "unsafeGuaranteed", "", Special) diff --git a/include/swift/AST/Types.h b/include/swift/AST/Types.h index 0bf41995878ab..2ae23c4370b64 100644 --- a/include/swift/AST/Types.h +++ b/include/swift/AST/Types.h @@ -2696,10 +2696,10 @@ enum class FunctionTypeRepresentation : uint8_t { /// A "thin" function that needs no context. Thin, - /// A C function pointer (or reference), which is thin and also uses the C - /// calling convention. + /// A C function pointer, which is thin and also uses the C calling + /// convention. CFunctionPointer, - + /// The value of the greatest AST function representation. Last = CFunctionPointer, }; @@ -2980,8 +2980,8 @@ class AnyFunctionType : public TypeBase { // We preserve a full clang::Type *, not a clang::FunctionType * as: // 1. We need to keep sugar in case we need to present an error to the user. // 2. The actual type being stored is [ignoring sugar] either a - // clang::PointerType, a clang::BlockPointerType, or a - // clang::ReferenceType which points to a clang::FunctionType. + // clang::PointerType or a clang::BlockPointerType which points to a + // clang::FunctionType. const clang::Type *ClangFunctionType; bool empty() const { return !ClangFunctionType; } diff --git a/include/swift/SIL/SILBuilder.h b/include/swift/SIL/SILBuilder.h index cb1ab417683a3..26fab8b9acd85 100644 --- a/include/swift/SIL/SILBuilder.h +++ b/include/swift/SIL/SILBuilder.h @@ -1422,9 +1422,10 @@ class SILBuilder { } RefElementAddrInst *createRefElementAddr(SILLocation Loc, SILValue Operand, - VarDecl *Field, SILType ResultTy) { + VarDecl *Field, SILType ResultTy, + bool IsImmutable = false) { return insert(new (getModule()) RefElementAddrInst( - getSILDebugLocation(Loc), Operand, Field, ResultTy)); + getSILDebugLocation(Loc), Operand, Field, ResultTy, IsImmutable)); } RefElementAddrInst *createRefElementAddr(SILLocation Loc, SILValue Operand, VarDecl *Field) { @@ -1434,9 +1435,10 @@ class SILBuilder { } RefTailAddrInst *createRefTailAddr(SILLocation Loc, SILValue Ref, - SILType ResultTy) { + SILType ResultTy, + bool IsImmutable = false) { return insert(new (getModule()) RefTailAddrInst(getSILDebugLocation(Loc), - Ref, ResultTy)); + Ref, ResultTy, IsImmutable)); } DestructureStructInst *createDestructureStruct(SILLocation Loc, @@ -1722,6 +1724,17 @@ class SILBuilder { return insert(new (getModule()) IsUniqueInst(getSILDebugLocation(Loc), operand, Int1Ty)); } + BeginCOWMutationInst *createBeginCOWMutation(SILLocation Loc, + SILValue operand, bool isNative) { + auto Int1Ty = SILType::getBuiltinIntegerType(1, getASTContext()); + return insert(BeginCOWMutationInst::create(getSILDebugLocation(Loc), operand, + Int1Ty, getFunction(), isNative)); + } + EndCOWMutationInst *createEndCOWMutation(SILLocation Loc, SILValue operand, + bool keepUnique = false) { + return insert(new (getModule()) EndCOWMutationInst(getSILDebugLocation(Loc), + operand, keepUnique)); + } IsEscapingClosureInst *createIsEscapingClosure(SILLocation Loc, SILValue operand, unsigned VerificationType) { diff --git a/include/swift/SIL/SILCloner.h b/include/swift/SIL/SILCloner.h index 5b9f2dfcea89f..474961ebaf76d 100644 --- a/include/swift/SIL/SILCloner.h +++ b/include/swift/SIL/SILCloner.h @@ -1966,8 +1966,8 @@ SILCloner::visitRefElementAddrInst(RefElementAddrInst *Inst) { getBuilder().setCurrentDebugScope(getOpScope(Inst->getDebugScope())); recordClonedInstruction( Inst, getBuilder().createRefElementAddr( - getOpLocation(Inst->getLoc()), getOpValue(Inst->getOperand()), - Inst->getField(), getOpType(Inst->getType()))); + getOpLocation(Inst->getLoc()), getOpValue(Inst->getOperand()), + Inst->getField(), getOpType(Inst->getType()), Inst->isImmutable())); } template @@ -1977,7 +1977,8 @@ SILCloner::visitRefTailAddrInst(RefTailAddrInst *Inst) { recordClonedInstruction( Inst, getBuilder().createRefTailAddr(getOpLocation(Inst->getLoc()), getOpValue(Inst->getOperand()), - getOpType(Inst->getType()))); + getOpType(Inst->getType()), + Inst->isImmutable())); } template @@ -2370,6 +2371,20 @@ void SILCloner::visitIsUniqueInst(IsUniqueInst *Inst) { Inst, getBuilder().createIsUnique(getOpLocation(Inst->getLoc()), getOpValue(Inst->getOperand()))); } +template +void SILCloner::visitBeginCOWMutationInst(BeginCOWMutationInst *Inst) { + getBuilder().setCurrentDebugScope(getOpScope(Inst->getDebugScope())); + recordClonedInstruction( + Inst, getBuilder().createBeginCOWMutation(getOpLocation(Inst->getLoc()), + getOpValue(Inst->getOperand()), Inst->isNative())); +} +template +void SILCloner::visitEndCOWMutationInst(EndCOWMutationInst *Inst) { + getBuilder().setCurrentDebugScope(getOpScope(Inst->getDebugScope())); + recordClonedInstruction( + Inst, getBuilder().createEndCOWMutation(getOpLocation(Inst->getLoc()), + getOpValue(Inst->getOperand()), Inst->doKeepUnique())); +} template void SILCloner::visitIsEscapingClosureInst( IsEscapingClosureInst *Inst) { diff --git a/include/swift/SIL/SILInstruction.h b/include/swift/SIL/SILInstruction.h index 23a47ff99198c..c2e9d4e2191c4 100644 --- a/include/swift/SIL/SILInstruction.h +++ b/include/swift/SIL/SILInstruction.h @@ -1162,8 +1162,7 @@ class InstructionBase : public InstBase { } }; -/// A template base class for instructions that take a single SILValue operand -/// and has no result or a single value result. +/// A template base class for instructions that take a single SILValue operand. template class UnaryInstructionBase : public InstructionBase { // Space for 1 operand. @@ -5806,11 +5805,24 @@ class RefElementAddrInst friend SILBuilder; RefElementAddrInst(SILDebugLocation DebugLoc, SILValue Operand, - VarDecl *Field, SILType ResultTy) - : UnaryInstructionBase(DebugLoc, Operand, ResultTy, Field) {} + VarDecl *Field, SILType ResultTy, bool IsImmutable) + : UnaryInstructionBase(DebugLoc, Operand, ResultTy, Field) { + setImmutable(IsImmutable); + } public: ClassDecl *getClassDecl() const { return cast(getParentDecl()); } + + /// Returns true if all loads of the same instance variable from the same + /// class reference operand are guaranteed to yield the same value. + bool isImmutable() const { + return SILInstruction::Bits.RefElementAddrInst.Immutable; + } + + /// Sets the immutable flag. + void setImmutable(bool immutable = true) { + SILInstruction::Bits.RefElementAddrInst.Immutable = immutable; + } }; /// RefTailAddrInst - Derive the address of the first element of the first @@ -5821,8 +5833,11 @@ class RefTailAddrInst { friend SILBuilder; - RefTailAddrInst(SILDebugLocation DebugLoc, SILValue Operand, SILType ResultTy) - : UnaryInstructionBase(DebugLoc, Operand, ResultTy) {} + RefTailAddrInst(SILDebugLocation DebugLoc, SILValue Operand, SILType ResultTy, + bool IsImmutable) + : UnaryInstructionBase(DebugLoc, Operand, ResultTy) { + setImmutable(IsImmutable); + } public: ClassDecl *getClassDecl() const { @@ -5832,6 +5847,17 @@ class RefTailAddrInst } SILType getTailType() const { return getType().getObjectType(); } + + /// Returns true if all loads of the same instance variable from the same + /// class reference operand are guaranteed to yield the same value. + bool isImmutable() const { + return SILInstruction::Bits.RefTailAddrInst.Immutable; + } + + /// Sets the immutable flag. + void setImmutable(bool immutable = true) { + SILInstruction::Bits.RefTailAddrInst.Immutable = immutable; + } }; /// MethodInst - Abstract base for instructions that implement dynamic @@ -6562,6 +6588,106 @@ class IsUniqueInst : UnaryInstructionBase(DebugLoc, Operand, BoolTy) {} }; +class BeginCOWMutationInst; + +/// A result for the begin_cow_mutation instruction. See documentation for +/// begin_cow_mutation for more information. +class BeginCOWMutationResult final : public MultipleValueInstructionResult { +public: + BeginCOWMutationResult(unsigned index, SILType type, + ValueOwnershipKind ownershipKind) + : MultipleValueInstructionResult(ValueKind::BeginCOWMutationResult, + index, type, ownershipKind) {} + + BeginCOWMutationInst *getParent(); // inline below + const BeginCOWMutationInst *getParent() const { + return const_cast(this)->getParent(); + } + + static bool classof(const SILNode *N) { + return N->getKind() == SILNodeKind::BeginCOWMutationResult; + } +}; + +/// Performs a uniqueness check of the operand for the purpose of modifying +/// a copy-on-write object. +/// +/// Returns two results: the first result is an Int1 which is the result of the +/// uniqueness check. The second result is the class reference operand, which +/// can be used for mutation. +class BeginCOWMutationInst final + : public UnaryInstructionBase, + public MultipleValueInstructionTrailingObjects< + BeginCOWMutationInst, BeginCOWMutationResult> +{ + friend SILBuilder; + friend TrailingObjects; + + bool native; + + BeginCOWMutationInst(SILDebugLocation loc, SILValue operand, + ArrayRef resultTypes, + ArrayRef resultOwnerships, + bool isNative); + + static BeginCOWMutationInst * + create(SILDebugLocation loc, SILValue operand, SILType BoolTy, SILFunction &F, + bool isNative); + +public: + using MultipleValueInstructionTrailingObjects::totalSizeToAlloc; + + /// Returns the result of the uniqueness check. + SILValue getUniquenessResult() const { + return &getAllResultsBuffer()[0]; + } + + /// Returns the class reference which can be used for mutation. + SILValue getBufferResult() const { + return &getAllResultsBuffer()[1]; + } + + bool isNative() const { + return SILInstruction::Bits.BeginCOWMutationInst.Native; + } + + void setNative(bool native = true) { + SILInstruction::Bits.BeginCOWMutationInst.Native = native; + } +}; + +// Out of line to work around forward declaration issues. +inline BeginCOWMutationInst *BeginCOWMutationResult::getParent() { + auto *Parent = MultipleValueInstructionResult::getParent(); + return cast(Parent); +} + +/// Marks the end of the mutation of a reference counted object. +class EndCOWMutationInst + : public UnaryInstructionBase +{ + friend SILBuilder; + + bool keepUnique; + + EndCOWMutationInst(SILDebugLocation DebugLoc, SILValue Operand, + bool keepUnique) + : UnaryInstructionBase(DebugLoc, Operand, Operand->getType()) { + setKeepUnique(keepUnique); + } + +public: + bool doKeepUnique() const { + return SILInstruction::Bits.EndCOWMutationInst.KeepUnique; + } + + void setKeepUnique(bool keepUnique = true) { + SILInstruction::Bits.EndCOWMutationInst.KeepUnique = keepUnique; + } +}; + /// Given an escaping closure return true iff it has a non-nil context and the /// context has a strong reference count greater than 1. class IsEscapingClosureInst diff --git a/include/swift/SIL/SILNode.h b/include/swift/SIL/SILNode.h index c5820ef1e59b9..6f1f50a2dc453 100644 --- a/include/swift/SIL/SILNode.h +++ b/include/swift/SIL/SILNode.h @@ -304,6 +304,18 @@ class alignas(8) SILNode { FieldNo : 32 ); + SWIFT_INLINE_BITFIELD(RefElementAddrInst, SingleValueInstruction, 1, + Immutable : 1 + ); + + SWIFT_INLINE_BITFIELD(RefTailAddrInst, SingleValueInstruction, 1, + Immutable : 1 + ); + + SWIFT_INLINE_BITFIELD(EndCOWMutationInst, NonValueInstruction, 1, + KeepUnique : 1 + ); + SWIFT_INLINE_BITFIELD_FULL(FieldIndexCacheBase, SingleValueInstruction, 32, : NumPadBits, FieldIndex : 32); @@ -355,6 +367,12 @@ class alignas(8) SILNode { NumCases : 32 ); + SWIFT_INLINE_BITFIELD_EMPTY(MultipleValueInstruction, SILInstruction); + + SWIFT_INLINE_BITFIELD(BeginCOWMutationInst, MultipleValueInstruction, 1, + Native : 1 + ); + } Bits; enum class SILNodeStorageLocation : uint8_t { Value, Instruction }; diff --git a/include/swift/SIL/SILNodes.def b/include/swift/SIL/SILNodes.def index d85bdb0bac175..9bd877c60c166 100644 --- a/include/swift/SIL/SILNodes.def +++ b/include/swift/SIL/SILNodes.def @@ -413,6 +413,7 @@ ABSTRACT_VALUE(SILArgument, ValueBase) ABSTRACT_VALUE(MultipleValueInstructionResult, ValueBase) MULTIPLE_VALUE_INST_RESULT(BeginApplyResult, MultipleValueInstructionResult) + MULTIPLE_VALUE_INST_RESULT(BeginCOWMutationResult, MultipleValueInstructionResult) MULTIPLE_VALUE_INST_RESULT(DestructureStructResult, MultipleValueInstructionResult) MULTIPLE_VALUE_INST_RESULT(DestructureTupleResult, MultipleValueInstructionResult) VALUE_RANGE(MultipleValueInstructionResult, BeginApplyResult, DestructureTupleResult) @@ -579,6 +580,9 @@ ABSTRACT_VALUE_AND_INST(SingleValueInstruction, ValueBase, SILInstruction) SINGLE_VALUE_INST(IsUniqueInst, is_unique, SingleValueInstruction, MayHaveSideEffects, DoesNotRelease) + SINGLE_VALUE_INST(EndCOWMutationInst, end_cow_mutation, + SingleValueInstruction, None, DoesNotRelease) + SINGLE_VALUE_INST(IsEscapingClosureInst, is_escaping_closure, SingleValueInstruction, MayRead, DoesNotRelease) @@ -876,6 +880,8 @@ NODE_RANGE(NonValueInstruction, UnreachableInst, CondFailInst) ABSTRACT_INST(MultipleValueInstruction, SILInstruction) FULLAPPLYSITE_MULTIPLE_VALUE_INST(BeginApplyInst, begin_apply, MultipleValueInstruction, MayHaveSideEffects, MayRelease) +MULTIPLE_VALUE_INST(BeginCOWMutationInst, begin_cow_mutation, + MultipleValueInstruction, None, DoesNotRelease) MULTIPLE_VALUE_INST(DestructureStructInst, destructure_struct, MultipleValueInstruction, None, DoesNotRelease) MULTIPLE_VALUE_INST(DestructureTupleInst, destructure_tuple, diff --git a/lib/AST/Builtins.cpp b/lib/AST/Builtins.cpp index 29a85b226ca3d..c5580e34748b4 100644 --- a/lib/AST/Builtins.cpp +++ b/lib/AST/Builtins.cpp @@ -431,13 +431,26 @@ createGenericParam(ASTContext &ctx, const char *name, unsigned index) { /// Create a generic parameter list with multiple generic parameters. static GenericParamList *getGenericParams(ASTContext &ctx, - unsigned numParameters) { + unsigned numParameters, + bool isAnyObject) { assert(numParameters <= llvm::array_lengthof(GenericParamNames)); SmallVector genericParams; for (unsigned i = 0; i != numParameters; ++i) genericParams.push_back(createGenericParam(ctx, GenericParamNames[i], i)); + + if (isAnyObject) { + CanType ao = ctx.getAnyObjectType(); + SmallVector req; + req.push_back(RequirementRepr::getTypeConstraint(TypeLoc::withoutLoc(genericParams[0]->getInterfaceType()), SourceLoc(), + TypeLoc::withoutLoc(ao))); + + auto paramList = GenericParamList::create(ctx, SourceLoc(), genericParams, + SourceLoc(), req, SourceLoc()); + return paramList; + } + auto paramList = GenericParamList::create(ctx, SourceLoc(), genericParams, SourceLoc()); return paramList; @@ -460,9 +473,10 @@ namespace { SmallVector addedRequirements; public: - BuiltinFunctionBuilder(ASTContext &ctx, unsigned numGenericParams = 1) + BuiltinFunctionBuilder(ASTContext &ctx, unsigned numGenericParams = 1, + bool isAnyObject = false) : Context(ctx) { - TheGenericParamList = getGenericParams(ctx, numGenericParams); + TheGenericParamList = getGenericParams(ctx, numGenericParams, isAnyObject); for (auto gp : TheGenericParamList->getParams()) { genericParamTypes.push_back( gp->getDeclaredInterfaceType()->castTo()); @@ -645,6 +659,14 @@ static ValueDecl *getIsUniqueOperation(ASTContext &Context, Identifier Id) { return builder.build(Id); } +static ValueDecl *getEndCOWMutation(ASTContext &Context, Identifier Id) { + // (@inout T) -> () + BuiltinFunctionBuilder builder(Context); + builder.addParameter(makeGenericParam(), ValueOwnership::InOut); + builder.setResult(makeConcrete(TupleType::getEmpty(Context))); + return builder.build(Id); +} + static ValueDecl *getBindMemoryOperation(ASTContext &Context, Identifier Id) { BuiltinFunctionBuilder builder(Context); builder.addParameter(makeConcrete(Context.TheRawPointerType)); @@ -908,6 +930,16 @@ static ValueDecl *getValueToBridgeObject(ASTContext &C, Identifier Id) { return builder.build(Id); } +static ValueDecl *getCOWBufferForReading(ASTContext &C, Identifier Id) { + // T -> T + // + BuiltinFunctionBuilder builder(C, 1, true); + auto T = makeGenericParam(); + builder.addParameter(T); + builder.setResult(T); + return builder.build(Id); +} + static ValueDecl *getUnsafeGuaranteed(ASTContext &C, Identifier Id) { // T -> (T, Int8Ty) // @@ -2249,9 +2281,16 @@ ValueDecl *swift::getBuiltinValueDecl(ASTContext &Context, Identifier Id) { case BuiltinValueKind::IsUnique: case BuiltinValueKind::IsUnique_native: + case BuiltinValueKind::BeginCOWMutation: + case BuiltinValueKind::BeginCOWMutation_native: if (!Types.empty()) return nullptr; + // BeginCOWMutation has the same signature as IsUnique. return getIsUniqueOperation(Context, Id); + case BuiltinValueKind::EndCOWMutation: + if (!Types.empty()) return nullptr; + return getEndCOWMutation(Context, Id); + case BuiltinValueKind::BindMemory: if (!Types.empty()) return nullptr; return getBindMemoryOperation(Context, Id); @@ -2380,6 +2419,10 @@ ValueDecl *swift::getBuiltinValueDecl(ASTContext &Context, Identifier Id) { if (!Types.empty()) return nullptr; return getValueToBridgeObject(Context, Id); + + case BuiltinValueKind::COWBufferForReading: + return getCOWBufferForReading(Context, Id); + case BuiltinValueKind::UnsafeGuaranteed: return getUnsafeGuaranteed(Context, Id); diff --git a/lib/AST/Type.cpp b/lib/AST/Type.cpp index e47e5832d478d..abdc06e298980 100644 --- a/lib/AST/Type.cpp +++ b/lib/AST/Type.cpp @@ -3376,8 +3376,7 @@ void AnyFunctionType::ExtInfo::Uncommon::printClangFunctionType( void AnyFunctionType::ExtInfo::assertIsFunctionType(const clang::Type *type) { #ifndef NDEBUG - if (!(type->isFunctionPointerType() || type->isBlockPointerType() || - type->isFunctionReferenceType())) { + if (!(type->isFunctionPointerType() || type->isBlockPointerType())) { SmallString<256> buf; llvm::raw_svector_ostream os(buf); os << "Expected a Clang function type wrapped in a pointer type or " diff --git a/lib/ClangImporter/ImportType.cpp b/lib/ClangImporter/ImportType.cpp index 0f4a2faf53086..08238b491c2a5 100644 --- a/lib/ClangImporter/ImportType.cpp +++ b/lib/ClangImporter/ImportType.cpp @@ -159,30 +159,6 @@ namespace { explicit operator bool() const { return (bool) AbstractType; } }; - static ImportResult importFunctionPointerLikeType(const clang::Type &type, - const Type &pointeeType) { - auto funcTy = pointeeType->castTo(); - return {FunctionType::get( - funcTy->getParams(), funcTy->getResult(), - funcTy->getExtInfo() - .withRepresentation( - AnyFunctionType::Representation::CFunctionPointer) - .withClangFunctionType(&type)), - type.isReferenceType() ? ImportHint::None - : ImportHint::CFunctionPointer}; - } - - static ImportResult importOverAlignedFunctionPointerLikeType( - const clang::Type &type, ClangImporter::Implementation &Impl) { - auto opaquePointer = Impl.SwiftContext.getOpaquePointerDecl(); - if (!opaquePointer) { - return Type(); - } - return {opaquePointer->getDeclaredType(), - type.isReferenceType() ? ImportHint::None - : ImportHint::OtherPointer}; - } - class SwiftTypeConverter : public clang::TypeVisitor { @@ -430,11 +406,23 @@ namespace { // alignment is greater than the maximum Swift alignment, import as // OpaquePointer. if (!pointeeType || Impl.isOverAligned(pointeeQualType)) { - return importOverAlignedFunctionPointerLikeType(*type, Impl); + auto opaquePointer = Impl.SwiftContext.getOpaquePointerDecl(); + if (!opaquePointer) + return Type(); + return {opaquePointer->getDeclaredType(), + ImportHint::OtherPointer}; } - + if (pointeeQualType->isFunctionType()) { - return importFunctionPointerLikeType(*type, pointeeType); + auto funcTy = pointeeType->castTo(); + return { + FunctionType::get(funcTy->getParams(), funcTy->getResult(), + funcTy->getExtInfo() + .withRepresentation( + AnyFunctionType::Representation::CFunctionPointer) + .withClangFunctionType(type)), + ImportHint::CFunctionPointer + }; } PointerTypeKind pointerKind; @@ -484,29 +472,7 @@ namespace { } ImportResult VisitReferenceType(const clang::ReferenceType *type) { - auto pointeeQualType = type->getPointeeType(); - auto quals = pointeeQualType.getQualifiers(); - Type pointeeType = - Impl.importTypeIgnoreIUO(pointeeQualType, ImportTypeKind::Value, - AllowNSUIntegerAsInt, Bridgeability::None); - - if (pointeeQualType->isFunctionType()) { - return importFunctionPointerLikeType(*type, pointeeType); - } - - if (Impl.isOverAligned(pointeeQualType)) { - return importOverAlignedFunctionPointerLikeType(*type, Impl); - } - - PointerTypeKind pointerKind; - if (quals.hasConst()) { - pointerKind = PTK_UnsafePointer; - } else { - pointerKind = PTK_UnsafeMutablePointer; - } - - return {pointeeType->wrapInPointer(pointerKind), - ImportHint::None}; + return Type(); } ImportResult VisitMemberPointer(const clang::MemberPointerType *type) { diff --git a/lib/ClangImporter/ImporterImpl.h b/lib/ClangImporter/ImporterImpl.h index 72de3a47bcde7..1a01a71a1c0ca 100644 --- a/lib/ClangImporter/ImporterImpl.h +++ b/lib/ClangImporter/ImporterImpl.h @@ -135,10 +135,9 @@ enum class ImportTypeKind { /// Import the type of a function parameter. /// - /// Special handling: - /// * C and C++ pointers become `UnsafePointer?` or `UnsafeMutablePointer?` - /// * C++ references become `UnsafePointer` or `UnsafeMutablePointer` - /// * Bridging that requires type conversions is allowed. + /// This provides special treatment for C++ references (which become + /// [inout] parameters) and C pointers (which become magic [inout]-able types), + /// among other things, and enables the conversion of bridged types. /// Parameters are always considered CF-audited. Parameter, diff --git a/lib/IRGen/GenBuiltin.cpp b/lib/IRGen/GenBuiltin.cpp index cccd52fe0ec5d..c33d650f11239 100644 --- a/lib/IRGen/GenBuiltin.cpp +++ b/lib/IRGen/GenBuiltin.cpp @@ -122,6 +122,13 @@ void irgen::emitBuiltinCall(IRGenFunction &IGF, const BuiltinInfo &Builtin, Identifier FnId, SILType resultType, Explosion &args, Explosion &out, SubstitutionMap substitutions) { + if (Builtin.ID == BuiltinValueKind::COWBufferForReading) { + // Just forward the incoming argument. + assert(args.size() == 1 && "Expecting one incoming argument"); + out = std::move(args); + return; + } + if (Builtin.ID == BuiltinValueKind::UnsafeGuaranteedEnd) { // Just consume the incoming argument. assert(args.size() == 1 && "Expecting one incoming argument"); diff --git a/lib/IRGen/GenExistential.cpp b/lib/IRGen/GenExistential.cpp index 4374b43d4794b..31238f50c0dac 100644 --- a/lib/IRGen/GenExistential.cpp +++ b/lib/IRGen/GenExistential.cpp @@ -1075,6 +1075,10 @@ class ClassExistentialTypeInfo final (void)e.claim(getNumStoredProtocols()); } + virtual ReferenceCounting getReferenceCountingType() const override { + return Refcounting; + } + // We can just re-use the reference storage types. #define NEVER_LOADABLE_CHECKED_REF_STORAGE_HELPER(Name, name) \ void name##LoadStrong(IRGenFunction &IGF, Address existential, \ diff --git a/lib/IRGen/HeapTypeInfo.h b/lib/IRGen/HeapTypeInfo.h index ca4f3e9dfcf8f..1b6640c844607 100644 --- a/lib/IRGen/HeapTypeInfo.h +++ b/lib/IRGen/HeapTypeInfo.h @@ -228,6 +228,10 @@ class HeapTypeInfo return LoadedRef(ptr, true); } + ReferenceCounting getReferenceCountingType() const override { + return asDerived().getReferenceCounting(); + } + // Extra inhabitants of heap object pointers. bool mayHaveExtraInhabitants(IRGenModule &IGM) const override { diff --git a/lib/IRGen/IRGenSIL.cpp b/lib/IRGen/IRGenSIL.cpp index 46dfa1b6c57d1..7a2ce3a6321b4 100644 --- a/lib/IRGen/IRGenSIL.cpp +++ b/lib/IRGen/IRGenSIL.cpp @@ -975,6 +975,8 @@ class IRGenSILFunction : void visitStrongRetainInst(StrongRetainInst *i); void visitStrongReleaseInst(StrongReleaseInst *i); void visitIsUniqueInst(IsUniqueInst *i); + void visitBeginCOWMutationInst(BeginCOWMutationInst *i); + void visitEndCOWMutationInst(EndCOWMutationInst *i); void visitIsEscapingClosureInst(IsEscapingClosureInst *i); void visitDeallocStackInst(DeallocStackInst *i); void visitDeallocBoxInst(DeallocBoxInst *i); @@ -4159,6 +4161,42 @@ void IRGenSILFunction::visitIsUniqueInst(swift::IsUniqueInst *i) { setLoweredExplosion(i, out); } +void IRGenSILFunction::visitBeginCOWMutationInst(BeginCOWMutationInst *i) { + SILValue ref = i->getOperand(); + Explosion bufferEx = getLoweredExplosion(ref); + llvm::Value *buffer = *bufferEx.begin(); + setLoweredExplosion(i->getBufferResult(), bufferEx); + + Explosion isUnique; + if (hasReferenceSemantics(*this, ref->getType())) { + if (i->getUniquenessResult()->use_empty()) { + // No need to call isUnique if the result is not used. + isUnique.add(llvm::UndefValue::get(IGM.Int1Ty)); + } else { + ReferenceCounting style = cast( + getTypeInfo(ref->getType())).getReferenceCountingType(); + if (i->isNative()) + style = ReferenceCounting::Native; + + llvm::Value *castBuffer = + Builder.CreateBitCast(buffer, IGM.getReferenceType(style)); + + isUnique.add(emitIsUniqueCall(castBuffer, i->getLoc().getSourceLoc(), + /*isNonNull*/ true)); + } + } else { + emitTrap("beginCOWMutation called for a non-reference", + /*EmitUnreachable=*/false); + isUnique.add(llvm::UndefValue::get(IGM.Int1Ty)); + } + setLoweredExplosion(i->getUniquenessResult(), isUnique); +} + +void IRGenSILFunction::visitEndCOWMutationInst(EndCOWMutationInst *i) { + Explosion v = getLoweredExplosion(i->getOperand()); + setLoweredExplosion(i, v); +} + void IRGenSILFunction::visitIsEscapingClosureInst( swift::IsEscapingClosureInst *i) { // The closure operand is allowed to be an optional closure. diff --git a/lib/IRGen/ReferenceTypeInfo.h b/lib/IRGen/ReferenceTypeInfo.h index 0595b85650223..78c65d00cb6dc 100644 --- a/lib/IRGen/ReferenceTypeInfo.h +++ b/lib/IRGen/ReferenceTypeInfo.h @@ -45,6 +45,10 @@ class ReferenceTypeInfo : public LoadableTypeInfo { virtual void strongRelease(IRGenFunction &IGF, Explosion &in, Atomicity atomicity) const = 0; + virtual ReferenceCounting getReferenceCountingType() const { + llvm_unreachable("not supported"); + } + #define REF_STORAGE_HELPER(Name) \ virtual const TypeInfo *create##Name##StorageType(TypeConverter &TC, \ bool isOptional) const = 0; diff --git a/lib/SIL/IR/AbstractionPattern.cpp b/lib/SIL/IR/AbstractionPattern.cpp index fd89e9d9d8d54..ce5d24fb5ab37 100644 --- a/lib/SIL/IR/AbstractionPattern.cpp +++ b/lib/SIL/IR/AbstractionPattern.cpp @@ -319,8 +319,6 @@ getClangFunctionType(const clang::Type *clangType) { clangType = ptrTy->getPointeeType().getTypePtr(); } else if (auto blockTy = clangType->getAs()) { clangType = blockTy->getPointeeType().getTypePtr(); - } else if (auto refTy = clangType->getAs()) { - clangType = refTy->getPointeeType().getTypePtr(); } return clangType->castAs(); } diff --git a/lib/SIL/IR/OperandOwnership.cpp b/lib/SIL/IR/OperandOwnership.cpp index dc5ba790c88b3..4befbd0596293 100644 --- a/lib/SIL/IR/OperandOwnership.cpp +++ b/lib/SIL/IR/OperandOwnership.cpp @@ -171,6 +171,8 @@ CONSTANT_OWNERSHIP_INST(Owned, MustBeInvalidated, DeallocExistentialBox) CONSTANT_OWNERSHIP_INST(Owned, MustBeInvalidated, DeallocRef) CONSTANT_OWNERSHIP_INST(Owned, MustBeInvalidated, DestroyValue) CONSTANT_OWNERSHIP_INST(Owned, MustBeInvalidated, EndLifetime) +CONSTANT_OWNERSHIP_INST(Owned, MustBeInvalidated, BeginCOWMutation) +CONSTANT_OWNERSHIP_INST(Owned, MustBeInvalidated, EndCOWMutation) CONSTANT_OWNERSHIP_INST(None, MustBeLive, AbortApply) CONSTANT_OWNERSHIP_INST(None, MustBeLive, AddressToPointer) CONSTANT_OWNERSHIP_INST(None, MustBeLive, BeginAccess) @@ -1021,6 +1023,7 @@ ANY_OWNERSHIP_BUILTIN(TypePtrAuthDiscriminator) ValueOwnershipKind::OWNERSHIP, \ UseLifetimeConstraint::USE_LIFETIME_CONSTRAINT); \ } +CONSTANT_OWNERSHIP_BUILTIN(Owned, MustBeInvalidated, COWBufferForReading) CONSTANT_OWNERSHIP_BUILTIN(Owned, MustBeInvalidated, UnsafeGuaranteed) #undef CONSTANT_OWNERSHIP_BUILTIN diff --git a/lib/SIL/IR/SILInstruction.cpp b/lib/SIL/IR/SILInstruction.cpp index 593858ce03784..24ab6b8fb1925 100644 --- a/lib/SIL/IR/SILInstruction.cpp +++ b/lib/SIL/IR/SILInstruction.cpp @@ -383,6 +383,16 @@ namespace { } return true; } + + bool visitBeginCOWMutationInst(const BeginCOWMutationInst *RHS) { + auto *left = cast(LHS); + return left->isNative() == RHS->isNative(); + } + + bool visitEndCOWMutationInst(const EndCOWMutationInst *RHS) { + auto *left = cast(LHS); + return left->doKeepUnique() == RHS->doKeepUnique(); + } bool visitAllocRefDynamicInst(const AllocRefDynamicInst *RHS) { return true; @@ -1126,6 +1136,7 @@ bool SILInstruction::mayRelease() const { bool SILInstruction::mayReleaseOrReadRefCount() const { switch (getKind()) { case SILInstructionKind::IsUniqueInst: + case SILInstructionKind::BeginCOWMutationInst: case SILInstructionKind::IsEscapingClosureInst: return true; default: diff --git a/lib/SIL/IR/SILInstructions.cpp b/lib/SIL/IR/SILInstructions.cpp index f72f53628ad8a..235b5098eb149 100644 --- a/lib/SIL/IR/SILInstructions.cpp +++ b/lib/SIL/IR/SILInstructions.cpp @@ -2166,6 +2166,36 @@ OpenExistentialValueInst::OpenExistentialValueInst(SILDebugLocation DebugLoc, SILType SelfTy) : UnaryInstructionBase(DebugLoc, Operand, SelfTy) {} +BeginCOWMutationInst::BeginCOWMutationInst(SILDebugLocation loc, + SILValue operand, + ArrayRef resultTypes, + ArrayRef resultOwnerships, + bool isNative) + : UnaryInstructionBase(loc, operand), + MultipleValueInstructionTrailingObjects(this, resultTypes, + resultOwnerships) { + assert(resultTypes.size() == 2 && resultOwnerships.size() == 2); + assert(operand->getType() == resultTypes[1]); + setNative(isNative); +} + +BeginCOWMutationInst * +BeginCOWMutationInst::create(SILDebugLocation loc, SILValue operand, + SILType boolTy, SILFunction &F, bool isNative) { + + SILType resultTypes[2] = { boolTy, operand->getType() }; + ValueOwnershipKind ownerships[2] = { ValueOwnershipKind::None, ValueOwnershipKind::Owned }; + + void *buffer = + allocateTrailingInst( + F, 1, 2); + return ::new(buffer) BeginCOWMutationInst(loc, operand, + ArrayRef(resultTypes, 2), + ArrayRef(ownerships, 2), + isNative); +} + UncheckedRefCastInst * UncheckedRefCastInst::create(SILDebugLocation DebugLoc, SILValue Operand, SILType Ty, SILFunction &F, diff --git a/lib/SIL/IR/SILPrinter.cpp b/lib/SIL/IR/SILPrinter.cpp index 452aca6f0e59e..948573e54c47f 100644 --- a/lib/SIL/IR/SILPrinter.cpp +++ b/lib/SIL/IR/SILPrinter.cpp @@ -1775,13 +1775,15 @@ class SILPrinter : public SILInstructionVisitor { *this << EI->getField()->getName().get(); } void visitRefElementAddrInst(RefElementAddrInst *EI) { - *this << getIDAndType(EI->getOperand()) << ", #"; + *this << (EI->isImmutable() ? "[immutable] " : "") + << getIDAndType(EI->getOperand()) << ", #"; printFullContext(EI->getField()->getDeclContext(), PrintState.OS); *this << EI->getField()->getName().get(); } void visitRefTailAddrInst(RefTailAddrInst *RTAI) { - *this << getIDAndType(RTAI->getOperand()) << ", " << RTAI->getTailType(); + *this << (RTAI->isImmutable() ? "[immutable] " : "") + << getIDAndType(RTAI->getOperand()) << ", " << RTAI->getTailType(); } void visitDestructureStructInst(DestructureStructInst *DSI) { @@ -1939,6 +1941,16 @@ class SILPrinter : public SILInstructionVisitor { void visitIsUniqueInst(IsUniqueInst *CUI) { *this << getIDAndType(CUI->getOperand()); } + void visitBeginCOWMutationInst(BeginCOWMutationInst *BCMI) { + if (BCMI->isNative()) + *this << "[native] "; + *this << getIDAndType(BCMI->getOperand()); + } + void visitEndCOWMutationInst(EndCOWMutationInst *ECMI) { + if (ECMI->doKeepUnique()) + *this << "[keep_unique] "; + *this << getIDAndType(ECMI->getOperand()); + } void visitIsEscapingClosureInst(IsEscapingClosureInst *CUI) { if (CUI->getVerificationType()) *this << "[objc] "; diff --git a/lib/SIL/IR/ValueOwnership.cpp b/lib/SIL/IR/ValueOwnership.cpp index 992a481bea876..88981eddc91f0 100644 --- a/lib/SIL/IR/ValueOwnership.cpp +++ b/lib/SIL/IR/ValueOwnership.cpp @@ -74,6 +74,7 @@ CONSTANT_OWNERSHIP_INST(None, AllocValueBuffer) CONSTANT_OWNERSHIP_INST(Owned, CopyBlock) CONSTANT_OWNERSHIP_INST(Owned, CopyBlockWithoutEscaping) CONSTANT_OWNERSHIP_INST(Owned, CopyValue) +CONSTANT_OWNERSHIP_INST(Owned, EndCOWMutation) CONSTANT_OWNERSHIP_INST(Owned, KeyPath) CONSTANT_OWNERSHIP_INST(Owned, InitExistentialValue) CONSTANT_OWNERSHIP_INST(Owned, GlobalValue) // TODO: is this correct? @@ -299,6 +300,11 @@ ValueOwnershipKind ValueOwnershipKindClassifier::visitBeginApplyResult( return Result->getOwnershipKind(); } +ValueOwnershipKind ValueOwnershipKindClassifier::visitBeginCOWMutationResult( + BeginCOWMutationResult *Result) { + return Result->getOwnershipKind(); +} + ValueOwnershipKind ValueOwnershipKindClassifier::visitSILFunctionArgument( SILFunctionArgument *Arg) { return Arg->getOwnershipKind(); @@ -386,6 +392,7 @@ struct ValueOwnershipKindBuiltinVisitor } // This returns a value at +1 that is destroyed strictly /after/ the // UnsafeGuaranteedEnd. This provides the guarantee that we want. +CONSTANT_OWNERSHIP_BUILTIN(Owned, COWBufferForReading) CONSTANT_OWNERSHIP_BUILTIN(Owned, UnsafeGuaranteed) CONSTANT_OWNERSHIP_BUILTIN(None, AShr) CONSTANT_OWNERSHIP_BUILTIN(None, GenericAShr) diff --git a/lib/SIL/Parser/ParseSIL.cpp b/lib/SIL/Parser/ParseSIL.cpp index 1ed4354feaca8..1436ac2e085ce 100644 --- a/lib/SIL/Parser/ParseSIL.cpp +++ b/lib/SIL/Parser/ParseSIL.cpp @@ -2976,6 +2976,24 @@ bool SILParser::parseSpecificSILInstruction(SILBuilder &B, #undef UNARY_INSTRUCTION #undef REFCOUNTING_INSTRUCTION + case SILInstructionKind::BeginCOWMutationInst: { + bool native = false; + if (parseSILOptional(native, *this, "native") || + parseTypedValueRef(Val, B) || + parseSILDebugLocation(InstLoc, B)) + return true; + ResultVal = B.createBeginCOWMutation(InstLoc, Val, native); + break; + } + case SILInstructionKind::EndCOWMutationInst: { + bool keepUnique = false; + if (parseSILOptional(keepUnique, *this, "keep_unique") || + parseTypedValueRef(Val, B) || + parseSILDebugLocation(InstLoc, B)) + return true; + ResultVal = B.createEndCOWMutation(InstLoc, Val, keepUnique); + break; + } case SILInstructionKind::IsEscapingClosureInst: { bool IsObjcVerifcationType = false; if (parseSILOptional(IsObjcVerifcationType, *this, "objc")) @@ -4439,7 +4457,9 @@ bool SILParser::parseSpecificSILInstruction(SILBuilder &B, case SILInstructionKind::RefElementAddrInst: { ValueDecl *FieldV; SourceLoc NameLoc; - if (parseTypedValueRef(Val, B) || + bool IsImmutable = false; + if (parseSILOptional(IsImmutable, *this, "immutable") || + parseTypedValueRef(Val, B) || P.parseToken(tok::comma, diag::expected_tok_in_sil_instr, ",") || parseSILDottedPath(FieldV) || parseSILDebugLocation(InstLoc, B)) return true; @@ -4450,18 +4470,21 @@ bool SILParser::parseSpecificSILInstruction(SILBuilder &B, VarDecl *Field = cast(FieldV); auto ResultTy = Val->getType().getFieldType(Field, SILMod, B.getTypeExpansionContext()); - ResultVal = B.createRefElementAddr(InstLoc, Val, Field, ResultTy); + ResultVal = B.createRefElementAddr(InstLoc, Val, Field, ResultTy, + IsImmutable); break; } case SILInstructionKind::RefTailAddrInst: { SourceLoc NameLoc; SILType ResultObjTy; - if (parseTypedValueRef(Val, B) || + bool IsImmutable = false; + if (parseSILOptional(IsImmutable, *this, "immutable") || + parseTypedValueRef(Val, B) || P.parseToken(tok::comma, diag::expected_tok_in_sil_instr, ",") || parseSILType(ResultObjTy) || parseSILDebugLocation(InstLoc, B)) return true; SILType ResultTy = ResultObjTy.getAddressType(); - ResultVal = B.createRefTailAddr(InstLoc, Val, ResultTy); + ResultVal = B.createRefTailAddr(InstLoc, Val, ResultTy, IsImmutable); break; } case SILInstructionKind::IndexAddrInst: { diff --git a/lib/SIL/Utils/MemAccessUtils.cpp b/lib/SIL/Utils/MemAccessUtils.cpp index 973cc216d8237..6f041f59995e3 100644 --- a/lib/SIL/Utils/MemAccessUtils.cpp +++ b/lib/SIL/Utils/MemAccessUtils.cpp @@ -528,6 +528,7 @@ static void visitBuiltinAddress(BuiltinInst *builtin, case BuiltinValueKind::Unreachable: case BuiltinValueKind::CondUnreachable: case BuiltinValueKind::DestroyArray: + case BuiltinValueKind::COWBufferForReading: case BuiltinValueKind::UnsafeGuaranteed: case BuiltinValueKind::UnsafeGuaranteedEnd: case BuiltinValueKind::Swift3ImplicitObjCEntrypoint: diff --git a/lib/SILGen/SILGenBuiltin.cpp b/lib/SILGen/SILGenBuiltin.cpp index 45e94b5575123..8ed54c56622c5 100644 --- a/lib/SILGen/SILGenBuiltin.cpp +++ b/lib/SILGen/SILGenBuiltin.cpp @@ -901,6 +901,60 @@ emitBuiltinIsUnique_native(SILGenFunction &SGF, return ManagedValue::forUnmanaged(result); } +static ManagedValue +emitBuiltinBeginCOWMutation(SILGenFunction &SGF, + SILLocation loc, + SubstitutionMap subs, + ArrayRef args, + SGFContext C) { + + assert(subs.getReplacementTypes().size() == 1 && + "BeginCOWMutation should have one sub."); + assert(args.size() == 1 && "isUnique_native should have one arg."); + + SILValue refAddr = args[0].getValue(); + auto *ref = SGF.B.createLoad(loc, refAddr, LoadOwnershipQualifier::Take); + BeginCOWMutationInst *beginCOW = SGF.B.createBeginCOWMutation(loc, ref, /*isNative*/ false); + SGF.B.createStore(loc, beginCOW->getBufferResult(), refAddr, StoreOwnershipQualifier::Init); + return ManagedValue::forUnmanaged(beginCOW->getUniquenessResult()); +} + +static ManagedValue +emitBuiltinBeginCOWMutation_native(SILGenFunction &SGF, + SILLocation loc, + SubstitutionMap subs, + ArrayRef args, + SGFContext C) { + + assert(subs.getReplacementTypes().size() == 1 && + "BeginCOWMutation should have one sub."); + assert(args.size() == 1 && "isUnique_native should have one arg."); + + SILValue refAddr = args[0].getValue(); + auto *ref = SGF.B.createLoad(loc, refAddr, LoadOwnershipQualifier::Take); + BeginCOWMutationInst *beginCOW = SGF.B.createBeginCOWMutation(loc, ref, /*isNative*/ true); + SGF.B.createStore(loc, beginCOW->getBufferResult(), refAddr, StoreOwnershipQualifier::Init); + return ManagedValue::forUnmanaged(beginCOW->getUniquenessResult()); +} + +static ManagedValue +emitBuiltinEndCOWMutation(SILGenFunction &SGF, + SILLocation loc, + SubstitutionMap subs, + ArrayRef args, + SGFContext C) { + + assert(subs.getReplacementTypes().size() == 1 && + "EndCOWMutation should have one sub."); + assert(args.size() == 1 && "isUnique_native should have one arg."); + + SILValue refAddr = args[0].getValue(); + auto ref = SGF.B.createLoad(loc, refAddr, LoadOwnershipQualifier::Take); + auto endRef = SGF.B.createEndCOWMutation(loc, ref); + SGF.B.createStore(loc, endRef, refAddr, StoreOwnershipQualifier::Init); + return ManagedValue::forUnmanaged(SGF.emitEmptyTuple(loc)); +} + static ManagedValue emitBuiltinBindMemory(SILGenFunction &SGF, SILLocation loc, SubstitutionMap subs, diff --git a/lib/SILOptimizer/Analysis/ARCAnalysis.cpp b/lib/SILOptimizer/Analysis/ARCAnalysis.cpp index 68d06dab96b1f..c64298fad8890 100644 --- a/lib/SILOptimizer/Analysis/ARCAnalysis.cpp +++ b/lib/SILOptimizer/Analysis/ARCAnalysis.cpp @@ -474,6 +474,10 @@ mayGuaranteedUseValue(SILInstruction *User, SILValue Ptr, AliasAnalysis *AA) { // FIXME: this is overly conservative. It should return true only of the // RC identity of the single operand matches Ptr. return true; + case SILInstructionKind::BeginCOWMutationInst: + // begin_cow_mutation takes the argument as owned and produces a new + // owned result. + return false; default: llvm_unreachable("Unexpected check-ref-count instruction."); } diff --git a/lib/SILOptimizer/SILCombiner/SILCombiner.h b/lib/SILOptimizer/SILCombiner/SILCombiner.h index 31585fee4ee03..8a7435272e26c 100644 --- a/lib/SILOptimizer/SILCombiner/SILCombiner.h +++ b/lib/SILOptimizer/SILCombiner/SILCombiner.h @@ -225,6 +225,8 @@ class SILCombiner : // Optimize the "isConcrete" builtin. SILInstruction *optimizeBuiltinIsConcrete(BuiltinInst *I); + SILInstruction *optimizeBuiltinCOWBufferForReading(BuiltinInst *BI); + // Optimize the "trunc_N1_M2" builtin. if N1 is a result of "zext_M1_*" and // the following holds true: N1 > M1 and M2>= M1 SILInstruction *optimizeBuiltinTruncOrBitCast(BuiltinInst *I); diff --git a/lib/SILOptimizer/SILCombiner/SILCombinerBuiltinVisitors.cpp b/lib/SILOptimizer/SILCombiner/SILCombinerBuiltinVisitors.cpp index bd5fd485adb83..d8fb87f60f5fd 100644 --- a/lib/SILOptimizer/SILCombiner/SILCombinerBuiltinVisitors.cpp +++ b/lib/SILOptimizer/SILCombiner/SILCombinerBuiltinVisitors.cpp @@ -109,6 +109,49 @@ SILInstruction *SILCombiner::optimizeBuiltinIsConcrete(BuiltinInst *BI) { return Builder.createIntegerLiteral(BI->getLoc(), BI->getType(), 1); } +/// Replace +/// \code +/// %b = builtin "COWBufferForReading" %r +/// %a = ref_element_addr %b +/// \endcode +/// with +/// \code +/// %a = ref_element_addr [immutable] %r +/// \endcode +/// The same for ref_tail_addr. +SILInstruction *SILCombiner::optimizeBuiltinCOWBufferForReading(BuiltinInst *BI) { + auto useIter = BI->use_begin(); + while (useIter != BI->use_end()) { + auto nextIter = std::next(useIter); + SILInstruction *user = useIter->getUser(); + SILValue ref = BI->getOperand(0); + switch (user->getKind()) { + case SILInstructionKind::RefElementAddrInst: { + auto *REAI = cast(user); + REAI->setOperand(ref); + REAI->setImmutable(); + break; + } + case SILInstructionKind::RefTailAddrInst: { + auto *RTAI = cast(user); + RTAI->setOperand(ref); + RTAI->setImmutable(); + break; + } + case SILInstructionKind::StrongReleaseInst: + cast(user)->setOperand(ref); + break; + default: + break; + } + useIter = nextIter; + } + // If there are unknown users, keep the builtin, and IRGen will handle it. + if (BI->use_empty()) + return eraseInstFromFunction(*BI); + return nullptr; +} + static unsigned getTypeWidth(SILType Ty) { if (auto BuiltinIntTy = Ty.getAs()) { if (BuiltinIntTy->isFixedWidth()) { @@ -541,6 +584,8 @@ SILInstruction *SILCombiner::visitBuiltinInst(BuiltinInst *I) { return optimizeBuiltinCanBeObjCClass(I); if (I->getBuiltinInfo().ID == BuiltinValueKind::IsConcrete) return optimizeBuiltinIsConcrete(I); + if (I->getBuiltinInfo().ID == BuiltinValueKind::COWBufferForReading) + return optimizeBuiltinCOWBufferForReading(I); if (I->getBuiltinInfo().ID == BuiltinValueKind::TakeArrayFrontToBack || I->getBuiltinInfo().ID == BuiltinValueKind::TakeArrayBackToFront || I->getBuiltinInfo().ID == BuiltinValueKind::TakeArrayNoAlias || diff --git a/lib/SILOptimizer/Transforms/AccessEnforcementReleaseSinking.cpp b/lib/SILOptimizer/Transforms/AccessEnforcementReleaseSinking.cpp index ee73a35d78a92..0ff7b4c2f1d9b 100644 --- a/lib/SILOptimizer/Transforms/AccessEnforcementReleaseSinking.cpp +++ b/lib/SILOptimizer/Transforms/AccessEnforcementReleaseSinking.cpp @@ -138,6 +138,7 @@ static bool isBarrier(SILInstruction *inst) { case BuiltinValueKind::PoundAssert: case BuiltinValueKind::TypePtrAuthDiscriminator: case BuiltinValueKind::GlobalStringTablePointer: + case BuiltinValueKind::COWBufferForReading: return false; // Handle some rare builtins that may be sensitive to object lifetime diff --git a/lib/SILOptimizer/UtilityPasses/SerializeSILPass.cpp b/lib/SILOptimizer/UtilityPasses/SerializeSILPass.cpp index bac10dffc61f1..11604e211c9d8 100644 --- a/lib/SILOptimizer/UtilityPasses/SerializeSILPass.cpp +++ b/lib/SILOptimizer/UtilityPasses/SerializeSILPass.cpp @@ -332,6 +332,8 @@ static bool hasOpaqueArchetype(TypeExpansionContext context, case SILInstructionKind::LinearFunctionInst: case SILInstructionKind::LinearFunctionExtractInst: case SILInstructionKind::DifferentiabilityWitnessFunctionInst: + case SILInstructionKind::BeginCOWMutationInst: + case SILInstructionKind::EndCOWMutationInst: // Handle by operand and result check. break; diff --git a/lib/SILOptimizer/Utils/SILInliner.cpp b/lib/SILOptimizer/Utils/SILInliner.cpp index 29df76e4b416f..ded5b22a009b0 100644 --- a/lib/SILOptimizer/Utils/SILInliner.cpp +++ b/lib/SILOptimizer/Utils/SILInliner.cpp @@ -778,6 +778,7 @@ InlineCost swift::instructionInlineCost(SILInstruction &I) { case SILInstructionKind::ThrowInst: case SILInstructionKind::UnwindInst: case SILInstructionKind::YieldInst: + case SILInstructionKind::EndCOWMutationInst: return InlineCost::Free; case SILInstructionKind::AbortApplyInst: @@ -869,6 +870,7 @@ InlineCost swift::instructionInlineCost(SILInstruction &I) { case SILInstructionKind::UnconditionalCheckedCastValueInst: case SILInstructionKind::IsEscapingClosureInst: case SILInstructionKind::IsUniqueInst: + case SILInstructionKind::BeginCOWMutationInst: case SILInstructionKind::InitBlockStorageHeaderInst: case SILInstructionKind::SelectEnumAddrInst: case SILInstructionKind::SelectEnumInst: diff --git a/lib/Serialization/DeserializeSIL.cpp b/lib/Serialization/DeserializeSIL.cpp index 7a650c6eaf1e1..5fbd28c55672b 100644 --- a/lib/Serialization/DeserializeSIL.cpp +++ b/lib/Serialization/DeserializeSIL.cpp @@ -1837,6 +1837,28 @@ bool SILDeserializer::readSILInstruction(SILFunction *Fn, SILBasicBlock *BB, break; } + case SILInstructionKind::BeginCOWMutationInst: { + assert(RecordKind == SIL_ONE_OPERAND && "Layout should be OneOperand."); + unsigned isNative = Attr; + ResultVal = Builder.createBeginCOWMutation( + Loc, + getLocalValue(ValID, + getSILType(MF->getType(TyID), (SILValueCategory)TyCategory, Fn)), + isNative != 0); + break; + } + + case SILInstructionKind::EndCOWMutationInst: { + assert(RecordKind == SIL_ONE_OPERAND && "Layout should be OneOperand."); + unsigned keepUnique = Attr; + ResultVal = Builder.createEndCOWMutation( + Loc, + getLocalValue(ValID, getSILType(MF->getType(TyID), + (SILValueCategory)TyCategory, Fn)), + keepUnique != 0); + break; + } + case SILInstructionKind::DestructureTupleInst: { assert(RecordKind == SIL_ONE_OPERAND && "Layout should be OneOperand."); SILValue Operand = getLocalValue( @@ -2298,19 +2320,19 @@ bool SILDeserializer::readSILInstruction(SILFunction *Fn, SILBasicBlock *BB, auto ResultTy = Val->getType().getFieldType( Field, SILMod, Builder.getTypeExpansionContext()); ResultVal = Builder.createRefElementAddr(Loc, Val, Field, - ResultTy); + ResultTy, /*Immutable*/ Attr & 0x1); break; } case SILInstructionKind::RefTailAddrInst: { assert(RecordKind == SIL_ONE_TYPE_ONE_OPERAND && "Layout should be OneTypeOneOperand."); - assert(Attr == 0); assert((SILValueCategory)TyCategory == SILValueCategory::Address); ResultVal = Builder.createRefTailAddr( Loc, getLocalValue(ValID, getSILType(MF->getType(TyID2), (SILValueCategory)TyCategory2, Fn)), - getSILType(MF->getType(TyID), SILValueCategory::Address, Fn)); + getSILType(MF->getType(TyID), SILValueCategory::Address, Fn), + /*Immutable*/ Attr & 0x1); break; } case SILInstructionKind::ClassMethodInst: diff --git a/lib/Serialization/ModuleFormat.h b/lib/Serialization/ModuleFormat.h index 5d00fbc3d4d5c..f338cde1562f1 100644 --- a/lib/Serialization/ModuleFormat.h +++ b/lib/Serialization/ModuleFormat.h @@ -55,7 +55,7 @@ const uint16_t SWIFTMODULE_VERSION_MAJOR = 0; /// describe what change you made. The content of this comment isn't important; /// it just ensures a conflict if two people change the module format. /// Don't worry about adhering to the 80-column limit for this line. -const uint16_t SWIFTMODULE_VERSION_MINOR = 556; // dont serialize Pattern::isImplicit +const uint16_t SWIFTMODULE_VERSION_MINOR = 557; // COW instructions /// A standard hash seed used for all string hashes in a serialized module. /// diff --git a/lib/Serialization/SerializeSIL.cpp b/lib/Serialization/SerializeSIL.cpp index 891996197c063..0ba068798906f 100644 --- a/lib/Serialization/SerializeSIL.cpp +++ b/lib/Serialization/SerializeSIL.cpp @@ -1288,6 +1288,8 @@ void SILSerializer::writeSILInstruction(const SILInstruction &SI) { case SILInstructionKind::StrongReleaseInst: case SILInstructionKind::StrongRetainInst: case SILInstructionKind::IsUniqueInst: + case SILInstructionKind::BeginCOWMutationInst: + case SILInstructionKind::EndCOWMutationInst: case SILInstructionKind::AbortApplyInst: case SILInstructionKind::EndApplyInst: case SILInstructionKind::ReturnInst: @@ -1309,6 +1311,10 @@ void SILSerializer::writeSILInstruction(const SILInstruction &SI) { Attr = unsigned(SILValue(UOCI).getOwnershipKind()); } else if (auto *IEC = dyn_cast(&SI)) { Attr = IEC->getVerificationType(); + } else if (auto *BCMI = dyn_cast(&SI)) { + Attr = BCMI->isNative(); + } else if (auto *ECMI = dyn_cast(&SI)) { + Attr = ECMI->doKeepUnique(); } writeOneOperandLayout(SI.getKind(), Attr, SI.getOperand(0)); break; @@ -1794,11 +1800,13 @@ void SILSerializer::writeSILInstruction(const SILInstruction &SI) { // where the field decl is streamed as a ValueID. SILValue operand; Decl *tDecl; + unsigned attr = 0; switch (SI.getKind()) { default: llvm_unreachable("Out of sync with parent switch"); case SILInstructionKind::RefElementAddrInst: operand = cast(&SI)->getOperand(); tDecl = cast(&SI)->getField(); + attr = unsigned(cast(&SI)->isImmutable()); break; case SILInstructionKind::StructElementAddrInst: operand = cast(&SI)->getOperand(); @@ -1827,7 +1835,7 @@ void SILSerializer::writeSILInstruction(const SILInstruction &SI) { } SILOneValueOneOperandLayout::emitRecord(Out, ScratchRecord, SILAbbrCodes[SILOneValueOneOperandLayout::Code], - (unsigned)SI.getKind(), 0, S.addDeclRef(tDecl), + (unsigned)SI.getKind(), attr, S.addDeclRef(tDecl), S.addTypeRef(operand->getType().getASTType()), (unsigned)operand->getType().getCategory(), addValueRef(operand)); @@ -1835,7 +1843,7 @@ void SILSerializer::writeSILInstruction(const SILInstruction &SI) { } case SILInstructionKind::RefTailAddrInst: { auto *RTAI = cast(&SI); - writeOneTypeOneOperandLayout(RTAI->getKind(), 0, + writeOneTypeOneOperandLayout(RTAI->getKind(), unsigned(RTAI->isImmutable()), RTAI->getType(), RTAI->getOperand()); break; diff --git a/test/IRGen/builtins.swift b/test/IRGen/builtins.swift index d051f5ede450a..d2dbff0b194c9 100644 --- a/test/IRGen/builtins.swift +++ b/test/IRGen/builtins.swift @@ -683,6 +683,12 @@ func isUniqueIUO(_ ref: inout Builtin.NativeObject?) -> Bool { return Builtin.isUnique(&iuo) } +// CHECK-LABEL: define hidden {{.*}} @"$s8builtins19COWBufferForReadingyAA1CCADnF" +// CHECK: ret %T8builtins1CC* %0 +func COWBufferForReading(_ ref: __owned C) -> C { + return Builtin.COWBufferForReading(ref) +} + // CHECK-LABEL: define {{.*}} @{{.*}}generic_ispod_test func generic_ispod_test(_: T) { // CHECK: [[T0:%.*]] = getelementptr inbounds %swift.vwtable, %swift.vwtable* [[T:%.*]], i32 10 diff --git a/test/Interop/Cxx/reference/Inputs/module.modulemap b/test/Interop/Cxx/reference/Inputs/module.modulemap deleted file mode 100644 index 06aa57364b223..0000000000000 --- a/test/Interop/Cxx/reference/Inputs/module.modulemap +++ /dev/null @@ -1,3 +0,0 @@ -module Reference { - header "reference.h" -} diff --git a/test/Interop/Cxx/reference/Inputs/reference.cpp b/test/Interop/Cxx/reference/Inputs/reference.cpp deleted file mode 100644 index 4e68fcfb42645..0000000000000 --- a/test/Interop/Cxx/reference/Inputs/reference.cpp +++ /dev/null @@ -1,19 +0,0 @@ -#include "reference.h" -#include - -static int staticInt = 42; - -int getStaticInt() { return staticInt; } -int &getStaticIntRef() { return staticInt; } -int &&getStaticIntRvalueRef() { return std::move(staticInt); } -const int &getConstStaticIntRef() { return staticInt; } -const int &&getConstStaticIntRvalueRef() { return std::move(staticInt); } - -void setStaticInt(int i) { staticInt = i; } -void setStaticIntRef(int &i) { staticInt = i; } -void setStaticIntRvalueRef(int &&i) { staticInt = i; } -void setConstStaticIntRef(const int &i) { staticInt = i; } -void setConstStaticIntRvalueRef(const int &&i) { staticInt = i; } - -auto getFuncRef() -> int (&)() { return getStaticInt; } -auto getFuncRvalueRef() -> int (&&)() { return getStaticInt; } diff --git a/test/Interop/Cxx/reference/Inputs/reference.h b/test/Interop/Cxx/reference/Inputs/reference.h deleted file mode 100644 index a5beb2aba9dc3..0000000000000 --- a/test/Interop/Cxx/reference/Inputs/reference.h +++ /dev/null @@ -1,19 +0,0 @@ -#ifndef TEST_INTEROP_CXX_REFERENCE_INPUTS_REFERENCE_H -#define TEST_INTEROP_CXX_REFERENCE_INPUTS_REFERENCE_H - -int getStaticInt(); -int &getStaticIntRef(); -int &&getStaticIntRvalueRef(); -const int &getConstStaticIntRef(); -const int &&getConstStaticIntRvalueRef(); - -void setStaticInt(int); -void setStaticIntRef(int &); -void setStaticIntRvalueRef(int &&); -void setConstStaticIntRef(const int &); -void setConstStaticIntRvalueRef(const int &&); - -auto getFuncRef() -> int (&)(); -auto getFuncRvalueRef() -> int (&&)(); - -#endif // TEST_INTEROP_CXX_REFERENCE_INPUTS_REFERENCE_H diff --git a/test/Interop/Cxx/reference/reference-irgen.swift b/test/Interop/Cxx/reference/reference-irgen.swift deleted file mode 100644 index cf5d26843849a..0000000000000 --- a/test/Interop/Cxx/reference/reference-irgen.swift +++ /dev/null @@ -1,71 +0,0 @@ -// RUN: %target-swift-emit-ir -I %S/Inputs -enable-cxx-interop %s | %FileCheck %s - -import Reference - -public func getCxxRef() -> UnsafeMutablePointer { - return getStaticIntRef() -} - -// CHECK: define {{(protected |dllexport )?}}swiftcc i8* @"$s4main9getCxxRefSpys5Int32VGyF"() -// CHECK: call i32* @{{_Z15getStaticIntRefv|"\?getStaticIntRef@@YAAEAHXZ"}}() - -public func getConstCxxRef() -> UnsafePointer { - return getConstStaticIntRef() -} - -// CHECK: define {{(protected |dllexport )?}}swiftcc i8* @"$s4main14getConstCxxRefSPys5Int32VGyF"() -// CHECK: call i32* @{{_Z20getConstStaticIntRefv|"\?getConstStaticIntRef@@YAAEBHXZ"}}() - -public func getCxxRvalueRef() -> UnsafeMutablePointer { - return getStaticIntRvalueRef() -} - -// CHECK: define {{(protected |dllexport )?}}swiftcc i8* @"$s4main15getCxxRvalueRefSpys5Int32VGyF"() -// CHECK: call i32* @{{_Z21getStaticIntRvalueRefv|"\?getStaticIntRvalueRef@@YA\$\$QEAHXZ"}}() - -public func getConstCxxRvalueRef() -> UnsafePointer { - return getConstStaticIntRvalueRef() -} - -// CHECK: define {{(protected |dllexport )?}}swiftcc i8* @"$s4main20getConstCxxRvalueRefSPys5Int32VGyF"() -// CHECK: call i32* @{{_Z26getConstStaticIntRvalueRefv|"\?getConstStaticIntRvalueRef@@YA\$\$QEBHXZ"}}() - -public func setCxxRef() { - var val: CInt = 21 - withUnsafeMutablePointer(to: &val) { - setStaticIntRef($0) - } -} - -// CHECK: define {{(protected |dllexport )?}}swiftcc void @"$s4main9setCxxRefyyF"() -// CHECK: call void @{{_Z15setStaticIntRefRi|"\?setStaticIntRef@@YAXAEAH@Z"}}(i32* {{nonnull %val|%2}}) - -public func setCxxConstRef() { - var val: CInt = 21 - withUnsafePointer(to: &val) { - setConstStaticIntRef($0) - } -} - -// CHECK: define {{(protected |dllexport )?}}swiftcc void @"$s4main14setCxxConstRefyyF"() -// CHECK: call void @{{_Z20setConstStaticIntRefRKi|"\?setConstStaticIntRef@@YAXAEBH@Z"}}(i32* {{nonnull %val|%2}}) - -public func setCxxRvalueRef() { - var val: CInt = 21 - withUnsafeMutablePointer(to: &val) { - setStaticIntRvalueRef($0) - } -} - -// CHECK: define {{(protected |dllexport )?}}swiftcc void @"$s4main15setCxxRvalueRefyyF"() -// CHECK: call void @{{_Z21setStaticIntRvalueRefOi|"\?setStaticIntRvalueRef@@YAX\$\$QEAH@Z"}}(i32* {{nonnull %val|%2}}) - -public func setCxxConstRvalueRef() { - var val: CInt = 21 - withUnsafePointer(to: &val) { - setConstStaticIntRvalueRef($0) - } -} - -// CHECK: define {{(protected |dllexport )?}}swiftcc void @"$s4main20setCxxConstRvalueRefyyF"() -// CHECK: call void @{{_Z26setConstStaticIntRvalueRefOKi|"\?setConstStaticIntRvalueRef@@YAX\$\$QEBH@Z"}}(i32* {{nonnull %val|%2}}) diff --git a/test/Interop/Cxx/reference/reference-module-interface.swift b/test/Interop/Cxx/reference/reference-module-interface.swift deleted file mode 100644 index 82744a0d6581d..0000000000000 --- a/test/Interop/Cxx/reference/reference-module-interface.swift +++ /dev/null @@ -1,14 +0,0 @@ -// RUN: %target-swift-ide-test -print-module -module-to-print=Reference -I %S/Inputs -source-filename=x -enable-cxx-interop | %FileCheck %s - -// CHECK: func getStaticInt() -> Int32 -// CHECK: func getStaticIntRef() -> UnsafeMutablePointer -// CHECK: func getStaticIntRvalueRef() -> UnsafeMutablePointer -// CHECK: func getConstStaticIntRef() -> UnsafePointer -// CHECK: func getConstStaticIntRvalueRef() -> UnsafePointer -// CHECK: func setStaticInt(_: Int32) -// CHECK: func setStaticIntRef(_: UnsafeMutablePointer) -// CHECK: func setStaticIntRvalueRef(_: UnsafeMutablePointer) -// CHECK: func setConstStaticIntRef(_: UnsafePointer) -// CHECK: func setConstStaticIntRvalueRef(_: UnsafePointer) -// CHECK: func getFuncRef() -> @convention(c) () -> Int32 -// CHECK: func getFuncRvalueRef() -> @convention(c) () -> Int32 diff --git a/test/Interop/Cxx/reference/reference-silgen.swift b/test/Interop/Cxx/reference/reference-silgen.swift deleted file mode 100644 index 9a58efb800319..0000000000000 --- a/test/Interop/Cxx/reference/reference-silgen.swift +++ /dev/null @@ -1,83 +0,0 @@ -// RUN: %target-swift-emit-sil -I %S/Inputs -enable-cxx-interop %s | %FileCheck %s - -import Reference - -func getCxxRef() -> UnsafeMutablePointer { - return getStaticIntRef() -} - -// CHECK: sil hidden @$s4main9getCxxRefSpys5Int32VGyF : $@convention(thin) () -> UnsafeMutablePointer -// CHECK: [[REF:%.*]] = function_ref @{{_Z15getStaticIntRefv|\?getStaticIntRef@@YAAEAHXZ}} : $@convention(c) () -> UnsafeMutablePointer -// CHECK: apply [[REF]]() : $@convention(c) () -> UnsafeMutablePointer - -func getConstCxxRef() -> UnsafePointer { - return getConstStaticIntRef() -} - -// CHECK: sil hidden @$s4main14getConstCxxRefSPys5Int32VGyF : $@convention(thin) () -> UnsafePointer -// CHECK: [[REF:%.*]] = function_ref @{{_Z20getConstStaticIntRefv|\?getConstStaticIntRef@@YAAEBHXZ}} : $@convention(c) () -> UnsafePointer -// CHECK: apply [[REF]]() : $@convention(c) () -> UnsafePointer - -func getCxxRvalueRef() -> UnsafeMutablePointer { - return getStaticIntRvalueRef() -} - -// CHECK: sil hidden @$s4main15getCxxRvalueRefSpys5Int32VGyF : $@convention(thin) () -> UnsafeMutablePointer -// CHECK: [[REF:%.*]] = function_ref @{{_Z21getStaticIntRvalueRefv|\?getStaticIntRvalueRef@@YA\$\$QEAHXZ}} : $@convention(c) () -> UnsafeMutablePointer -// CHECK: apply [[REF]]() : $@convention(c) () -> UnsafeMutablePointer - -func getConstCxxRvalueRef() -> UnsafePointer { - return getConstStaticIntRvalueRef() -} - -// CHECK: sil hidden @$s4main20getConstCxxRvalueRefSPys5Int32VGyF : $@convention(thin) () -> UnsafePointer -// CHECK: [[REF:%.*]] = function_ref @{{_Z26getConstStaticIntRvalueRefv|\?getConstStaticIntRvalueRef@@YA\$\$QEBHXZ}} : $@convention(c) () -> UnsafePointer -// CHECK: apply [[REF]]() : $@convention(c) () -> UnsafePointer - -func setCxxRef() { - var val: CInt = 21 - withUnsafeMutablePointer(to: &val) { - setStaticIntRef($0) - } -} - -// CHECK: // closure #1 in setCxxRef() -// CHECK: sil private @$s4main9setCxxRefyyFySpys5Int32VGXEfU_ : $@convention(thin) (UnsafeMutablePointer) -> () -// CHECK: [[REF:%.*]] = function_ref @{{_Z15setStaticIntRefRi|\?setStaticIntRef@@YAXAEAH@Z}} : $@convention(c) (UnsafeMutablePointer) -> () -// CHECK: apply [[REF]](%0) : $@convention(c) (UnsafeMutablePointer) -> () - -func setCxxConstRef() { - var val: CInt = 21 - withUnsafePointer(to: &val) { - setConstStaticIntRef($0) - } -} - -// CHECK: // closure #1 in setCxxConstRef() -// CHECK: sil private @$s4main14setCxxConstRefyyFySPys5Int32VGXEfU_ : $@convention(thin) (UnsafePointer) -> () -// CHECK: [[REF:%.*]] = function_ref @{{_Z20setConstStaticIntRefRKi|\?setConstStaticIntRef@@YAXAEBH@Z}} : $@convention(c) (UnsafePointer) -> () -// CHECK: apply [[REF]](%0) : $@convention(c) (UnsafePointer) -> () - -func setCxxRvalueRef() { - var val: CInt = 21 - withUnsafeMutablePointer(to: &val) { - setStaticIntRvalueRef($0) - } -} - -// CHECK: // closure #1 in setCxxRvalueRef() -// CHECK: sil private @$s4main15setCxxRvalueRefyyFySpys5Int32VGXEfU_ : $@convention(thin) (UnsafeMutablePointer) -> () -// CHECK: [[REF:%.*]] = function_ref @{{_Z21setStaticIntRvalueRefOi|\?setStaticIntRvalueRef@@YAX\$\$QEAH@Z}} : $@convention(c) (UnsafeMutablePointer) -> () -// CHECK: apply [[REF]](%0) : $@convention(c) (UnsafeMutablePointer) -> () - -func setCxxConstRvalueRef() { - var val: CInt = 21 - withUnsafePointer(to: &val) { - setConstStaticIntRvalueRef($0) - } -} - -// CHECK: // closure #1 in setCxxConstRvalueRef() -// CHECK: sil private @$s4main20setCxxConstRvalueRefyyFySPys5Int32VGXEfU_ : $@convention(thin) (UnsafePointer) -> () -// CHECK: [[REF:%.*]] = function_ref @{{_Z26setConstStaticIntRvalueRefOKi|\?setConstStaticIntRvalueRef@@YAX\$\$QEBH@Z}} : $@convention(c) (UnsafePointer) -> () -// CHECK: apply [[REF]](%0) : $@convention(c) (UnsafePointer) -> () diff --git a/test/Interop/Cxx/reference/reference.swift b/test/Interop/Cxx/reference/reference.swift deleted file mode 100644 index 71d211b04761c..0000000000000 --- a/test/Interop/Cxx/reference/reference.swift +++ /dev/null @@ -1,92 +0,0 @@ -// RUN: %empty-directory(%t) -// RUN: %target-clang -c %S/Inputs/reference.cpp -I %S/Inputs -o %t/reference.o -std=c++17 -// RUN: %target-build-swift %s -I %S/Inputs -o %t/reference %t/reference.o -Xfrontend -enable-cxx-interop -Xcc -std=c++17 -// RUN: %target-codesign %t/reference -// RUN: %target-run %t/reference -// -// REQUIRES: executable_test - -import Reference -import StdlibUnittest - -var ReferenceTestSuite = TestSuite("Reference") - -ReferenceTestSuite.test("read-lvalue-reference") { - expectNotEqual(13, getStaticInt()) - setStaticInt(13) - expectEqual(13, getStaticIntRef().pointee) - expectEqual(13, getConstStaticIntRef().pointee) -} - -ReferenceTestSuite.test("read-rvalue-reference") { - expectNotEqual(32, getStaticInt()) - setStaticInt(32) - expectEqual(32, getStaticIntRvalueRef().pointee) - expectEqual(32, getConstStaticIntRvalueRef().pointee) -} - -ReferenceTestSuite.test("write-lvalue-reference") { - expectNotEqual(14, getStaticInt()) - getStaticIntRef().pointee = 14 - expectEqual(14, getStaticInt()) -} - -ReferenceTestSuite.test("write-rvalue-reference") { - expectNotEqual(41, getStaticInt()) - getStaticIntRvalueRef().pointee = 41 - expectEqual(41, getStaticInt()) -} - -ReferenceTestSuite.test("pass-lvalue-reference") { - expectNotEqual(21, getStaticInt()) - var val: CInt = 21 - withUnsafeMutablePointer(to: &val) { - setStaticIntRef($0) - } - expectEqual(21, getStaticInt()) -} - -ReferenceTestSuite.test("pass-const-lvalue-reference") { - expectNotEqual(22, getStaticInt()) - var val: CInt = 22 - withUnsafePointer(to: &val) { - setConstStaticIntRef($0) - } - expectEqual(22, getStaticInt()) -} - -ReferenceTestSuite.test("pass-rvalue-reference") { - expectNotEqual(52, getStaticInt()) - var val: CInt = 52 - withUnsafeMutablePointer(to: &val) { - setStaticIntRvalueRef($0) - } - expectEqual(52, getStaticInt()) -} - -ReferenceTestSuite.test("pass-const-rvalue-reference") { - expectNotEqual(53, getStaticInt()) - var val: CInt = 53 - withUnsafePointer(to: &val) { - setConstStaticIntRvalueRef($0) - } - expectEqual(53, getStaticInt()) -} - -ReferenceTestSuite.test("func-reference") { - let cxxF: @convention(c) () -> Int32 = getFuncRef() - - expectNotEqual(15, getStaticInt()) - setStaticInt(15) - expectEqual(15, cxxF()) -} - -ReferenceTestSuite.test("func-rvalue-reference") { - let cxxF: @convention(c) () -> Int32 = getFuncRvalueRef() - - expectNotEqual(61, getStaticInt()) - setStaticInt(61) - expectEqual(61, cxxF()) -} - -runAllTests() diff --git a/test/SIL/Parser/basic.sil b/test/SIL/Parser/basic.sil index f1ffb71a6fd22..ccc691560c6b8 100644 --- a/test/SIL/Parser/basic.sil +++ b/test/SIL/Parser/basic.sil @@ -775,6 +775,25 @@ bb0(%0 : $Class1, %1 : $Builtin.Word): return %0 : $Class1 } +// CHECK-LABEL: sil @test_cow_mutation +sil @test_cow_mutation : $@convention(thin) (@owned Class1) -> @owned Class1 { +bb0(%0 : $Class1): + // CHECK: (%1, %2) = begin_cow_mutation %0 : $Class1 + (%1, %2) = begin_cow_mutation %0 : $Class1 + // CHECK: %3 = end_cow_mutation [keep_unique] %2 : $Class1 + %3 = end_cow_mutation [keep_unique] %2 : $Class1 + // CHECK: (%4, %5) = begin_cow_mutation [native] %0 : $Class1 + (%4, %5) = begin_cow_mutation [native] %0 : $Class1 + // CHECK: %6 = end_cow_mutation %5 : $Class1 + %6 = end_cow_mutation %5 : $Class1 + // CHECK: [[T:%[0-9]+]] = ref_tail_addr [immutable] %3 : $Class1, $Val + %7 = ref_tail_addr [immutable] %3 : $Class1, $Val + // CHECK: ref_element_addr [immutable] %3 : $Class1, #Class1.a + %8 = ref_element_addr [immutable] %3 : $Class1, #Class1.a + return %0 : $Class1 +} + + // CHECK-LABEL: closure_test sil @takes_closure : $@convention(thin) (@callee_owned () -> ()) -> () diff --git a/test/SILGen/builtins.swift b/test/SILGen/builtins.swift index fa7792c4c3aab..0589e64fd985b 100644 --- a/test/SILGen/builtins.swift +++ b/test/SILGen/builtins.swift @@ -600,6 +600,32 @@ func castBitPatternFromBridgeObject(_ bo: Builtin.BridgeObject) -> Builtin.Word return Builtin.castBitPatternFromBridgeObject(bo) } +// CHECK-LABEL: sil hidden [ossa] @$s8builtins16beginCOWMutationySbAA1CCzF +// CHECK: [[L:%.*]] = load [take] [[ADDR:%[0-9]*]] +// CHECK: ([[U:%.*]], [[B:%.*]]) = begin_cow_mutation [[L]] +// CHECK: store [[B]] to [init] [[ADDR]] +// CHECK: apply {{%[0-9]*}}([[U]] +func beginCOWMutation(_ c: inout C) -> Bool { + return Bool(_builtinBooleanLiteral: Builtin.beginCOWMutation(&c)) +} + +// CHECK-LABEL: sil hidden [ossa] @$s8builtins23beginCOWMutation_nativeySbAA1CCzF +// CHECK: [[L:%.*]] = load [take] [[ADDR:%[0-9]*]] +// CHECK: ([[U:%.*]], [[B:%.*]]) = begin_cow_mutation [native] [[L]] +// CHECK: store [[B]] to [init] [[ADDR]] +// CHECK: apply {{%[0-9]*}}([[U]] +func beginCOWMutation_native(_ c: inout C) -> Bool { + return Bool(_builtinBooleanLiteral: Builtin.beginCOWMutation_native(&c)) +} + +// CHECK-LABEL: sil hidden [ossa] @$s8builtins14endCOWMutationyyAA1CCzF +// CHECK: [[L:%.*]] = load [take] [[ADDR:%[0-9]*]] +// CHECK: [[B:%.*]] = end_cow_mutation [[L]] +// CHECK: store [[B]] to [init] [[ADDR]] +func endCOWMutation(_ c: inout C) { + Builtin.endCOWMutation(&c) +} + // ---------------------------------------------------------------------------- // isUnique variants // ---------------------------------------------------------------------------- diff --git a/test/Serialization/Inputs/def_basic.sil b/test/Serialization/Inputs/def_basic.sil index e0c98537cb0b0..d9aeaa7ee4b0d 100644 --- a/test/Serialization/Inputs/def_basic.sil +++ b/test/Serialization/Inputs/def_basic.sil @@ -707,6 +707,24 @@ bb0(%0 : $Class1, %1 : $Builtin.Word): return %0 : $Class1 } +// CHECK-LABEL: sil public_external [transparent] [serialized] @test_cow_mutation +sil [transparent] [serialized] @test_cow_mutation : $@convention(thin) (@owned Class1) -> @owned Class1 { +bb0(%0 : $Class1): + // CHECK: (%1, %2) = begin_cow_mutation %0 : $Class1 + (%1, %2) = begin_cow_mutation %0 : $Class1 + // CHECK: %3 = end_cow_mutation [keep_unique] %2 : $Class1 + %3 = end_cow_mutation [keep_unique] %2 : $Class1 + // CHECK: (%4, %5) = begin_cow_mutation [native] %0 : $Class1 + (%4, %5) = begin_cow_mutation [native] %0 : $Class1 + // CHECK: %6 = end_cow_mutation %5 : $Class1 + %6 = end_cow_mutation %5 : $Class1 + // CHECK: [[T:%[0-9]+]] = ref_tail_addr [immutable] %3 : $Class1, $Val + %7 = ref_tail_addr [immutable] %3 : $Class1, $Val + // CHECK: ref_element_addr [immutable] %3 : $Class1, #Class1.a + %8 = ref_element_addr [immutable] %3 : $Class1, #Class1.a + return %0 : $Class1 +} + // CHECK-LABEL: @closure_test sil [transparent] [serialized] @takes_closure : $@convention(thin) (@callee_owned () -> ()) -> () sil [transparent] [serialized] @closure0 : $@convention(thin) (<τ_0_0> { var τ_0_0 } , @inout Int) -> () @@ -1489,7 +1507,8 @@ bb0: %87 = function_ref @test_tail_elems : $@convention(thin) (Builtin.Word, Builtin.Word) -> () %88 = function_ref @test_tail_elems_dynamic : $@convention(thin) (Builtin.Word, Builtin.Word, @thick Class1.Type) -> () %89 = function_ref @test_tail_addr : $@convention(thin) (@owned Class1, Builtin.Word) -> @owned Class1 - %90 = function_ref @closure_test : $@convention(thin) () -> () + %90 = function_ref @test_cow_mutation : $@convention(thin) (@owned Class1) -> @owned Class1 + %91 = function_ref @closure_test : $@convention(thin) () -> () %97 = function_ref @test_switch_union : $@convention(thin) (MaybePair) -> () %99 = function_ref @test_switch_union_addr : $@convention(thin) (@in MaybeAddressOnlyPair) -> () %101 = function_ref @test_switch_value : $@convention(thin) (Builtin.Word) -> ()