[SR-13328] Conditional differentiability crash

[dabrahams]

Previous ID	SR-13328
Radar	None
Original Reporter	@dabrahams
Type	Bug

Additional Detail from JIRA

Votes	0
Component/s	Swift for TensorFlow
Labels	Bug
Assignee	@dan-zheng
Priority	Medium

md5: 68fa318b048a4316d5f49084bee400ec

Issue Description:

From github.com:borglab/SwiftFusion.git, check out the conditional-differentiability-crash and swift test it for this crash:

Stack dump:
0.  Program arguments: /Library/Developer/Toolchains/swift-tensorflow-DEVELOPMENT-2020-06-03-a.xctoolchain/usr/bin/swift -frontend -c /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/AnyArrayBufferProtocol.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/AnyDifferentiable.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/ArraySlices.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/DataTypes.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/Dictionary+Differentiable.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/EuclideanVector.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/EuclideanVectorN.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/FixedSizeMatrix.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/Jacobian.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/LieGroup.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/Manifold.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/MathUtil.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/Matrix.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/TrappingDouble.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/Tuple+EuclideanVectorN.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/TypeKeyedArrayBuffers.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/Vector.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Core/VectorN.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Datasets/DatasetCache.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Datasets/G2OReader.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Geometry/Pose2.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Geometry/Pose3.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Geometry/Rot2.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Geometry/Rot3.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/AllVectors.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/AnyArrayBuffer+Differentiable.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/ArrayBuffer+Differentiable.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/ArrayBuffer+Tensor.swift -primary-file /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/ArrayBuffer+Vector.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/ArrayStorage+Tensor.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/BearingRangeFactor.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/BetweenFactor.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/BetweenFactorAlternative.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/ChordalInitialization.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/Factor.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/FactorBoilerplate.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/FactorGraph.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/FactorsStorage.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/GaussianFactorGraph.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/IdentityLinearizationFactor.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/JacobianFactor.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/PenguinExtensions.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/PriorFactor.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/ScalarJacobianFactor.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/ValuesStorage.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/VariableAssignments.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/MCMC/RandomWalkMetropolis.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/MCMC/TransitionKernel.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/MCMC/sample.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Optimizers/CGLS.swift /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Optimizers/LM.swift -emit-module-path /Users/dabrahams/src/SwiftFusion/.build/x86_64-apple-macosx/debug/SwiftFusion.build/Inference/ArrayBuffer+Vector~partial.swiftmodule -emit-module-doc-path /Users/dabrahams/src/SwiftFusion/.build/x86_64-apple-macosx/debug/SwiftFusion.build/Inference/ArrayBuffer+Vector~partial.swiftdoc -emit-module-source-info-path /Users/dabrahams/src/SwiftFusion/.build/x86_64-apple-macosx/debug/SwiftFusion.build/Inference/ArrayBuffer+Vector~partial.swiftsourceinfo -emit-dependencies-path /Users/dabrahams/src/SwiftFusion/.build/x86_64-apple-macosx/debug/SwiftFusion.build/Inference/ArrayBuffer+Vector.d -emit-reference-dependencies-path /Users/dabrahams/src/SwiftFusion/.build/x86_64-apple-macosx/debug/SwiftFusion.build/Inference/ArrayBuffer+Vector.swiftdeps -target x86_64-apple-macosx10.14 -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -I /Users/dabrahams/src/SwiftFusion/.build/x86_64-apple-macosx/debug -I /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib -F /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -enable-testing -g -module-cache-path /Users/dabrahams/src/SwiftFusion/.build/x86_64-apple-macosx/debug/ModuleCache -swift-version 5 -Onone -D SWIFT_PACKAGE -D DEBUG -enable-anonymous-context-mangled-names -target-sdk-version 10.15.6 -parse-as-library -module-name SwiftFusion -o /Users/dabrahams/src/SwiftFusion/.build/x86_64-apple-macosx/debug/SwiftFusion.build/Inference/ArrayBuffer+Vector.swift.o -index-store-path /Users/dabrahams/src/SwiftFusion/.build/x86_64-apple-macosx/debug/index/store -index-system-modules 
1.  Swift version 5.3-dev (LLVM 37f84d51a3, Swift 91d4726b3d)
2.  While evaluating request IRGenSourceFileRequest(IR Generation for file "/Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/ArrayBuffer+Vector.swift")
3.  While emitting IR SIL function "@AD__$s17PenguinStructures11ArrayBufferVyxG11SwiftFusion15EuclideanVectorA2eFRzlAeFP1soiyxx_xtFZTW_vjp_SSU".
 for '-(_:_:)' (at /Users/dabrahams/src/SwiftFusion/Sources/SwiftFusion/Inference/ArrayBuffer+Tensor.swift:124:10)
0  swift                    0x0000000107653b58 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1  swift                    0x0000000107652e15 llvm::sys::RunSignalHandlers() + 85
2  swift                    0x0000000107654126 SignalHandler(int) + 262
3  libsystem_platform.dylib 0x00007fff72d825fd _sigtramp + 29
4  libsystem_platform.dylib 0x00007ffeec5ae870 _sigtramp + 18446744071453196944
5  swift                    0x00000001038c3583 swift::irgen::IRGenModule::getConformanceInfo(swift::ProtocolDecl const*, swift::ProtocolConformance const*) + 163
6  swift                    0x00000001038c6866 swift::irgen::emitWitnessTableRef(swift::irgen::IRGenFunction&, swift::CanType, llvm::Value**, swift::ProtocolConformanceRef) + 166
7  swift                    0x00000001038c732a swift::irgen::emitGenericRequirementFromSubstitutions(swift::irgen::IRGenFunction&, swift::CanGenericSignature, swift::ModuleDecl&, swift::irgen::GenericRequirement, swift::SubstitutionMap) + 154
8  swift                    0x00000001038caf33 void llvm::function_ref<void (swift::irgen::GenericRequirement)>::callback_fn<(anonymous namespace)::EmitPolymorphicArguments::emit(swift::SubstitutionMap, swift::irgen::WitnessMetadata*, swift::irgen::Explosion&)::$_21>(long, swift::irgen::GenericRequirement) + 35
9  swift                    0x00000001038cb03d void llvm::function_ref<void (swift::irgen::GenericRequirement)>::callback_fn<(anonymous namespace)::PolymorphicConvention::enumerateUnfulfilledRequirements(llvm::function_ref<void (swift::irgen::GenericRequirement)> const&)::$_11>(long, swift::irgen::GenericRequirement) + 205
10 swift                    0x00000001038c6e74 (anonymous namespace)::PolymorphicConvention::enumerateUnfulfilledRequirements(llvm::function_ref<void (swift::irgen::GenericRequirement)> const&) + 180
11 swift                    0x00000001038c69da swift::irgen::emitPolymorphicArguments(swift::irgen::IRGenFunction&, swift::CanTypeWrapper<swift::SILFunctionType>, swift::SubstitutionMap, swift::irgen::WitnessMetadata*, swift::irgen::Explosion&) + 314
12 swift                    0x0000000103941896 (anonymous namespace)::IRGenSILFunction::visitFullApplySite(swift::FullApplySite) + 3638
13 swift                    0x00000001039254d8 swift::irgen::IRGenModule::emitSILFunction(swift::SILFunction*) + 6792
14 swift                    0x0000000103834668 swift::irgen::IRGenerator::emitLazyDefinitions() + 1176
15 swift                    0x00000001038ff9b0 performIRGeneration(swift::IRGenOptions const&, swift::ModuleDecl*, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, llvm::StringRef, swift::PrimarySpecificPaths const&, llvm::StringRef, swift::SourceFile*, llvm::GlobalVariable**, llvm::StringSet<llvm::MallocAllocator>*) + 1472
16 swift                    0x00000001038ffeb0 swift::IRGenSourceFileRequest::evaluate(swift::Evaluator&, swift::IRGenDescriptor) const + 80
17 swift                    0x0000000103923a15 swift::SimpleRequest<swift::IRGenSourceFileRequest, swift::GeneratedModule (swift::IRGenDescriptor), (swift::RequestFlags)9>::evaluateRequest(swift::IRGenSourceFileRequest const&, swift::Evaluator&) + 37
18 swift                    0x00000001039039d9 llvm::Expected<swift::IRGenSourceFileRequest::OutputType> swift::Evaluator::getResultUncached<swift::IRGenSourceFileRequest>(swift::IRGenSourceFileRequest const&) + 473
19 swift                    0x00000001038ffe0e swift::performIRGeneration(swift::IRGenOptions const&, swift::SourceFile&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, llvm::StringRef, swift::PrimarySpecificPaths const&, llvm::StringRef, llvm::GlobalVariable**, llvm::StringSet<llvm::MallocAllocator>*) + 286
20 swift                    0x00000001036b9511 performCompileStepsPostSILGen(swift::CompilerInstance&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpe

[dabrahams]

Changing line 98 of ArrayBuffer+Tensor.swift to

extension ArrayBuffer: AdditiveArithmetic
  where Element: AdditiveArithmetic, Element: Differentiable {

suppresses the crash.

SR-13329 was an attempt to reproduce this in the small, but a similar attempt to work around that one does not succeed.

[BradLarson]

I'm almost entirely sure this has since been fixed, so I'll close it for now, but if we encounter it in the future when completing the migration of SwiftFusion off the old Swift for TensorFlow toolchains I'll be sure to reopen it.