[AutoDiff upstream] Add differentiation transform. #30781
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The differentiation transform does the following:
- Canonicalizes differentiability witnesses by filling in missing derivative
function entries.
- Canonicalizes `differentiable_function` instructions by filling in missing
derivative function operands.
- If necessary, performs automatic differentiation: generating derivative
functions for original functions.
- When encountering non-differentiability code, produces a diagnostic and
errors out.
Partially resolves TF-1211: add the main canonicalization loop.
To incrementally stage changes, derivative functions are currently created
with empty bodies that fatal error with a nice message.
Derivative emitters will be upstreamed separately.
rxwei
approved these changes
Apr 2, 2020
dan-zheng
commented
Apr 2, 2020
There was a problem hiding this comment.
I'll work on canonicalizing differentiable_function instructions next.
Differential operators trap right now because differentiable_function instructions are filled in with undef.
import _Differentiation
@differentiable
func basic(_ x: Float) -> Float {
return x + 2
}
print(gradient(at: 3, in: basic))
$ swift grad.swift Assertion failed: (getStrongExtraRefCount() >= dec && "releasing reference whose refcount is already zero"), function decrementStrongExtraRefCount, file /Users/danielzheng/swift-merge/swift/include/swift/Runtime/../../../stdlib/public/SwiftShims/RefCount.h, line 573. Stack dump: 0. Program arguments: swift -frontend -interpret grad.swift -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -color-diagnostics -module-name grad 1. Swift version 5.3-dev (LLVM aa70751bec, Swift 941f24e378) 2. While running user code "grad.swift" 0 swift 0x000000010ff56fc5 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 37 1 swift 0x000000010ff55f78 llvm::sys::RunSignalHandlers() + 248 2 swift 0x000000010ff575bc SignalHandler(int) + 268 3 libsystem_platform.dylib 0x00007fff6deebb5d _sigtramp + 29 4 libsystem_platform.dylib 0x00000000000001a0 _sigtramp + 18446603338671801952 5 libsystem_c.dylib 0x00007fff6dda56a6 abort + 127 6 libsystem_c.dylib 0x00007fff6dd6e20d basename_r + 0 7 libswiftCore.dylib 0x000000011922ea83 bool swift::RefCounts >::doDecrement<(swift::PerformDeinit)1>(unsigned int) (.cold.2) + 35 8 libswiftCore.dylib 0x00000001191c2cd3 bool swift::RefCounts >::doDecrement<(swift::PerformDeinit)1>(unsigned int) + 147 9 libswiftCore.dylib 0x000000011963f0e3 bool swift::RefCounts >::doDecrement<(swift::PerformDeinit)1>(unsigned int) + 4703395 10 swift 0x000000010bac25be llvm::orc::runAsMain(int (*)(int, char**), llvm::ArrayRef, std::__1::allocator > >, llvm::Optional) + 1694 11 swift 0x000000010baa1f0a swift::RunImmediately(swift::CompilerInstance&, std::__1::vector, std::__1::allocator >, std::__1::allocator, std::__1::allocator > > > const&, swift::IRGenOptions const&, swift::SILOptions const&, std::__1::unique_ptr >&&) + 5674 12 swift 0x000000010ba85869 performCompileStepsPostSILGen(swift::CompilerInstance&, swift::CompilerInvocation const&, std::__1::unique_ptr >, bool, llvm::PointerUnion, swift::PrimarySpecificPaths const&, bool, int&, swift::FrontendObserver*) + 2089 13 swift 0x000000010ba79d6b performCompile(swift::CompilerInstance&, swift::CompilerInvocation const&, llvm::ArrayRef, int&, swift::FrontendObserver*) + 9403 14 swift 0x000000010ba76929 swift::performFrontend(llvm::ArrayRef, char const*, void*, swift::FrontendObserver*) + 3593 15 swift 0x000000010ba0a72d main + 861 16 libdyld.dylib 0x00007fff6dd003d5 start + 1 17 libdyld.dylib 0x000000000000000a start + 18446603338673814582 [1] 38798 abort swift grad.swift
|
@swift-ci Please test |
|
Build failed |
Emit symbols for `@differentiable` and `@derivative` declaration attributes: - Differentiability witness symbols. - Derivative function (JVP/VJP) symbols. - Linear map (differential/pullback) symbols. Add TBDGen test.
dan-zheng
force-pushed
the
differentiation-transform-skeleton
branch
from
acd7816
to
4671dc5
Compare
|
@swift-ci Please test |
|
Build failed |
dan-zheng
commented
Apr 3, 2020
| // RUN: %target-swift-frontend -emit-ir -o/dev/null -parse-as-library -module-name test -validate-tbd-against-ir=all %s -O | ||
| // RUN: %target-swift-frontend -emit-ir -o/dev/null -parse-as-library -module-name test -validate-tbd-against-ir=missing %s -enable-testing | ||
| // RUN: %target-swift-frontend -emit-ir -o/dev/null -parse-as-library -module-name test -validate-tbd-against-ir=missing %s -enable-testing -O | ||
|
|
There was a problem hiding this comment.
I removed some lines from the test:
// RUN: %empty-directory(%t)
// RUN: %target-swift-frontend -typecheck -parse-as-library -module-name test %s -emit-tbd -emit-tbd-path %t/typecheck.tbd
// RUN: %target-swift-frontend -emit-ir -parse-as-library -module-name test %s -emit-tbd -emit-tbd-path %t/emit-ir.tbd
// RUN: diff -u %t/typecheck.tbd %t/emit-ir.tbdThese seem to crash on Linux: swift::writeTBDFile calls MachO APIs, which seems Darwin-specific.
Some but not all TBD tests use REQUIRES: VENDOR=apple.
unexpected platform UNREACHABLE executed at llvm-project/llvm/lib/TextAPI/MachO/TextStubCommon.cpp:60! Stack dump: 0. Program arguments: build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_64/bin/swift -frontend -target x86_64-unknown-linux-gnu -module-cache-path build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_ 64/swift-test-results/x86_64-unknown-linux-gnu/clang-module-cache -swift-version 4 -ignore-module-source-info -typo-correction-limit 10 -typecheck -parse-as-library -module-name test swift/test/AutoDiff/TBD/derivative_symbols.swift -emit-tbd -emit -tbd-path build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_64/test-linux-x86_64/AutoDiff/TBD/Output/derivative_symbols.swift.tmp/typecheck.tbd 1. Swift version 5.3-dev (LLVM 898aa0b6a4, Swift acd7816780) #0 0x0000000005112474 PrintStackTraceSignalHandler(void*) (build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_64/bin/swift+0x5112474) #1 0x000000000511006e llvm::sys::RunSignalHandlers() (build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_64/bin/swift+0x511006e) #2 0x000000000511274c SignalHandler(int) (build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_64/bin/swift+0x511274c) #3 0x00007fd0807f1890 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12890) #4 0x00007fd07ee5ce97 raise /build/glibc-OTsEL5/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:51:0 #5 0x00007fd07ee5e801 abort /build/glibc-OTsEL5/glibc-2.27/stdlib/abort.c:81:0 #6 0x00000000050685b6 (build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_64/bin/swift+0x50685b6) #7 0x0000000004c7c625 llvm::yaml::ScalarTraits >, void>::output(llvm::SmallSet > const&, void*, llvm::raw_ostream&) (/home/dan ielzheng/swift-master/build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_64/bin/swift+0x4c7c625) #8 0x0000000004c7b1f2 std::enable_if > >::value, void>::type llvm::yaml::yamlize > >(llvm::yam l::IO&, llvm::SmallSet >&, bool, llvm::yaml::EmptyContext&) (build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_64/bin/swift+0x4c7b1f2) #9 0x0000000004c6c798 llvm::yaml::MappingTraits::mapKeysToValues(llvm::MachO::FileType, llvm::yaml::IO&, llvm::MachO::InterfaceFile const*&) (build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_64/bin/swift +0x4c6c798) #10 0x0000000004c6b0e2 llvm::MachO::TextAPIWriter::writeToStream(llvm::raw_ostream&, llvm::MachO::InterfaceFile const&) (build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_64/bin/swift+0x4c6b0e2) #11 0x00000000007a147b swift::writeTBDFile(swift::ModuleDecl*, llvm::raw_ostream&, swift::TBDGenOptions const&) (build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_64/bin/swift+0x7a147b) #12 0x00000000005723da swift::writeTBD(swift::ModuleDecl*, llvm::StringRef, swift::TBDGenOptions const&) (build/Ninja-ReleaseAssert+stdlib-Release/swift-linux-x86_64/bin/swift+0x5723da)
|
Build failed |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The differentiation transform does the following:
function entries.
differentiable_functioninstructions by filling in missingderivative function operands.
functions for original functions.
errors out.
Partially resolves TF-1211: add the main canonicalization loop.
To incrementally stage changes:
error with a nice message.
differentiable_functioninstructions are filled in withundefderivativefunction operands.
Proper canonicalization logic will be upstreamed separately.
Also update TBDGen: emit AutoDiff symbols based on
@differentiableand@derivativedeclaration attributes.