Skip to content

[AutoDiff] [API] Rewrite adjoints with VJPs. #21740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jan 16, 2019
Merged

[AutoDiff] [API] Rewrite adjoints with VJPs. #21740

merged 2 commits into from
Jan 16, 2019

Conversation

dan-zheng
Copy link
Contributor

@dan-zheng dan-zheng commented Jan 9, 2019 •
edited
Loading

Rewrite all @differentiable(adjoint: ...) usages with @differentiable(vjp: ...)
in the stdlib, including core and TensorFlow.

Add logSoftmax differentiation runtime test.

@dan-zheng dan-zheng added the tensorflow This is for "tensorflow" branch PRs. label Jan 9, 2019
@dan-zheng dan-zheng requested a review from rxwei January 9, 2019 15:58
@dan-zheng dan-zheng force-pushed the revamp-stdlib-differentiable branch from afc2063 to 414a554 Compare January 9, 2019 16:02
@dan-zheng
Copy link
Contributor Author

I believe test/TensorFlowRuntime/tensor_autodiff_runtime.swift crashes during partitioning at GraphFunctionDeviceInfo::finalizeUsedDevices(). Gotta debug.

@swift-ci Please test tensorflow

@dan-zheng dan-zheng force-pushed the revamp-stdlib-differentiable branch 2 times, most recently from d8e40da to dc4598a Compare January 9, 2019 16:13
@dan-zheng
Copy link
Contributor Author

@swift-ci Please clean test tensorflow

@dan-zheng
Copy link
Contributor Author

dan-zheng commented Jan 9, 2019 •
edited
Loading

test/TensorFlowRuntime/tensor_autodiff_runtime.swift currently crashes during partitioning:

Assertion failed: (!multiDeviceWithRuntimeDevice && "Cannot yet handle a multi-device function involving the " "RUNTIME device"), function finalizeUsedDevices, file /Users/dan/swift-build/swift/lib/SIL/GraphFunctionDeviceInfo.cpp, line 204.
Stack dump:
0.	Program arguments: /Users/dan/swift-build/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64/bin/swift -frontend -c -primary-file /Users/dan/swift-build/swift/test/TensorFlowRuntime/tensor_autodiff_runtime.swift -target x86_64-apple-macosx10.9 -enable-objc-interop -sdk /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -F /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -F /Users/dan/swift-build/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64/lib -module-cache-path /Users/dan/swift-build/build/Ninja-ReleaseAssert+swift-DebugAssert/swift-macosx-x86_64/swift-test-results/x86_64-apple-macosx10.9/clang-module-cache -swift-version 4 -O -module-name main -o /var/folders/_n/jrt44kln18v1b79b9xg4fb140000gp/T/lit_tmp_U2L0dC/tensor_autodiff_runtime-d68609.o
1.	While running pass #0 SILModuleTransform "TFPartition".
2.	TFPartition on function $s10TensorFlow0A0VAAs14DifferentiableRzSFRzrlE12_vjpMultiply3lhs3rhsACyxG_AH_AHtAHctAH_AHtFZAH_AHtAHcfU_Sf_Tg5
0  swift                    0x000000010d762758 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1  swift                    0x000000010d761728 llvm::sys::RunSignalHandlers() + 248
2  swift                    0x000000010d762d72 SignalHandler(int) + 258
3  libsystem_platform.dylib 0x00007fff6754db3d _sigtramp + 29
4  swift                    0x000000010eb6be43 cmark_strbuf__initbuf + 779646
5  libsystem_c.dylib        0x00007fff6740b1c9 abort + 127
6  libsystem_c.dylib        0x00007fff673d3868 basename_r + 0
7  swift                    0x00000001099c31d0 swift::tf::GraphFunctionDeviceInfo::finalizeUsedDevices() + 240
8  swift                    0x000000010951e913 (anonymous namespace)::TFFunctionPartition::markFunction(bool&) + 803
9  swift                    0x000000010951dcff (anonymous namespace)::TFPartition::partitionFunction(swift::SILFunction*, std::__1::unique_ptr<(anonymous namespace)::TFFunctionPartition, std::__1::default_delete<(anonymous namespace)::TFFunctionPartition> >&) + 1679
10 swift                    0x000000010951bc6d (anonymous namespace)::TFPartition::processFunction(swift::SILFunction*, llvm::SmallVectorImpl<std::__1::pair<swift::SILFunction*, std::__1::unique_ptr<(anonymous namespace)::TFFunctionPartition, std::__1::default_delete<(anonymous namespace)::TFFunctionPartition> > > >&) + 605
11 swift                    0x000000010951a0f9 (anonymous namespace)::TFPartition::run() + 1049

Full log here.

However, test/TensorFlowRuntime/tensor_autodiff_runtime.swift passes fine with RUN: %target-run-dynamic-compilation-swift.

@rxwei Shall we temporarily disable the test in GPE mode (and file a bug) and enable the test in dynamic compilation mode to unblock the stdlib revamp and model development?

EDIT: Never mind, there are also other test/AutoDiff test failures.

@dan-zheng dan-zheng force-pushed the revamp-stdlib-differentiable branch from dc4598a to 2678306 Compare January 13, 2019 09:26
@dan-zheng
[AutoDiff] [API] Rewrite adjoints with VJPs in the stdlib.
ed7338c 
Rewrite all `@differentiable(adjoint: ...)` usages with `@differentiable(vjp: ...)`
in the stdlib, including `core` and `TensorFlow`.

Add `logSoftmax` differentiation runtime test.
@dan-zheng dan-zheng force-pushed the revamp-stdlib-differentiable branch from 2678306 to ed7338c Compare January 16, 2019 08:11
@dan-zheng
Update test.
a566d0f 
Use VJP instead of adjoint.
@dan-zheng
Copy link
Contributor Author

@swift-ci Please test tensorflow

@rxwei rxwei merged commit 9e96980 into swiftlang:tensorflow Jan 16, 2019
@jekbradbury jekbradbury mentioned this pull request Jan 16, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rxwei rxwei Awaiting requested review from rxwei

Assignees

No one assigned

Labels

tensorflow This is for "tensorflow" branch PRs.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dan-zheng @rxwei