Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AutoDiff] Diagnose unsupported coroutine differentiation. #28921

Merged
merged 2 commits into from Dec 21, 2019

Conversation

dan-zheng
Copy link
Collaborator

Coroutines are functions that can yield values and suspend/resume execution.
SIL has dedicated coroutine types. Coroutines are applied via begin_apply.
The read and modify accessors are coroutines.

Coroutine differentiation requires extra support, so it should be diagnosed
for now.


Differentiation transform:

  • Generalize differentiation utilities from ApplyInst to FullApplySite.
    • bool isArrayLiteralIntrinsic(FullApplySite)
    • void forEachApplyDirectResult(FullApplySite, llvm::function_ref<...>)
      • apply: direct result is the apply result, unless the apply result
        has tuple-type and a single destructure_tuple user. Then, direct results
        are the destructure_tuple results.
      • begin_apply: direct results are the begin_apply results.
      • try_apply: direct results are successor blocks' arguments.
    • bool LinearMapInfo::shouldDifferentiateApplySite(FullApplySite)
  • Generalize activity analysis from ApplyInst to FullApplySite.
    • Propagate variedness for FullApplySite through direct/indirect results and
      inout arguments.
    • Propagate usefulness for FullApplySite through arguments.
  • Diagnose begin_apply instructions with active arguments and results in
    PullbackEmitter::visitBeginApplyInst.

Sema:

  • Diagnose @differentiable attribute on read and modify coroutines.

Resolves TF-1081.

TF-1080 tracks coroutine differentiation support.
TF-1083 tracks throwing function differentiation support.


Example:

@differentiable
func foo(array: [Float], x: Float) -> Float {
  var array = array
  // `read` and `modify` accessors are coroutines.
  // Array subscript assignment below calls `Array.subscript.modify`.
  array[0] = x * x
  return array[0]
}
print(valueWithGradient(at: [3, 4], 5, in: foo))

// Before: silently incorrect results.
// (value: 25.0, gradient: ([1.0, 0.0], 0.0)).

// After: diagnosed.
// coroutine.swift:1:2: error: function is not differentiable
// @differentiable
// ~^~~~~~~~~~~~~~
// coroutine.swift:2:6: note: when differentiating this function definition
// func foo(array: [Float], x: Float) -> Float {
//   ^
// coroutine.swift:5:12: note: differentiation of coroutine calls is not yet supported
//   array[0] = x * x
//     ^

Coroutines are functions that can yield values and suspend/resume execution.
SIL has dedicated coroutine types. Coroutines are applied via `begin_apply`.
The `read` and `modify` accessors are coroutines.

Coroutine differentiation requires extra support, so it should be diagnosed
for now.

Differentiation transform:
- Generalize differentiation utilities from `ApplyInst` to `FullApplySite`.
  - `bool isArrayLiteralIntrinsic(FullApplySite)`
  - `void forEachApplyDirectResult(FullApplySite, llvm::function_ref<...>)`
    - `apply`: direct result is the `apply` result, unless the `apply` result
      has tuple-type and a single `destructure_tuple` user. Then, direct results
      are the `destructure_tuple` results.
    - `begin_apply`: direct results are the `begin_apply` results.
    - `try_apply`: direct results are successor blocks' arguments.
  - `bool LinearMapInfo::shouldDifferentiateApplySite(FullApplySite)`
- Generalize activity analysis from `ApplyInst` to `FullApplySite`.
  - Propagate variedness for `FullApplySite` through direct/indirect results and
    `inout` arguments.
  - Propagate usefulness for `FullApplySite` through arguments.
- Diagnose `begin_apply` instructions with active arguments and results in
  `PullbackEmitter::visitBeginApplyInst`.

Sema:
- Diagnose `@differentiable` attribute on `read` and `modify` coroutines.

Resolves TF-1081.

TF-1080 tracks coroutine differentiation support.
TF-1083 tracks throwing function differentiation support.
@dan-zheng dan-zheng added the tensorflow This is for "tensorflow" branch PRs. label Dec 21, 2019
@dan-zheng
Copy link
Collaborator Author

@swift-ci Please test tensorflow

@dan-zheng
Copy link
Collaborator Author

Merging now. Happy to address review feedback later!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tensorflow This is for "tensorflow" branch PRs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant