[AD] Serialize/type-check differentiable attribute trailing where clauses.

[dan-zheng]

• Store where clause Requirements in DifferentiableAttr and SILDifferentiableAttr.
• Implement serialization/deserialization logic for where clause requirements.
• Implement type-checking for where clauses.
  • The primal/adjoint/JVP/VJP generic signature is expected to be the union of the
    original function generic signature and the where clause generic signature.

Where clauses enable the definition of autodiff associated functions that are more
constrained than the original function. This is useful for <Scalar : Numeric>
operations, for example, which are differentiable only where Scalar : FloatingPoint.

TODOs:

• Fix where clauses with member type constraints e.g. <Self == Self.CotangentVector>,
  which type-check but crash at serialization.
• Add where clauses to @differentiable attributes in stdlib. (I've started this.)

[rxwei]

Looks good but I didn’t look super carefully. The general direction makes sense. Let’s merge and progress towards JVPs-in-stdlib.

[dan-zheng]

@swift-ci Please test tensorflow

[dan-zheng]

@swift-ci Please test tensorflow

[jekbradbury]

I haven't tracked down the root causes here, but these are some programs that crash the compiler (my guess is I'm doing something basic incorrectly; if you can't reproduce this it could also be an artifact of other patches I'm carrying):

@differentiable(adjoint: adjointFoo
                where T: Differentiable, T == T.CotangentVector)
func foo<T: Numeric>(_ x: T) -> T {
  return x
}

func adjointFoo<T: Differentiable & Numeric>(
    _ seed: T, _ originalValue: T, _ dy: T
) -> T where T == T.CotangentVector {
  return dy
}

let x = 1.0
let pb = pullback(at: x, in: foo)
print(pb(1.0))

crashes in mandatory inlining:

Cannot construct Inlined loc from the given location.
UNREACHABLE executed at /usr/local/google/home/jekbradbury/swift-sources/swift/lib/SIL/SILLocation.cpp:220!
Stack dump:
0.      [...]
1.      While running pass #60 SILModuleTransform "MandatoryInlining".

and

@differentiable(adjoint: adjointFoo
                where T: Differentiable, T == T.CotangentVector)
func foo<T: Numeric>(_ x: Tensor<T>) -> Tensor<T> {
  return x
}

func adjointFoo<T: Differentiable & Numeric>(
    _ seed: Tensor<T>, _ originalValue: Tensor<T>, _ dy: Tensor<T>
) -> Tensor<T> where T == T.CotangentVector {
  return dy
}

let x = Tensor<Float>(1.0)
let pb = pullback(at: x, in: foo)
print(pb(Tensor<Float>(1.0)))

crashes in IRGen:

swift: /usr/local/google/home/jekbradbury/swift-sources/llvm/include/llvm/ADT/Optional.h:176: T *llvm::Optional<swift::ProtocolConformanceRef>::getPointer() [T = swift::ProtocolConformanceRef]: Assertion `Storage.hasVal' failed.
Stack dump:
0.      [...]
1.      While emitting IR SIL function "@main".

[rxwei]

@jekbradbury does vjp: work?