POC: Use dispatch to Python to implement Crypten autograd

[ezyang]

This proof of concept is sufficient to run "Tutorial 7: Training an
Encrypted Neural Network" while directly reusing PyTorch's autograd
engine.

The basic structure:

• CrypTensor is now a subclass of Tensor. It is a very simple
  wrapper class around InnerCrypTensor, which is what was previously
  known as CrypTensor. MPCTensor inherits from InnerCrypTensor.

• The torch_dispatch on CrypTensor simply unwraps CrypTensor into
  InnerCrypTensor and invokes the true computation on the inner tensor,
  wrapping it back into a CrypTensor in the end. There's also a little
  bit of extra faffing about:

  • Sometimes torch_dispatch comes back with an unexpected
    function name and I massage it back into the name that
    InnerCrypTensor expects

  • Anomaly mode tests if the inputs are NaNs and I shut it up by
    just making everything not NaN

  • InnerCrypTensor has a number of extra methods on top of Tensor, so
    I also implement getattr to forward those along

• The torch_function on CrypTensor shows how we can override
  autograd behavior when PyTorch's default formula is inappropriate for
  CryTen. In particular, in PyTorch ReLU is implemented in terms of
  threshold/threshold_backward, which CrypTen doesn't support. So I
  override the behavior to do a more direct implementation with masking.

• All of the autograd pieces are redirected to point at normal PyTorch
  autograd

• Anywhere in CrypTen where a test "is Tensor" was done, now must be a
  test "is not CrypTensor" (because previously CrypTensor was not a
  tensor, and now it is.)

One major downside of this implementation is that CrypTensor holds
an InnerCrypTensor; it would be better if these were collapsed.
However, in CrypTen's current implementation, this is not so easy to do
because internal implementations frequently allocate empty CrypTensors
and then fill them in later and I could not easily figure out how to
make this work well with Tensor (Greg thinks it should work, I just
gave up).

Signed-off-by: Edward Z. Yang ezyang@fb.com

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Docs change / refactoring / dependency upgrade

Motivation and Context / Related issue

How Has This Been Tested (if it applies)

Checklist

• The documentation is up-to-date with the changes I made.
• I have read the CONTRIBUTING document and completed the CLA (see CONTRIBUTING).
• All tests passed, and additional code has been covered with new tests.

[ezyang]

For the record: @albanD gave a nice presentation about the various pieces in this prototype