Add Secret dialect #78
Conversation
j2kun
force-pushed
the
secret
branch
6 times, most recently
from
07800c9
to
18f59ba
Compare
|
Note to self: check out llvm-project/mlir/lib/Dialect/Linalg/Transforms/Detensorize.cpp as a guiding pass, and use the DialectConversion framework to make the secret type illegal. |
j2kun
force-pushed
the
secret
branch
2 times, most recently
from
bc33106
to
1fe91b4
Compare
j2kun
force-pushed
the
secret
branch
5 times, most recently
from
48eddb9
to
a1b2d4f
Compare
|
There are a handful of TODOs still littered throughout the PR, for improved documentation and my confusion about the dialect conversion framework. But I think now is a good time for an initial review by @asraa and @AlexanderViand-Intel |
j2kun
force-pushed
the
secret
branch
2 times, most recently
from
e6c1876
to
a2d7beb
Compare
j2kun
force-pushed
the
secret
branch
2 times, most recently
from
e220400
to
cb8fa6a
Compare
|
I think there is some benefit to having this checked in even if we think doing it in full generality is too ambitious. It will be useful for some experimentation and @asraa and I are interested in trying it out as an entry point for some ML-specific dialects. |
This dialect adds secret.secret<T>, a generic wrapper for secret types, along with three basic ops, secret.conceal/reveal for generic encrypt/decrypt ops (and dialect conversion materialization), as well as a secret.generic op that lifts a plaintext computation to operate on secrets, as an interface layer to the rest of the MLIR ecosystem. Found a workaround for upstream issue https://reviews.llvm.org/D158107 The workaround is that using ConversionPatternRerwiter::replaceOp does not exercise the same stack trace, whereas combining replaceAllUsesWith + eraseOp does.
|
|
||
| include "mlir/Pass/PassBase.td" | ||
|
|
||
| def SecretForgetSecrets : Pass<"secret-forget-secrets"> { |
There was a problem hiding this comment.
"Forgetting" secrets to me sounds like something that removes secret values, rather than the secret annotation. How about naming this "drop-secret-status" or something along those lines?
There was a problem hiding this comment.
I was thinking of the "forgetful functor" but maybe that is too big of a stretch.
I like the slightly more specific "drop-secret-to-cleartext" or "interpret-secret-as-cleartext", will update it on Monday.
|
Something got messed up with the internal integration of this PR, so I'm going to cut an internal patch which will show up as a PR with some weird changes in it (including the changes for Alex's two comments) |
…namespaces were deleted by the cleanups, which was marked as irreversible. I just removed them for simplicity. Also add two changes in response to @AlexanderViand-Intel's comments on #78 FUTURE_COPYBARA_INTEGRATE_REVIEW=#78 from j2kun:secret cb8fa6a PiperOrigin-RevId: 558818663
…namespaces were deleted by the cleanups, which was marked as irreversible. I just removed them for simplicity. Also add two changes in response to @AlexanderViand-Intel's comments on #78 FUTURE_COPYBARA_INTEGRATE_REVIEW=#78 from j2kun:secret cb8fa6a PiperOrigin-RevId: 558818663
|
Manually merged as 69db9f0 |
This PR implements a
secretdialect to represent generic, high level computation on encrypted data, along with a demonstration lowering pass that drops the secret types.It defines a single new type,
secret.secret<AnyType>which wraps a non-secret type, and centers around thesecret.genericop, which "lifts" a computation on the plaintext types to a computation on the corresponding secret types, with some metadata attached to the high level operation encapsulated by asecret.genericso that it can be lowered to custom ops in specific dialects.I also defined a pass,
--secret-forget-secrets, which simply drops the secrets from all types and returns an equivalent program that operates on plaintext data.Some notes:
secret.generic's body containing a single op, but I think more realistically we will have front ends that convert large regions of code into a largesecret.generic, and then passes in this dialect that transform it by doing things like extracting loops with static bounds outside of the body of asecret.genericand only having the inner-most loop body containsecret.genericops (which could then be further analyzed/optimized with partial loop unrolling and fusing), or something like transforming the body of asecret.genericinto a combinational circuit, then applying a circuit optimizer to it.secret.generic.