[RELAY]Add rsqrt, exp(a,x) and log(b,x) operators

[oovm]

I request to add tvm.relay.op.tensor.rsqrt

(and tvm.relay.op.tensor.exponent, tvm.relay.op.tensor.logarithm)

rsqrt(x) = 1/sqrt(x), but only on math.

considering numerical errors, it is a completely different situation

consider quantification the problem may be more serious

I don't know whether power has considered this

if so, just add rsqrt in tvm.relay.frontend

[ajtulloch]

It might be worth considering following Halide's distinction between fast_inv_sqrt (which maps to the approximate inverse sqrt functions available in vectorized formats on most ISAs) and inv_sqrt (the more numerically precise but expensive variant).

[oovm]

Well, there may be some ambiguity in my description.

I am not referring to the trade-off between high speed and high precision.

My main point is that Relay/NNVM needs to keep as much semantics as possible as the highest level of TVM.

In addition to rsqrt, there are also no exp(a,x) and log(b,x) operators.

When log2(x) expand as log(x) / log(2) also lose precision and semantics

---

Translate From:

我的本意不是高速和高精度之间的取舍

我在从 mxnet 转化的时候发现了 rsqrt 的缺失, 我已经打了个补丁, tf 也有人补了, 但是 caffe, onnx 还没有.

我认为从其他框架转化的时候, Relay 应该最大程度的保留各种语义, 只加一个 rsqrt 也很丑, rcbrt 呢还有...

除了幂函数 x^-a 是这种情况, 广义指数 exp(a,x) 和广义对数 log(b,x) 也没有这样的 OP.

我暂时使用了换底公式补了一下, 量化的话损失也不小, log2,log10 很多模型也要用.

[jroesch]

exp should be available as power in TVM, we are missing log and rsqrt though. Someone is welcome to add them, there is a tutorial on adding new operators for anyone who is interested https://docs.tvm.ai/dev/relay_add_op.html.

[yongwww]

log, exp and rsqrt are in relay now, I will send a pr to add them all in tf frontend