Add a unit test for softmax

[prigoyal]

we have a weird bug where we don't allow reusing the same variable twice

FAILS:
def softmax(float(N, D) I) -> (O, tmp) {
        tmp(n) max=! I(n, d)
        O(n, d) = exp(I(n, d) - tmp(n))
        tmp(n) +=! O(n, d)
        O(n, d) = O(n, d) / tmp(n)
}
ERROR: Func tmp cannot be given a new update definition, because it has already been realized or used in the definition of another Func. Aborted (core dump)

FAILS:
def softmax(float(N, D) I) -> (O, tmp, tmp1) {
        tmp(n) max=! I(n, d)
        O(n, d) = exp(I(n, d) - tmp(n))
        tmp1(n) +=! O(n, d)
        O(n, d) = O(n, d) / tmp1(n)
}
ERROR: Func O cannot be given a new update definition, because it has already been realized or used in the definition of another Func

PASSES:
def softmax(float(N, D) I) -> (O, expsum, maxVal) {
        maxVal(n) max= I(n, d)
        expsum(n) +=! exp(I(n, d) - maxVal(n))
        O(n, d) = exp(I(n, d) - maxVal(n)) / expsum(n)
 }

It throws error:

Func tmp cannot be given a new update definition, because it has already been realized or used in the definition of another Func. Aborted (core dump)

cc @jekbradbury

[abadams]

It's not necessary to go down to three lines of code, you just can't reuse names on the LHS once they've been used in another tensor. Otherwise the sizes of the tensors is ill-defined (in the general case - there's a solution in this case).

def softmax(float(N, D) I) -> (O, maxVal, expDistance, expSum) {
        maxVal(n) max=! I(n, d)
        expDistance(n, d) = exp(I(n, d) - maxVal(n))
        expSum(n) +=! expDistance(n, d)
        O(n, d) = expDistance(n, d) / expSum(n)
}

[prigoyal]

yep, thanks for the clarification @abadams that makes sense, however, I am still wondering about the max=! issue and why that fails compilation

@zdevito can you take a look at that case, there's issue #63 filed as well

[prigoyal]

CI is green, merging, simple test cases added

[nicolasvasilache]

@prigoyal lots of dead code in this PR, please:

1. enable and expect proper failure
2. create issue (if not already existing)
3. tag test with TODO(#numissue)

Thanks!