Inductor indexing kernels don't check for valid indices

[ngimel]

When eager mode gets invalid indices for indexing operations (e.g. negative or larger than input size) it generates kernel assert and hard errors with invalidated context. Inductor isn't set up to error, it can produce silent wrong results or error out with illegal memory access, depending on the caching allocator state and indices values.
It's unclear if triton can have internal asserts in its kernels, but we should still think what our behavior should be in these cases, see pytorch/torchdynamo#1515 (comment)

cc @ezyang @soumith @msaroufim @wconstab @bdhirsh

[soumith]

i think this is more than bad UX, this is a fundamental correctness issue.

I'd suggest that we do the following if Triton doesn't support device asserts:

1. write to a global memory flag on invalid indices
2. after the fused kernel finished running, run a regular tiny CUDA kernel that all it does is check that global memory flag and do a device assert

Alternatively, take a performance hit and run a "indices validation" kernel.

Bonus, do (2), have the device assert trigger a proper runtime error with the correct operation that failed. This is probably hard.

[jansel]

@ptillet would it be possible/easy to add support for device asserts to Triton?

[ptillet]

It's on the roadmap, but we haven't started looking into yet. I don't really know what the PTX/LLVM should look like for device-side asserts in CUDA, so it may or may not be difficult :)

[ngimel]

i think this is more than bad UX, this is a fundamental correctness issue.

It is, but tf/xla/jax always just wrote/used bogus values in these cases. I agree we should mimic eager behavior though.

[ngimel]

PR on the triton side is landed triton-lang/triton#1143, we need updating triton pin to land to be able to use it #95896