GPU User Experience Improvements

[tbennun]

• Post-SDFG error checking and informative guidance (Fixes GPU debugging guidance #1277)
• More information in syncdebug prints (e.g., actual grid/block sizes)
• Tell users where invalid/failing SDFGs were saved
• Always error-check (without synchronization) unless configured otherwise
• Validation error upon reading from a non-accessible host memory (e.g., GPU global) outside an accelerator context
• __launch_bounds__ support in map nodes: -1 disables, 0 (default) sets if block size is constant, any other number sets explicitly
• Argument errors (Fixes DaCe does not warn on excess arguments to dace.program #1263)
• Validate nested SDFG parent pointers
• Warnings/errors on conflicting behavior with gpu_block_size, configuration, and internal maps (Fixes Fewer dimension GPU maps with higher-dimension GPU blocks should fallback #1271)
  • Warning when multiple internal Threadblock maps with different sizes exist + the final block size used
  • Configurable (+default) maximal block size x, total block size and error when surpassed + recommendations (tile block with another map)
  • Reduce default_block_size dimensionality to kernel dimensionality and warn if that happens for any reason
  • Error when both gpu_block_size and threadblock maps are given
  • The warning for using default block size should guide the user towards gpu_block_size or thread-block sub maps
• Fix CUDA 12+ CUB includes

[mcopik]

If I understand correctly, the current proposal is to check for errors after execution. I think it might be worth considering a case where we fail at the first CUDA error, not just check at the end - this will make debugging much easier and faster. We could make it an optional feature if there's any concern that an additional check of a return value from CUDA calls would add an unnecessary performance tax.

Happy to help with this feature by improving & merging my solution for this.

[tbennun]

We don’t want the overhead by default, that’s what syncdebug is for. It does what you proposed and more (and even more after this PR).

[tbennun]

@mcopik I followed your suggestion in the latest commit

[phschaad]

LGTM, minor comments to consider.