standard practice for setting "cpu", "cuda", "mps", "gpu" for torch-based tutorials notebooks and functions/classes

[bobleesj]

Problem

I've been reviewing numpy to torch code including @henrygbell's MAPED #204 and @ehrhardtkm PDF #177 and my recent drift correction #208

At the moment, it appears device selection is a bit scattered:

1. config.set_device()
2. validate_device
3. quantem.yaml where cpu is set
4. config.get_device() and manually entered within a function/class signature
5. ...

What is our best practice? A few constraints for users:

1. device selection should be automated in general
2. memory usage should be chunked, avoid OOM error as much as we can
3. ...

Proposed solution

I don't have one yet. I will write some thoughts as I encounter more. Please feel free to suggest any alternatives.

[arthurmccray]

notes from discussion :)

• If using a single GPU, I prefer to do config.set_device(XX) up top, and then you can use "cuda" as the device everywhere for torch things, as this will put it on the appropriate GPU. For stuff that we write that's user-facing we will also support "gpu" which is equiv to "cuda"
• in terms of memory usage and chunking, agreed that we want that to be optimized, and that's the point of config. The idea would be one would set the chunk_size in the local config (and write it to disk) and then everything afterwards would use that. The main catch/thing is that in whatever functions we write, we would need to look to config as the default (which currently doesn't happen)