[Offload] Offloading API always initializes all plugins and devices

[jhuber6]

The offloading API currently loads all available plugins when we initialize. The overhead of initializing a plugin that isn't needed is quite large, and existing runtimes like OpenMP intentionally avoid this by checking if any images require a plugin before loading them. It would be nice if something similar could be implemented through the Offloading API.

Right now the API exposes the platform as an implementation detail of the device. I'm wondering if there's an easy solution to this. Either we change the initialization code to accept a callback that states if a platform should be used, or we perform this lazily until someone actually needs it.

[llvmbot]

@llvm/issue-subscribers-offload

Author: Joseph Huber (jhuber6)

The offloading API currently loads all available plugins when we initialize. The overhead of initializing a plugin that isn't needed is quite large, and existing runtimes like OpenMP intentionally avoid this by checking if any images require a plugin before loading them. It would be nice if something similar could be implemented through the Offloading API.

Right now the API exposes the platform as an implementation detail of the device. I'm wondering if there's an easy solution to this. Either we change the initialization code to accept a callback that states if a platform should be used, or we perform this lazily until someone actually needs it.

[RossBrunton]

I think there's three main ways for a user to "discover" devices:

1. By iterating over all devices and inspecting them to determine the "best" device using whatever metrics they want.
2. By explicitly specifying the platform via an environment variable or config file.
3. By having a binary or set of binaries, and only selecting devices that can run the binary.

Number 1 I think has to initialize all the plugins, since it needs to look at them all. Number 3 probably also needs to do that, since I don't think it's trivial for liboffload to check whether a platform can consume a binary without access to devices (e.g. AMD needs to know the gfx architecture thingie).

Which leaves number 2. I think the thing that makes sense is for olInit or olIterateDevices to accept a list of plugins. Any plugin not on the list isn't initialized and simply ignored.

[jhuber6]

I think there's three main ways for a user to "discover" devices:

1. By iterating over all devices and inspecting them to determine the "best" device using whatever metrics they want.

2. By explicitly specifying the platform via an environment variable or config file.

3. By having a binary or set of binaries, and only selecting devices that can run the binary.

Number 1 I think has to initialize all the plugins, since it needs to look at them all. Number 3 probably also needs to do that, since I don't think it's trivial for liboffload to check whether a platform can consume a binary without access to devices (e.g. AMD needs to know the gfx architecture thingie).

Which leaves number 2. I think the thing that makes sense is for olInit or olIterateDevices to accept a list of plugins. Any plugin not on the list isn't initialized and simply ignored.

Specifying it on iterating devices sounds like the most straightforward. The problem is that we eagerly initialize devices in olInit. So, I think what we'd need to do is delay plugin initialization until olIterateDevices and then we probably want to delay device initialization until someone actually tries to load an image on it maybe. I'd like to avoid changing the olInit if possible.