Repository navigation
System
Note
Section _System (group _Tools) · Namespace utils::system · Headers utils/system/
Tools for the program itself: id distribution, delayed tasks, a pool of workers and memory kept on the GPU.
IdHandler<T> (T integer) gives unique ids, starting at 1 (0 is reserved as "no id"), and reuses the freed ones first. Every method takes safe_mode = true (lock, thread safe).
| Method | Description |
|---|---|
allocate() / allocate(T& id)
|
Give a free id (and store it in id) |
free(id) |
Give back an id (DoubleFree, UnknownId for an id never given) |
use(id) |
Force an id as used (DoubleUse when already used) |
preview() |
Next id allocate would give |
id() / actual()
|
Counter / last id given |
clear() / free()
|
Free every id |
Warning
When the type limit is reached, allocate throws FatalException(IdOverflow) (ends the program), preview an ErrorException(IdOverflow).
Runs a function after a delay, each task on its own thread.
| Method | Description |
|---|---|
schedule(delay, fn) |
Run fn() after delay (std::chrono::milliseconds), returns the task id |
cancel(id) |
Cancel one task (UnknownId when finished / unknown) |
cancel() |
Cancel every pending task (also done by the destructor) |
utils::system::Scheduler scheduler;
std::size_t id = scheduler.schedule(std::chrono::milliseconds{500}, [](void) {std::cout << "later" << std::endl;});
scheduler.cancel(id);LoadBalancer<T>(limit = 1, lifespan = 0ms) keeps a pool of workers (T is a utils::type::Worker or derives it, see Type) and gives a free one.
| Method | Description |
|---|---|
spawn(n = 1) |
Add n workers (OutOfBounds over the limit, 0 = no limit) |
getWorker() |
std::future<T&> resolved with a free worker, marked working (waits until one is free) |
kill() / kill(n)
|
Remove the free workers / n workers (the free ones first) |
kill<true>() / kill<true>(n)
|
Same, also removing the working ones |
setLimit(n) / getLimit()
|
Maximum number of workers |
setLifespan(ms) / getLifespan()
|
A worker free for longer is removed (0 = never) |
size() |
Number of workers |
The worker is given back with worker.setWorkingStatus(false). A getWorker still waiting when the balancer is destroyed throws PromiseCanceled from future.get().
utils::system::LoadBalancer<utils::type::Worker> balancer(4, std::chrono::seconds{30});
balancer.spawn(2);
utils::type::Worker& worker = balancer.getWorker().get();
// ... work ...
worker.setWorkingStatus(false);HybridMemory<T> (T trivially copyable, header utils/system/HybridMemory.hpp) keeps n values of T in the memory of a GPU when one is available, else in the RAM, for future computes on it. The backend is chosen at the construction with a MemoryMode; the GPU libraries are loaded at runtime (dlopen / LoadLibrary), so nothing is needed to build or link and a machine without GPU simply uses the RAM.
| Mode | Backend |
|---|---|
Auto |
First GPU found (Cuda -> OpenCL -> Vulkan), else the RAM |
Gpu |
First GPU found, GpuUnavailable without one |
Ram |
Host memory only |
Cuda / OpenCL / Vulkan
|
Forced backend (libcuda / libOpenCL / libvulkan), GpuUnavailable when missing |
Custom |
Handler given by the program (IGpu, see below) |
| Method | Description |
|---|---|
HybridMemory(mode = Auto) / HybridMemory(size, mode = Auto)
|
Select the backend (and allocate size values) |
HybridMemory(size, std::shared_ptr<IGpu> gpu) |
Custom mode with a handler (InvalidArgument when nullptr or unavailable) |
allocate(size) / resize(size) / free()
|
New memory (uninitialized) / new size, the first values kept / release |
store(span<const T>, offset = 0) |
Copy values into the memory from the index offset (OutOfBounds) |
fetch(span<T>, offset = 0) |
Copy out.size() values from the memory (OutOfBounds) |
get(i) / set(i, value) / operator[]
|
One value (a transfer on the GPU: prefer fetch / store for many) |
fill(value) / clear()
|
Every value set to value / to 0
|
toVector() |
Whole memory copied in a std::vector<T>
|
join() |
Wait for the pending GPU operations (nothing in RAM) |
getMode() / getRequestedMode() / isOnGpu()
|
Mode really used (Ram after an Auto without GPU) / mode asked / on a GPU handler |
size() / bytes() / empty()
|
Number of T / size in bytes / nothing allocated |
native() |
Device buffer (CUdeviceptr, cl_mem...) for the computes, 0 in RAM |
data() |
Host pointer (RAM only, nullptr on a GPU) |
getGpu() |
The handler (IGpu), nullptr in RAM |
The class is move only. IGpu (utils/system/gpu/IGpu.hpp) is the interface of a backend, implemented by CudaGpu, OpenCLGpu and VulkanGpu: isAvailable, getMode, getName, getMemory, allocate, free, store, fetch, memset, memcpy, join (offsets in bytes). make_gpu(mode) (and make_raw_gpu / make_shared_gpu / make_unique_gpu) builds one by hand, nullptr for Ram.
Note
HybridMemory only stores and moves the data: the computes are written with the API of the backend on native() (kernel launch, cl_mem argument...). Every call of the Cuda handler makes its context current on the calling thread.
Warning
On a GPU every get / set is a transfer, and the Vulkan handler creates a staging buffer for each one: use store / fetch for blocks.
#define _System
#include <utils/utils.hpp>
#include <vector>
std::vector<float> values = {1.0f, 2.0f, 3.0f, 4.0f};
std::vector<float> out(values.size());
utils::system::HybridMemory<float> memory(values.size()); // Auto: GPU when available, else RAM
memory.store(values); // host -> memory
memory.set(0, 10.0f); // one value
memory.join(); // wait for the GPU
memory.fetch(out); // memory -> host: {10, 2, 3, 4}
utils::system::HybridMemory<float> vulkan(8, utils::system::MemoryMode::Vulkan); // GpuUnavailable without Vulkan| Code | Raised when |
|---|---|
GpuUnavailable |
A forced backend (or Gpu) isn't available on the machine |
GpuAllocation |
The GPU memory allocation failed |
GpuTransfer |
A transfer (store, fetch, memset, memcpy) failed |
OutOfBounds |
A range outside of the memory |
OutOfMemory |
The size overflows or the RAM allocation failed |
InvalidArgument |
Custom without a handler (or with an unavailable one) |
