(jitify2) Serialisation post-NVRTC

[Robadob]

Within jitify2, the latest point of implemented serialisation appears to be that of PreprocessedProgram. Hence, after deserialization it requires compilation via NVRTC. In contrast jitify1 serialises the ptx blob output by NVRTC.

The jitify2 approach is presumably more portable, however in our use-case we are serialising to memory and/or /tmp/, so there's no requirement of portability. Furthermore, with our large compilation units, the serialised output is ~2x bigger and the time to load them and ~50x slower.

A quick test with CUDA 12.3

	Serialised size (KB)	Serialised load time (ms)
jitify1	1200-1800	12-31
jitify2	3200-3300	685-1372

Is there a reason post-NVRTC serialisation is no-longer present/am I mistaken?
Is it on your roadmap/would you be happy with me submitting a PR?

[benbarsdell]

Program, PreprocessedProgram, CompiledProgram, and LinkedProgram all support serialization: https://github.com/NVIDIA/jitify/blob/fa694ec/jitify2_test.cu#L113-L126

ProgramCache will cache the LinkedPrograms (cubins) in memory, and optionally on disk as noted in the user guide.

[Robadob]

Thanks, I'll try that today.

Looking at our current code it appears we go from PreprocessedProgram to LoadedProgram to Kernel (the latter which does not have serialise/deserialise implemented). That's probably how I missed this.

[Robadob]

As noted above, we were calling PreProcessedProgram::load() which returns LoadedProgram, bypassing CompiledProgram and LinkedProgram. I've now split up that call into it's constituent parts and serializing LinkedProgram is working much the same as Jitify1.

Thanks