[LLVM] Support CodeGenBlob for large >2GB models

[apivovarov]

If large const data (>2GB) is saved to default .rodata section then linking it to shared library will fail - relocation truncated to fit: R_X86_64_PC32.
The issue exists on Linux x86_64 platform.
GCC puts large const data to .lrodata segment if -mcmodel=medium parameter is used, but LLVM ignores it.
The workaround is to explicitly put large const data to .lrodata section.
Lets put const data which is larger than 1GB to .lrodata section.

Model compilation and execution was tested with large MXNet model - it was compiled using TVM-TensorRT:
gen-large-mxnet-model-py

compile-mxnet-tvm-trt-py

Serialized TensorRT TVM Module was saved to model.so file. (size is greater than 2GB).

2,361,256,184  model.so

[junrushao]

Thanks @apivovarov for this PR. LGTM!

[FrozenGene]

Could we use setCodeModel to solve this issue ? @apivovarov

[apivovarov]

GCC has the following x86-Options related to 2GB limit issue on x86_64 Linux platform

-mlarge-data-threshold=threshold
When -mcmodel=medium is specified, data objects larger than threshold are placed in the large data section. This value must be the same across all objects linked into the binary, and defaults to 65535.

-mcmodel=medium
Generate code for the medium model: the program is linked in the lower 2 GB of the address space. Small symbols are also placed there. Symbols with sizes larger than -mlarge-data-threshold are put into large data or BSS sections and can be located above 2GB. Programs can be statically or dynamically linked.

I checked the difference in object (.o) and asm (.s) files generated by gcc. The diff is in the section name for Read-Only Data. mcmodel small uses .rodata and mcmodel medium uses .lrodata for const pre-defined data large than 64K.

LLVM/clang also has -mcmodel param but it does not have -mlarge-data-threshold param.
I tried to use -mcmodel=medium parameter with llvm/clang on x86_64 platform - the generated object, asm and llvm files are the same for -mcmodel=small and -mcmodel=medium. This is why linking object file .o to .so fails with error relocation truncated to fit: R_X86_64_PC32

LLVM/clang mostly use mcmodel parameter on RISC-V and XCore architectures.
mcmodel parameter is ignored on X86 platform.

The workaround for clang and LLVM x86 is to explicitly put section name for the large const pre-defined arrays. For example:

C/C++:

__attribute__((section(".lrodata")))
extern const long long int B100[131072];
const long long int B100[131072] = {...};

LLVM API:

auto* B100 = new llvm::GlobalVariable(...)
B100->setSection(".lrodata");

[wanglei91]

@apivovarov Thanks, I use this patch to test SD unet, it indeed support large >2GB models(mold instead of ld linker could also support it), but both this patch and mold linker still truncate when more than 3.1G(for may case, this patch produces a 3.1G unet.so, save more tensors than mold), I don't know if somebody else also meet this issue...

print when save:
const tensor 605
header: 15951258332257624383
cpu_dev.device_type: 1
tensor->ndim: 3
shape0: 16
shape1: 4096
shape2: 77

const tensor 606
header: 15951258332257624383
cpu_dev.device_type: 1
tensor->ndim: 3
shape0: 640
shape1: 1
shape2: 1

print when load:
const tensor 605
header: 15951258332257624383
shape0: 16
shape1: 4096
shape2: 77

const tensor 606
header: 160445370461692650
include/tvm/runtime/ndarray.h", line 512
TVMError: Check failed: (header == kTVMNDArrayMagic) is false: Invalid DLTensor file format

[tqchen]

@wanglei91 we recommend using lift transform params for large parameters, as in https://github.com/mlc-ai/web-stable-diffusion.

cc @vinx13 @psrivas2 , this is also useful information

[wanglei91]

@tqchen Got it, much appreciated for the reply!