[VL] prefer to use static packaging

[zhouyuan]

Description

The dynamic packaging will make a jar with all necessary dependencies for shared libs. On gluten start it will try to extract the libs and then load into JVM.
This method is introduced in early stage of Gluten. The production env is quite different: Cent7, Cent8, Ubuntu 20, Debian, ArchLinux and some customized Linux releases. For different OS the dependencies are different, especially on the library path and the library versions. The code is quite difficult to maintain as Velox is bumping the dependencies.

For better packaging releasing/maintaining, gluten may should pick the static packaging(vcpkg based) so all dependencies are all statically linked and can work on all OS.

Edit:
We need to keep the dynamic packaging method as it's already used in some production env.

CC: @weiting-chen

Thanks,
-yuan

[FelixYBW]

We may mention this in dev maillist.

static build makes the deployment much easier. The only issue may be the jar size but now even with debug info, it's just ~88MB.

For developer with current build script, all static library will only be built once. Or build again after vcpkg script changed. velox will be built when commit id changed. So not an issue to development either. It may increase the link time somehow.

See if anyone have concerns about this.

[boneanxs]

We may mention this in dev maillist.

Hey @zhouyuan @FelixYBW Since we're in Apache incubator community now, if we decide to use dev maillist, better update

incubator-gluten/README.md

Line 113 in 62746c4

## 4.1 Community

to reflect email address whereas others can join it.

[zhouyuan]

We may mention this in dev maillist.

Hey @zhouyuan @FelixYBW Since we're in Apache incubator community now, if we decide to use dev maillist, better update

incubator-gluten/README.md

Line 113 in 62746c4

## 4.1 Community

to reflect email address whereas others can join it.

Hi Rex, yes, will add the maillist to the doc, thanks!

Thanks,
-yuan

[PHILO-HE]

FYI.
Just tried using static link on centos-8, then running test on centos-7. An issue was reported at runtime:

java.lang.UnsatisfiedLinkError: 
version `GLIBC_2.27' not found (required by /tmp/gluten-de464be8-5c4e-45eb-8fda-8749954938c4/jni/6730a5e2-3e06-4112-b509-6941ceb4126f/gluten-6403925142698728788/libarrow.so.1400.1.0)

[zhouyuan]

#5013

We may mention this in dev maillist.

Hey @zhouyuan @FelixYBW Since we're in Apache incubator community now, if we decide to use dev maillist, better update

incubator-gluten/README.md

Line 113 in 62746c4

## 4.1 Community

to reflect email address whereas others can join it.

Hi Rex, yes, will add the maillist to the doc, thanks!

Thanks, -yuan

[ulysses-you]

I have an another issue about libarrow.so. It seems similar with @PHILO-HE

[zhouyuan]

I have an another issue about libarrow.so. It seems similar with @PHILO-HE

@ulysses-you the issue should be fixed by using static building libarrow.so/libparquet.so, will try to fix this in
#5033

Thanks,
-yuan

[ulysses-you]

thank you @zhouyuan !

[Yohahaha]

We can provide options to choose building static or dynamic artifacts, and enable static by default. Just not force ban dynamic packaging directly, thank you.