Switch to Ubuntu 16.04 based build

[chris-allan]

During work on adding TileDB support to bioformats2raw in glencoesoftware/bioformats2raw#57 glibc and libstdc++ missing symbols were noticed in the native code. This was on CentOS 7 which is both our most common deployment platform and the most common Linux distribution in use on the HPC platforms we interact with.

Building on Ubuntu 16.04 should mean that CentOS 7 is supported. This does not go as far as TileDB-Py which currently targets CentOS 6 via the use of the manylinux2010 [1] for its binary wheels and by extension supports back to CentOS 6.

1. https://github.com/TileDB-Inc/TileDB-Py/blob/dev/HISTORY.md#tiledb-py-066-release-notes

[Shelnutt2]

@chris-allan do you have a minimum architecture you target? Previously we used to set the c/cxxflags for march to haswell as a minimum architecture for AVX1 support but old enough to generally cover any users with the prebuilt binary.

[chris-allan]

For our use Haswell (~2H2013) or later architecture is definitely acceptable. As far as GCC is concerned that march should ensure AVX2 support as well.

[Shelnutt2]

For our use Haswell (~2H2013) or later architecture is definitely acceptable. As far as GCC is concerned that march should ensure AVX2 support as well.

Thanks for the information. We'll send another PR for setting the march version and then we'll cut a new release for these changes.

[chris-allan]

Spoke too soon. Ubuntu 16.04 doesn't look sufficient.

I'm not sure exactly why this didn't come up earlier but I may have been glossing over the C++ ABI. With the current build, even on Ubuntu 16.04, CXXABI_1.3.8 seems to currently be used. CentOS 7 is on GCC 4.8 with CXXABI_1.3.7 [1] I will need to do a little more sleuthing.

$ uname -an
Linux us-demo.glencoesoftware.com 3.10.0-1127.13.1.el7.x86_64 #1 SMP Tue Jun 23 15:46:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
$ cat /etc/redhat-release
CentOS Linux release 7.8.2003 (Core)
$ unzip tiledb-java-0.3.2-SNAPSHOT.jar lib/libtiledbjni.so
Archive:  tiledb-java-0.3.2-SNAPSHOT.jar
  inflating: lib/libtiledbjni.so
$ ldd lib/libtiledbjni.so
lib/libtiledbjni.so: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by lib/libtiledbjni.so)
        linux-vdso.so.1 =>  (0x00007fffbdf9d000)
        libjvm.so => not found
        libtiledb.so.2.0 => not found
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f0cdcd7a000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f0cdc9ac000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f0cdc6aa000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f0cdd2b6000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f0cdc494000)

1. https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html

[chris-allan]

After a little more investigation, definitely completely missed the C++ ABI scenario somehow. Ubuntu 14.04 is out with no images to my knowledge available on Azure Pipelines. Looks like we might have to do something similar to TileDB-Py after all:

• https://github.com/TileDB-Inc/TileDB-Py/blob/dev/misc/pypi_linux/Dockerfile2010

[Shelnutt2]

@chris-allan okay we'll get the linux binary built inside the manylinux2010 image, please give us a day or so and we'll try to get the build updated. Once we have a snapshot jar to test we'll ping you for validation in your environment.

[chris-allan]

@Shelnutt2: Don't want to duplicate work but I'm happy to scratch my own itch and work on it.

[Shelnutt2]

@chris-allan if you are interested to make the build change for the prebuilt jar we are always welcome of a PR. Please go ahead with making the change for Azure CI. If you run into problems or decide you'd rather not just let us know. We appreciate your contribution!

[chris-allan]

👍

Currently working on it. Currently running in to issues with the cURL/OpenSSL linking when building using the quay.io/pypa/manylinux2010_x86_64:latest container. If I can't sort that out today I'll at least push what I have for review.

[Shelnutt2]

@chris-allan feel free to push up what you have and highlight the errors/problems. @ihnorton and I are very familiar with curl build and link problems. Curl is the bane of all our dependencies 😞 .

[chris-allan]

Thanks, @Shelnutt2. Think I managed to work around the problem. The summary is on glencoesoftware/TileDB-Java@3ecd343 which is part of #194. Happy to discuss more. To do this "properly" I think changes to the TileDB build itself are required.