[regression in 0.20.0] error: relocation R_X86_64_32S cannot be used against local symbol; recompile with -fPIC

[yurivict]

ld: error: relocation R_X86_64_32S cannot be used against local symbol; recompile with -fPIC
>>> defined in htslib/bcf_sr_sort.o
>>> referenced by bcf_sr_sort.c
>>>               htslib/bcf_sr_sort.o:(bcf_sr_sort_next)

ld: error: relocation R_X86_64_32S cannot be used against local symbol; recompile with -fPIC
>>> defined in htslib/bcf_sr_sort.o
>>> referenced by bcf_sr_sort.c
>>>               htslib/bcf_sr_sort.o:(bcf_sr_sort_next)

clang-14
Python-3.9
OS: FreeBSD 13.1

[jmarshall]

Are you setting $CFLAGS or other similar environment variables?

[yurivict]

Yes: CFLAGS=-O2 -pipe -fno-omit-frame-pointer -fstack-protector-strong -fno-strict-aliasing

But this shouldn't affect the issue.

[jmarshall]

You have encountered the same problem as the Bioconda pysam package build did in bioconda/bioconda-recipes#37686.

As discussed in #1112 and PR #1140, when using the bundled htslib, pysam 0.20.0 uses HTSlib's Makefile to build libhts.a rather than listing htslib/*.c as extension module source files and having them compiled via setuptools's infrastructure.

Prior to this, $CFLAGS/etc were used only to provide information for hts_feature_string() and for use during configure's test compilations. But with this change, the provided $CFLAGS/etc are used to compile htslib/*.c — and the compiler options provided by setuptools are no longer used for these source files.

I realised rather late in the 0.20.0 development cycle (see e23c7fe) that in general it is necessary to add sysconfig.get_config_var('CCSHARED') to $CFLAGS to get a suitable PIC option into the mix. (On some platforms that need it, it is misleadingly already present in sysconfig.get_config_var('CFLAGS'), but in general it is not.)

That commit explicitly adds $CCSHARED to $CFLAGS when the latter comes from sysconfig. It was probably an oversight not to consider whether similar treatment would be necessary when the latter comes from an externally-supplied environment variable.

This problem will go away in a future pysam version when running configure uses settings as set up by build_config_dict instead of set_compiler_envvars. In the meantime, a patch something like the following may be appropriate, to hack the value in a similar way as the code a few lines later does for the sysconfig case:

--- a/setup.py
+++ b/setup.py
@@ -170,6 +170,8 @@ def set_compiler_envvars():
     tmp_vars = []
     for var in ['CC', 'CFLAGS', 'LDFLAGS']:
         if var in os.environ:
+            if var == 'CFLAGS' and 'CCSHARED' in sysconfig.get_config_vars():
+                os.environ[var] += ' ' + sysconfig.get_config_var('CCSHARED')
             print("# pysam: (env) {}={}".format(var, os.environ[var]))
         elif var in sysconfig.get_config_vars():
             value = sysconfig.get_config_var(var)

Alternatively you should simply set the option yourself: CFLAGS=-O2 -pipe -fPIC -fno-omit-frame-pointer -fstack-protector-strong -fno-strict-aliasing