Error with latest python 3.7.4

[Martoni]

I compiled the latest python on my computer (debian) with standard options :

./configure
make
sudo make install

Then I used it with virtualenv in my cocotb project :

$ cd mycocotb/
$ virtualenv --python=/usr/local/bin/python3.7 envp37
$ source envp37/bin/activate
$  python --version
Python 3.7.4
$ python -m pip install cocotb
$ make

I get the following gcc error :

gcc -Werror -Wcast-qual -Wcast-align -Wwrite-strings -Wall -Wno-unused-parameter -fno-common -g -DDEBUG -fpic -DICARUS -Wstrict-prototypes -Waggregate-return -DFILTER -DICARUS -shared -Xlinker -L/usr/local/lib -o /usr/local/projects/videautom/gateware/hw_validation/spi2ksz/cocotb/build/libs/x86_64/libgpilog.so /usr/local/projects/videautom/gateware/hw_validation/spi2ksz/cocotb/build/obj/x86_64/gpi_logging.o   -lpython3.7m 
/usr/bin/ld: /usr/local/lib/libpython3.7m.a(ceval.o): relocation R_X86_64_PC32 against symbol `_PyRuntime' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
/usr/local/projects/videautom/gateware/hw_validation/spi2ksz/cocotb/envp37/lib/python3.7/site-packages/cocotb/share/makefiles/Makefile.rules:58: recipe for target '/usr/local/projects/videautom/gateware/hw_validation/spi2ksz/cocotb/build/libs/x86_64/libgpilog.so' failed

I modified the Makefile.inc in installed package to get -fPIC:

vim cocotb/envp37/lib/python3.7/site-packages/cocotb/share/makefiles
[replace -fpic with -fPIC]

And I have exactly the same error:

gcc -Werror -Wcast-qual -Wcast-align -Wwrite-strings -Wall -Wno-unused-parameter -fno-common -g -DDEBUG -fPIC -DICARUS -Wstrict-prototypes -Waggregate-return -DFILTER -DICARUS -shared -Xlinker -L/usr/local/lib -o /usr/local/projects/videautom/gateware/hw_validation/spi2ksz/cocotb/build/libs/x86_64/libgpilog.so /usr/local/projects/videautom/gateware/hw_validation/spi2ksz/cocotb/build/obj/x86_64/gpi_logging.o   -lpython3.7m 
/usr/bin/ld: /usr/local/lib/libpython3.7m.a(ceval.o): relocation R_X86_64_PC32 against symbol `_PyRuntime' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
/usr/local/projects/videautom/gateware/hw_validation/spi2ksz/cocotb/envp37/lib/python3.7/site-packages/cocotb/share/makefiles/Makefile.rules:58: recipe for target '/usr/local/projects/videautom/gateware/hw_validation/spi2ksz/cocotb/build/libs/x86_64/libgpilog.so' failed

If somebody have an idea ?

[themperek]

I can only guess this is something with the way you compile/link your python version since with coda and python 3.7.4 it works for me.

[imphil]

Can you give more details on the version of Debian you're using? Do you have added anything "special" in terms of package sources?

[Martoni]

For special package sources I don't know, I did lot of things since I have this computer.
My debian version is 9:

cat /etc/issue
Debian GNU/Linux 9 \n \l

And kernel version is :

uname -a
Linux fabien 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u5 (2019-08-11) x86_64 GNU/Linux

[Martoni]

I forgot to give the Makefile modified from ping_tun_tap example :

TOPLEVEL_LANG ?= verilog
SIM=icarus

ifneq ($(TOPLEVEL_LANG),verilog)

all:
	@echo "Skipping example due to TOPLEVEL_LANG=$(TOPLEVEL_LANG) not being verilog"
clean::

else

TOPLEVEL = TopSpi2Ksz

ifeq ($(OS),Msys)
WPWD=$(shell sh -c 'pwd -W')
else
WPWD=$(shell pwd)
endif

PWD=$(shell pwd)

VERILOG_SOURCES = $(PWD)/$(TOPLEVEL)Cocotb.v\
									$(PWD)/../TriStateBuffer.v
MODULE=test_top_spi_2_ksz

include $(shell cocotb-config --makefiles)/Makefile.inc
include $(shell cocotb-config --makefiles)/Makefile.sim

endif

$(PWD)/$(TOPLEVEL)Cocotb.v: $(PWD)/../$(TOPLEVEL).v
	python cocotbify.py -v $(PWD)/../$(TOPLEVEL).v -o $(PWD)/$(TOPLEVEL)Cocotb.v

[Martoni]

I think I found the solution. In fact I compiled python myself with official python source.
To use it with cocotb I had to add option --enable-shared.

Here the procedure that worked for me is :

$ ./configure --enable-shared
$ make
$ sudo make install

Then, under the cocotb directory (I had to configure my libpath for local lib also):

$ cd mycocotb/
$ export LD_LIBRARY_PATH=/usr/local/lib
$ virtualenv --python=/usr/local/bin/python3.7 envp37
$ source envp37/bin/activate
$  python --version
Python 3.7.4
$ python -m pip install cocotb
$ make

And things done 👍