libpthread package does not contain libpthread.so

[ec1oud]

Maintainer: @felix Fietkau
Environment: OpenWrt 19.07.2, r10947-65030d81f3 on GnuBee 2, ramips MediaTek MT7621

Description:
The musl faq http://www.musl-libc.org/faq.html has this near the bottom:

Why is libm.a empty?

On musl, the entire standard library is included in a single library file — libc.a for static linking, and libc.so for dynamic linking. This significantly improves the efficiency of dynamic linking, and avoids all sorts of symbol interposition bugs that arise when you split the libraries up — bugs which have plagued glibc for more than a decade.

Why not just omit libm.a, libpthread.a, etc. entirely? POSIX says -lm, -lpthread, etc. are valid options to the compiler and conforming applications must use these options when they need the corresponding functionality, and the easiest way to comply with that requirement (and avoid breaking applications) is with empty .a files by those names.

So the fact that the libpthread package is completely empty is not POSIX-compliant. Some software will try to link with it, and that fails. In my case I'm trying to build https://github.com/ipfs/go-ipfs; I don't know where -lpthread comes from, but it fails. libpthread.a or .so should exist and be linkable, even if it doesn't define any functions.

root@gnubee:~# opkg list | grep pthread
libpthread - 1.1.24-2
root@gnubee:~# opkg files libpthread
Package libpthread (1.1.24-2) is installed on root and has the following files:
root@gnubee:~# 

Now I try to build ipfs as an unprivileged user:

cd ~/src/ipfs/go-ipfs/
$ make nofuse 
go version go1.13.12 linux/mipsle
bin/check_go_version 1.13.8
plugin/loader/preload.sh > plugin/loader/preload.go
go fmt plugin/loader/preload.go >/dev/null
go build  "-asmflags=all='-trimpath=/home/rutledge/src/go'" "-gcflags=all='-trimpath=/home/rutledge/src/go'" -ldflags="-X "github.com/ipfs/go-ipfs".CurrentCommit=36789eaa8" -tags="nofuse" -o "cmd/ipfs/ipfs" "github.com/ipfs/go-ipfs/cmd/ipfs"
# runtime/cgo
/usr/bin/ld: cannot find -lpthread
collect2: error: ld returned 1 exit status
make: *** [cmd/ipfs/Rules.mk:22: cmd/ipfs/ipfs] Error 2

It should be possible to build without using cgo, but that takes much more memory, to the point that it runs out of memory and reboots, even after I've added and enabled a swap partition. (There seems to be a 2GB limit for swap, but with 512M memory I guess that's reasonable.)

BTW it would be nice to have ipfs packaged for OpenWRT. It seems a very good use for the GnuBee https://www.crowdsupply.com/gnubee/personal-cloud-2 if the memory is enough (I'm not sure if it is though - that's why I'm trying). It seems I will need to get set up to cross compile OpenWrt, to go any further with this experiment.

[neheb]

You're compiling on the device itself?

[ec1oud]

Yes I tried. But libdl is also missing. I made stub libraries libpthread.a and libdl.a just so the linking step would work... but then ran out of memory again anyway.

But it's not so important now that I realized I need to set the softfloat option, then I can cross-compile and it actually runs on the gnubee. GOARCH=mipsle GOOS=linux GOMIPS=softfloat

Still, not having libpthread must be an obstacle for compiling a lot of things.

[neheb]

Most people cross compile. The highest version of GCC available is GCC7 while GCC10 is available as a pull request for cross compilation.

Maybe @noblepepper can help.

[mabachel]

I don't know what this command does and where it comes from but it helped me with the libpthred issue (/usr/bin/ld: cannot find -lpthread).
ar -rc /usr/lib/libpthread.a

https://forum.openwrt.org/t/howto-setup-zigbee2mqtt-on-openwrt/31856
https://linux.die.net/man/1/ar

[noblepepper]

The gcc package has an option to include libpthread on the target but it is default no. To get a gcc package with it you would need to compile your own image and set this option in the gcc package, at that point cross compiling would be much faster and probably easier.

You could also use libpthread from a downloaded sdk by putting in on the target under /usr/lib. The command from the makefile is
cp -a $(TOOLCHAIN_DIR)/lib/libpthread.a $(1)/usr/lib/$(PKG_NAME)/$(REAL_GNU_TARGET_NAME)/$(PKG_VERSION)
but I think the linker will find it anywhere in /usr/lib. I believe ar -rc /usr/lib/libpthread.a creates an empty archive (which is all that is needed) so it should work as well.

On target compiling is very limited, ram is one of the main issues. Swap doesn't always help me much I still get crashes if large amounts of ram are needed.