Adjust dependencies to make it possible to statically link dqlite.

[evsign]

Hi. Currently, if you try to build a statically linked executable you'll see a lot of errors like:

$ CGO_LDFLAGS="-static" CGO_LDFLAGS_ALLOW="-Wl,-z,now" go build -tags libsqlite3 ./cmd/dqlite-demo
....
(.text+0x3b8): undefined reference to `uv_read_start'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libdqlite.a(transport.o): in function `transport__close':
(.text+0x351): undefined reference to `uv_close'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libdqlite.a(transport.o): in function `transport__write':
(.text+0x440): undefined reference to `uv_write'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libdqlite.a(command.o): in function `command__encode':
(.text+0x74a): undefined reference to `raft_malloc'
/usr/bin/ld: (.text+0x7e0): undefined reference to `raft_malloc'
/usr/bin/ld: (.text+0x86b): undefined reference to `raft_malloc'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libdqlite.a(command.o): in function `command__decode':
(.text+0x92b): undefined reference to `raft_malloc'
/usr/bin/ld: (.text+0x99c): undefined reference to `raft_malloc'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libdqlite.a(command.o):(.text+0x9e6): more undefined references to `raft_malloc' follow
collect2: error: ld returned 1 exit status

To get rid of these errors we need to ensure that all of dqlite's dependencies are also linked to the executable. So we need to specify them in the correct order after the dqlite library itself.

[cole-miller]

I guess -luv_a forces cc to link against libuv.a instead of libuv.so? I appreciate the virtues of static linking and I'd be happy to support it better in go-dqlite, but I'm very wary of modifying the cgo setup in a way that affects all users of go-dqlite like this. People may be depending on libuv being linked dynamically.

If you're willing to wait a bit, I'm aiming to make the upcoming C client library for dqlite support static linking, and I have a setup for building a fully-static libdqlite that I can publish and point you to.

[evsign]

Not really. It appears that my distribution (ubuntu 22.04) comes with the libuv_a.a file instead of libuv.a, and I assumed that this could be the default naming convention for the static version of the libuv library.

However, the switch for static linking comes with the -static ld flag, which forces ld to look for *.a files instead of *.so files. I initially thought that this wouldn't affect dynamic linking at all since the .a extension would be ignored during dynamic linking. At least it seems like that's what's happening on my system. But yeah, it looks a little dirty.

Didn't know that you already had plans to support fully-static builds. That will help a lot, thank you!

[cole-miller]

Ah, yeah, this is tricky. I guess libuv doesn't ship libuv.a at all (I thought it did, but on double-checking I have the same thing in /usr/lib/x86_64-linux-gnu as you do). That means you really have to use a different -l flag depending on whether you want libuv to be statically or dynamically linked -- go-dqlite can't straightforwardly support both at once. I definitely don't want to silently switch the default behavior (and I don't know whether non-Ubuntu systems have libuv_a.a as well), but we should definitely figure out a way to produce static binaries with go-dqlite. Maybe we could use a build tag to select between two versions of build.go, one of which has -static -luv_a and one of which has -luv?

[evsign]

It could be an option, but I'm unsure about including -static in the #cgo linux LDFLAGS: directive. This flag forces the static build for the entire application. But if we don't include this, then users would need to add it themselves. Or maybe we could do so just with the big red warning in the readme?:)

From the user experience perspective it just looks like the providing a static version of the dqlite library with all dependencies included might be the best choice. In this case, the build process will remain unchanged. However, it might not be very flexible for end-users who want to build with their own versions of the dependencies.

A lot of tradeoffs:)

[evsign]

After some thought, I think it would be better to stick with the dqlitestatic tag.

build_static.go

// +build dqlitestatic

package bindings

/*
#cgo linux LDFLAGS: -static -ldqlite -lraft -llz4 -luv_a -lm -Wl,-z,now
*/
import "C"

and readme warning about this. It just combines the possibility to replace dependencies if you want and a convenient build process where you just need to specify a new tag. What do you think?

[evsign]

After some investigation, it looks like this filename has changed starting from ubuntu jammy. And the old releases contain normal libuv.a file. And this comes from the libuv itself. Starting from the v.1.40.0 they started shipping libuv_a libuv/libuv#2988

Considering this, maybe it would be better to provide several tags? Like dqlitestatic_libuva and dqlitestatic_libuv? I just ran out of ideas:)

Or maybe there could be some workarounds with pkg-config?

[freeekanayaka]

Hi. Currently, if you try to build a statically linked executable you'll see a lot of errors like:

$ CGO_LDFLAGS="-static" CGO_LDFLAGS_ALLOW="-Wl,-z,now" go build -tags libsqlite3 ./cmd/dqlite-demo

IIRC passing -tags libsqlite3 to go build ends up in linking libsqlite3.so dynamically and not statically. Why would you be willing to link SQLite dynamically but not dqlite? Is it because dqlite packages are not available for your Linux distribution (or they are not recent enough)?

Just trying to understand your use case, so we have better information when deciding for the most appropriate solution.

[evsign]

Hi. Currently, if you try to build a statically linked executable you'll see a lot of errors like:

$ CGO_LDFLAGS="-static" CGO_LDFLAGS_ALLOW="-Wl,-z,now" go build -tags libsqlite3 ./cmd/dqlite-demo

IIRC passing -tags libsqlite3 to go build ends up in linking libsqlite3.so dynamically and not statically. Why would you be willing to link SQLite dynamically but not dqlite? Is it because dqlite packages are not available for your Linux distribution (or they are not recent enough)?

Just trying to understand your use case, so we have better information when deciding for the most appropriate solution.

As far as I see, this tag just adds -lsqlite3 to the LDFLAGS, but what really switches static or dynamic linking is the -static flag. With -static flag ld will try to link all libraries statically, i.e. it will force the linker to look for *.a files instead of *.so. And since the sqlite3 apt package includes libsqlite3.a, this results in a statically linked sqlite3 library. From Matt's go-sqlite3 package perspective, if I understand correctly, the libsqlite3 tag allows you to build go-sqlite3 with your own version of the sqlite3 (or link to the one from the system) instead of using the one provided by the package (which includes an amalgamated sqlite3 in the sources).

Regarding my use case, I just want to build a fully static go executable:) With all dependencies included. And with these changes, it actually works.

Static example:

$ CGO_LDFLAGS="-static" CGO_LDFLAGS_ALLOW="-Wl,-z,now" go build -tags libsqlite3 ./cmd/dqlite-demo
#.... some warnings related to the golang itself
$ ldd dqlite-demo
> not a dynamic executable

Dynamic example:

$ CGO_LDFLAGS_ALLOW="-Wl,-z,now" go build -tags libsqlite3 ./cmd/dqlite-demo
$ ldd dqlite-demo
> linux-vdso.so.1 (0x00007fffe0d54000)
  libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007fd6e146f000)
  libdqlite.so.0 => /lib/x86_64-linux-gnu/libdqlite.so.0 (0x00007fd6e13f0000)
  libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd6e11c8000)
  libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd6e10e1000)
  libuv.so.1 => /lib/x86_64-linux-gnu/libuv.so.1 (0x00007fd6e10af000)
  libraft.so.3 => /lib/x86_64-linux-gnu/libraft.so.3 (0x00007fd6e1047000)
  /lib64/ld-linux-x86-64.so.2 (0x00007fd6e15c7000)
  liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007fd6e1027000)

[freeekanayaka]

IIRC passing -tags libsqlite3 to go build ends up in linking libsqlite3.so dynamically and not statically. Why would you be willing to link SQLite dynamically but not dqlite? Is it because dqlite packages are not available for your Linux distribution (or they are not recent enough)?
Just trying to understand your use case, so we have better information when deciding for the most appropriate solution.

As far as I see, this tag just adds -lsqlite3 to the LDFLAGS, but what really switches static or dynamic linking is the -static flag. With -static flag ld will try to link all libraries statically, i.e. it will force the linker to look for *.a files instead of *.so. And since the sqlite3 apt package includes libsqlite3.a, this results in a statically linked sqlite3 library. From Matt's go-sqlite3 package perspective, if I understand correctly, the libsqlite3 tag allows you to build go-sqlite3 with your own version of the sqlite3 (or link to the one from the system) instead of using the one provided by the package (which includes an amalgamated sqlite3 in the sources).

I might be confused or not remember correctly, but if don't pass -tags libsqlite3 to go build, then I believe that the go-sqlite3 will still compile and use the amalgamented sqlite3 source that it includes, regardless of whether GO_LDFLAGS="-static" was used or not. I didn't check this, so it's just my intuition.

[evsign]

I might be confused or not remember correctly, but if don't pass -tags libsqlite3 to go build, then I believe that the go-sqlite3 will still compile and use the amalgamented sqlite3 source that it includes, regardless of whether GO_LDFLAGS="-static" was used or not. I didn't check this, so it's just my intuition.

I understand:) But as I see, it cannot happen. Just wanted to share my observations to make it easier to check:

When I said "this tag just adds -lsqlite3 to the LDFLAGS" it's been a little roughly speaking. In fact, it also adds USE_LIBSQLITE3 define and then if we take a look at the sqlite3-binding.c we can see that at the very first line, there is a check #ifndef USE_LIBSQLITE3 which includes amalgamation only if USE_LIBSQLITE3 is not defined.

And at line #241718, there is:

#else // USE_LIBSQLITE3
 // If users really want to link against the system sqlite3 we
// need to make this file a noop.
 #endif

So to summarize, if USE_LIBSQLITE3 is defined, then sqlite3-binding.c is almost empty, while if it is not defined (#ifndef USE_LIBSQLITE3), then it contains the sqlite3 code.

[freeekanayaka]

I might be confused or not remember correctly, but if don't pass -tags libsqlite3 to go build, then I believe that the go-sqlite3 will still compile and use the amalgamented sqlite3 source that it includes, regardless of whether GO_LDFLAGS="-static" was used or not. I didn't check this, so it's just my intuition.

I understand:) But as I see, it cannot happen. Just wanted to share my observations to make it easier to check:

When I said "this tag just adds -lsqlite3 to the LDFLAGS" it's been a little roughly speaking. In fact, it also adds USE_LIBSQLITE3 define and then if we take a look at the sqlite3-binding.c we can see that at the very first line, there is a check #ifndef USE_LIBSQLITE3 which includes amalgamation only if USE_LIBSQLITE3 is not defined.

And at line #241718, there is:

#else // USE_LIBSQLITE3
 // If users really want to link against the system sqlite3 we
// need to make this file a noop.
 #endif

So to summarize, if USE_LIBSQLITE3 is defined, then sqlite3-binding.c is almost empty, while if it is not defined (#ifndef USE_LIBSQLITE3), then it contains the sqlite3 code.

That was my understanding indeed. But I was assuming that if you pass -tags libsqlite3 (in order to avoid the compilation of the vendored version) you would be "forced" to link to libsqlite3.so dynamically. I guess that bit is wrong, and that if you pass -tags libsqlite3 and -static then you'd statically link to libsqlite3.a? Again, I didn't check this, just trying to understand the situation.

[Zxilly]

I have successfully static link dqlite in golang. You can ref to https://github.com/openPanel/core/blob/master/Taskfile.yml

Seems it works well without change the dqlite source.

Of course, symbol like dlopen still needs glibc at runtime. But I think it won't be a big problem.

[Zxilly]

And as my personal curious, why -lm.
I didn't find libm in dqlite's dependencies graph.

[evsign]

I have successfully static link dqlite in golang. You can ref to https://github.com/openPanel/core/blob/master/Taskfile.yml

Seems it works well without change the dqlite source.

Of course, symbol like dlopen stills needs glibc at runtime. But I think it won't be a big problem.

It works! Thank you very much. I had previously tried passing everything through the CGO_LDFLAGS variable, but it seemed to concatenate the flags to the beginning of the ld flags. While -ldflags '-extldflags places the flags at the end of the list? Anyway, it solves the issue. Thank you!

And as my personal curious, why -lm. I didn't find libm in dqlite's dependencies graph.

It appears to be a sqlite dependency in my system. Without it, I get the following errors

CGO_LDFLAGS_ALLOW="-Wl,-z,now" go build -tags libsqlite3 -ldflags '-extldflags "-static -lraft -llz4 -luv_a"' ./cmd/dqlite-demo
....
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(fts5.o): in function `fts5Bm25Function':
(.text+0x1cc3): undefined reference to `log'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o): in function `logFunc':
(.text+0x1f49): undefined reference to `log'
/usr/bin/ld: (.text+0x1fb1): undefined reference to `log'
/usr/bin/ld: (.text+0x1fe1): undefined reference to `log'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x1328): undefined reference to `trunc'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x14d8): undefined reference to `exp'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x1520): undefined reference to `pow'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x1568): undefined reference to `pow'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x15b0): undefined reference to `fmod'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x15f8): undefined reference to `acos'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x1640): undefined reference to `asin'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x1688): undefined reference to `atan'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x16d0): undefined reference to `atan2'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x1718): undefined reference to `cos'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x1760): undefined reference to `sin'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x17a8): undefined reference to `tan'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x17f0): undefined reference to `cosh'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x1838): undefined reference to `sinh'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x1880): undefined reference to `tanh'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x18c8): undefined reference to `acosh'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x1910): undefined reference to `asinh'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x1958): undefined reference to `atanh'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libsqlite3.a(func.o):(.data.rel+0x19a0): undefined reference to `sqrt'
collect2: error: ld returned 1 exit status

[Zxilly]

seems it because i didn't pass libsqlite3 build tag. I just use go-sqlite3 builtin version.
It seems that the reason I made this decision at the time was because go-sqlite came with a newer version than the one I installed from the system package manager.