Compiler error on arm64 / rpi

[svenger87]

I´m trying to compile the project mentioned in #28 on a raspberry Pi4 running ubuntu 20.10 arm64.

The compiler interrupts with

/root/go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.32/backup.go:44:13: could not determine kind of name for C.rocksdb_backup_engine_open_opts /root/go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.32/backup.go:136:2: could not determine kind of name for C.rocksdb_backup_engine_restore_db_from_backup

Do you have any idea how to get around this?

[linxGnu]

@svenger87 could you please describe more how you install rocksdb:

• Which version of rocksdb?
• Shared or Static?
• How you link grocksdb to shared/static rocksdb lib?
• Which command did you invoke to build?
• Could you please show me full error logs?

There is no builtin-static lib for arm64 at the moment. So, please make sure that you install rocksdb before building grocksdb.

[svenger87]

Thanks for your answer so far.

It already fails for the go get.
I compiled rocksdb from source.

Package: librocksdb
Status: install ok installed
Priority: optional
Section: misc
Installed-Size: 5390
Maintainer: None <none@example.com>
Architecture: arm64
Version: 6.5.3-deb3-1

dpkg -s librocksdb-dev
Package: librocksdb-dev
Status: install ok installed
Priority: optional
Section: misc
Installed-Size: 804
Maintainer: None <none@example.com>
Architecture: all
Source: librocksdb
Version: 6.5.3-deb3-1
Depends: librocksdb (= 6.5.3-deb3-1)

go get -u github.com/linxGnu/grocksdb
go: downloading github.com/linxGnu/grocksdb v1.6.32
# github.com/linxGnu/grocksdb
/root/go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.32/backup.go:44:13: could not determine kind of name for C.rocksdb_backup_engine_open_opts
/root/go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.32/backup.go:136:2: could not determine kind of name for C.rocksdb_backup_engine_restore_db_from_backup```

[linxGnu]

@svenger87 as your rocksdb version (6.5.3), I think you should use this version of grocksdb: https://github.com/linxGnu/grocksdb/releases/tag/v1.6.11

Please try:

go get github.com/linxGnu/grocksdb@1.6.11

[svenger87]

@linxGnu i recompiled to 6.16.3 and get the same.
dpkg -s librocksdb-dev
Package: librocksdb-dev
Status: install ok installed
Priority: optional
Section: misc
Installed-Size: 804
Maintainer: None none@example.com
Architecture: all
Source: librocksdb
Version: 6.16.3-deb3-1
Depends: librocksdb (= 6.16.3-deb3-1)
Description: RocksDB Key-Value database (development files)

[linxGnu]

@svenger87

Could you please try this:

CGO_CFLAGS="-I/path/to/rocksdb/include" \
CGO_LDFLAGS="-L/path/to/rocksdb -lrocksdb -lstdc++ -lm -lz -lsnappy -llz4 -lzstd" \
go get github.com/linxGnu/grocksdb@1.6.11

In this way, please replace /path/to/rocksdb with your shared lib. Normally, it's /usr/lib64 or /usr/local/lib

[linxGnu]

Just one more thing, as I see your upgraded to rocksdb 6.16.3. I think you could use latest version of grocksdb. I hope you could build it fluently.

Please let me know if I could help

[svenger87]

1.6.11 is working. But as soon as i try to fetch a more recent version i get the errors above.
1.6.32 is needed for a depencency. So i can´t really go with a lower version.

This really only happens on arm64. amd64 is building fine.

[linxGnu]

Yeah, as I remember: from rocksdb 1.1x, there are breaking changes. That's why you should use proper version of grocksdb for rocksdb 1.1x.

I am sorry, I don't have arm64 device here to help you. So, currently:

• 1.6.11 works for you with your rocksdb 6.5.3?
• 1.6.32 not work for rocksdb 6.16.3?

Could you please confirm your current situation. I see the mix of issue reporting. First: rocksdb 6.5.3, then rocksdb 6.16.3.

[svenger87]

1.6.11 is working for both. With 6.5.3 and 6.16.3.
1.6.32 and lower versions throw the errors during go get.

[linxGnu]

@svenger87 I could go get pretty well with my current ubuntu 20.04 ARM

OS:

uname -a
Linux ubuntuarm 5.4.0-67-generic #75-Ubuntu SMP Fri Feb 19 18:03:59 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

Go env:

GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/home/gnu/.cache/go-build"
GOENV="/home/gnu/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/gnu/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/gnu/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_arm64"
GOVCS=""
GOVERSION="go1.16.2"
GCCGO="/usr/bin/gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/gnu/go/src/github.com/linh/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build613075368=/tmp/go-build -gno-record-gcc-switches"

Build rocksdb with this script:

#!/bin/bash
DIRECTORY="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

INSTALL_PREFIX=/usr/local

export CFLAGS='-fPIC -O3 -pipe -funroll-loops' 
export CXXFLAGS='-fPIC -O3 -pipe -funroll-loops'

BUILD_PATH=/tmp/build
mkdir -p $BUILD_PATH

CMAKE_REQUIRED_PARAMS="-DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_INSTALL_PREFIX=${INSTALL_PREFIX}"

zlib_version="1.2.11"
cd $BUILD_PATH && wget https://github.com/madler/zlib/archive/v${zlib_version}.tar.gz && tar xzf v${zlib_version}.tar.gz && cd zlib-${zlib_version} && \
    ./configure --prefix=$INSTALL_PREFIX --static && make -j16 install && \
    cd $BUILD_PATH && rm -rf *

snappy_version="1.1.8"
cd $BUILD_PATH && wget https://github.com/google/snappy/archive/${snappy_version}.tar.gz && tar xzf ${snappy_version}.tar.gz && cd snappy-${snappy_version} && \
    mkdir -p build_place && cd build_place && cmake $CMAKE_REQUIRED_PARAMS -DSNAPPY_BUILD_TESTS=OFF .. && make install/strip -j16 && \
    cd $BUILD_PATH && rm -rf *

lz4_version="1.9.3"
cd $BUILD_PATH && wget https://github.com/lz4/lz4/archive/v${lz4_version}.tar.gz && tar xzf v${lz4_version}.tar.gz && cd lz4-${lz4_version}/build/cmake && \
    cmake $CMAKE_REQUIRED_PARAMS -DLZ4_BUILD_LEGACY_LZ4C=OFF -DBUILD_SHARED_LIBS=OFF -DLZ4_POSITION_INDEPENDENT_LIB=ON && make -j16 install && \
    cd $BUILD_PATH && rm -rf *

zstd_version="1.4.9"
cd $BUILD_PATH && wget https://github.com/facebook/zstd/archive/v${zstd_version}.tar.gz && tar xzf v${zstd_version}.tar.gz && \
    cd zstd-${zstd_version}/build/cmake && mkdir -p build_place && cd build_place && \
    cmake -DZSTD_BUILD_PROGRAMS=OFF -DZSTD_BUILD_CONTRIB=OFF -DZSTD_BUILD_STATIC=ON -DZSTD_BUILD_SHARED=OFF -DZSTD_BUILD_TESTS=OFF \
    -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DZSTD_ZLIB_SUPPORT=ON -DZSTD_LZMA_SUPPORT=OFF -DCMAKE_BUILD_TYPE=Release .. && make -j$(nproc) install && \
    cd $BUILD_PATH && rm -rf * && ldconfig

rocksdb_version="6.16.3"
cd $BUILD_PATH && wget https://github.com/facebook/rocksdb/archive/v${rocksdb_version}.tar.gz && tar xzf v${rocksdb_version}.tar.gz && cd rocksdb-${rocksdb_version}/ && \
    mkdir -p build_place && cd build_place && cmake -DCMAKE_BUILD_TYPE=Release $CMAKE_REQUIRED_PARAMS -DCMAKE_PREFIX_PATH=$INSTALL_PREFIX -DWITH_TESTS=OFF -DWITH_GFLAGS=OFF \
    -DWITH_BENCHMARK_TOOLS=OFF -DWITH_TOOLS=OFF -DWITH_MD_LIBRARY=OFF -DWITH_RUNTIME_DEBUG=OFF -DROCKSDB_BUILD_SHARED=OFF -DWITH_SNAPPY=ON -DWITH_LZ4=ON -DWITH_ZLIB=ON \
    -DWITH_ZSTD=ON -DWITH_BZ2=OFF -WITH_GFLAGS=OFF .. && make -j16 install/strip && \
    cd $BUILD_PATH && rm -rf *

Noted:

• Above script build all dependencies within /usr/local
• Above script build static version of rocksdb. You could build share libs as well.

Then

go get -u github.com/linxGnu/grocksdb
# no problem

I think you could give it a try.

And, grocksdb 1.6.32 is built up on rocksdb 6.16.3. It DOES NOT work for rocksdb 6.5.3

[svenger87]

Okay. go get is working now. No i have tons of other errors compiling the binary i want to build. 🛠️

Thanks for your help so far. Much appreciated.

[svenger87]

I was missing gccgo. Fixed.