Test suite failure on 32 bit architectures, like armhf or x86

[slyon]

What version of Go are you using (go version)?

$ go version
go version go1.14.3 linux/armhf # on Ubuntu
go version go1.14.2 linux/i386 # on Debian

What operating system are you using?

Debian/Ubuntu Linux (32 bit)

What version of Badger are you using?

2.0.3

Does this issue reproduce with the latest master?

(Not yet tested)

Steps to Reproduce the issue

Running the following command inside an Ubuntu Groovy armhf machine (e.g. RaspberryPi) within the badger 2.0.3 source:
Download and install http://cdimage.ubuntu.com/ubuntu-server/daily-preinstalled/current/groovy-preinstalled-server-armhf+raspi.img.xz (e.g.: https://wiki.ubuntu.com/ARM/RaspberryPi)
Then:

ssh ubuntu@ubuntu #password: ubuntu
sudo apt install golang-go git devscripts
sudo apt install dh-golang golang-any golang-github-cespare-xxhash-dev golang-github-datadog-zstd-dev golang-github-dgraph-io-ristretto-dev golang-github-dgryski-go-farm-dev golang-github-dustin-go-humanize-dev golang-github-pkg-errors-dev golang-github-spf13-cobra-dev golang-github-stretchr-testify-dev golang-golang-x-net-dev golang-golang-x-sys-dev golang-goprotobuf-dev golang-snappy-go-dev
git clone https://salsa.debian.org/go-team/packages/badger.git
wget http://deb.debian.org/debian/pool/main/b/badger/badger_2.0.3.orig.tar.gz
cd badger
debuild

What Badger options were set?

Default test suite options.

What did you do?

Running the test suite while building the Debian/Ubuntu package:

go test -vet=off -v -p 4 github.com/dgraph-io/badger/badger github.com/dgraph-io/badger/badger/cmd github.com/dgraph-io/badger/integration/testgc github.com/dgraph-io/badger/options github.com/dgraph-io/badger/pb github.com/dgraph-io/badger/skl github.com/dgraph-io/badger/table github.com/dgraph-io/badger/trie github.com/dgraph-io/badger/y

What did you expect to see?

A successful run of the badger test suite and thus successful build of the Debian/Ubuntu package. Similar to this Ubuntu Groovy arm64 (64 bit) test & build:
https://launchpadlibrarian.net/477251471/buildlog_ubuntu-groovy-arm64.badger_2.0.3-1_BUILDING.txt.gz

What did you see instead?

A failure of the badger test suite and thus failed build of the 32 bit Debian/Ubuntu packages. The detailed logs can be found here for a Ubuntu Groovy armhf (32 bit) test run: https://launchpadlibrarian.net/482048892/buildlog_ubuntu-groovy-armhf.badger_2.0.3-1_BUILDING.txt.gz

Looking at the Debian build logs, we can find very similar issues for all 32 bit architectures (e.g. i386, armhf, armel, mipsel, ...):
https://buildd.debian.org/status/package.php?p=badger&suite=sid – while it is working on the 64 bit architectures for Debian and Ubuntu.

[jarifibrahim]

Hey @slyon , badger tests do fail on 32 bit machine. We tried to run badger tests on 32 bit machine on travis #1216 but it looks like travis is using a 64 bit machine.

We're looking for ways to run 32 bit tests on travis CI. if you have any suggestions, that would be great. Fixing the tests for 32 bit is easy, the difficult part is ensuring we don't break the build again.

[slyon]

Hey @jarifibrahim , is this "easy fix" already available somewhere? Or what would I need to do, in order to fix the 32 bit tests for Debian & Ubuntu? It seems like the code changes from #1216 are already part of the 2.0.3 release, which is failing to build here...

TravisCI seems to only support 64 bit machines, indeed. But you might be able to run a 32 bit image or container on a 64 bit host machine, e.g. 32 bit Ubuntu Groovy armhf on travis arm64 linux machine or 32 bit Debian testing i386 on travis amd64 linux machine. I never did that myself, though..

[jarifibrahim]

Most of the crashes I see are invalid filename https://buildd.debian.org/status/fetch.php?pkg=badger&arch=i386&ver=2.0.3-1&stamp=1587719525&raw=0 and this could be because an int overflowing on 32-bit system. I also see a cannot allocate memory which could be because the CI system doesn't have enough memory.

We don't have a fix for the failing tests yet.

I see delve runs 32 bit tests in a container and badger could do the same https://github.com/chainhelen/delve/blob/master/.travis.yml#L51-L82

[slyon]

Most of the crashes I see are invalid filename https://buildd.debian.org/status/fetch.php?pkg=badger&arch=i386&ver=2.0.3-1&stamp=1587719525&raw=0 and this could be because an int overflowing on 32-bit system. I also see a cannot allocate memory which could be because the CI system doesn't have enough memory.

That's what I thought at first.. But as it is failing on all 32 bit architectures across different projects, I have the impression that maybe the 2GB memory limit of 32 bit arches might be exceeded during the test, which would fail with a similar error. Especially as the Debian & Ubuntu autopkgtest-cloud uses very similar VMs for the different architectures, so if there is enough memory to run it on arm64, there should also be enough memory to run it on armhf.

We don't have a fix for the failing tests yet.

Okay, let's see if I can find some time to dig into it.

I see delve runs 32 bit tests in a container and badger could do the same https://github.com/chainhelen/delve/blob/master/.travis.yml#L51-L82

Yes! It would be great if 32 bit compatibility could be checked regularly. :-)

[jarifibrahim]

Yes! It would be great if 32 bit compatibility could be checked regularly. :-)

I've created #1385 for it

That's what I thought at first.. But as it is failing on all 32 bit architectures across different projects, I have the impression that maybe the 2GB memory limit of 32 bit arches might be exceeded during the test, which would fail with a similar error. Especially as the Debian & Ubuntu autopkgtest-cloud uses very similar VMs for the different architectures, so if there is enough memory to run it on arm64, there should also be enough memory to run it on armhf.

Some tests in badger might be running in pure-in-memory mode (for instance the write batch in-memory tests) and they could take up more than 2 GB.

[minhaj-shakeel]

Github issues have been deprecated.
This issue has been moved to discuss. You can follow the conversation there and also subscribe to updates by changing your notification preferences.