New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Something wrong with i686 (magic numbers) #8
Comments
Thanks for catching it.
The first thing I want to distinguish is whether the error happens always
or not.
I mean, I want to distinguish whether it is a threading problem (race
condition) or not.
Do you reproduce the same error if you run the check command again?
…On Fri, Jan 22, 2021 at 12:56 AM Eugene ***@***.***> wrote:
Due this
<https://kojipkgs.fedoraproject.org//work/tasks/9580/60149580/build.log>
log make check on x86 ends with:
LD_LIBRARY_PATH=.:/usr/local/lib: ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks \
--iter 20000 --threads 5 --size 8 --step_unit 4 --max_level 10
Setting: impl=skip num_iterations=20000 value_size=8 num_threads=5
...
.................................................. (00020000)
Synchronizing: ... done (elapsed=0.178743)
Removing done: elapsed_time=0.258109 num_records=0 qps=387433 mem=6052000
file_size=128 eff_data_size=0 efficiency=0.00%
make[1]: Leaving directory '/builddir/build/BUILD/tkrzw-0.9.3'
RPM build errors:
Get failed: Get failed: BROKEN_DATA_ERROR: invalid record magic number
Get failed: BROKEN_DATA_ERROR: invalid record magic number
Get failed: BROKEN_DATA_ERROR: invalid record magic number
Get failed: BROKEN_DATA_ERROR: invalid record magic number
BROKEN_DATA_ERROR: invalid record magic number
make[1]: *** [Makefile:307: check-skipdbm-perf] Error 1
make: *** [Makefile:126: check] Error 2
This is exactly i686 error. All others
<https://koji.fedoraproject.org/koji/taskinfo?taskID=60149461> builds OK.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#8>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQGJVRECTZCGM4M434LIAKTS3BFEFANCNFSM4WNDDYYQ>
.
|
The say - not always.
As for me I'm at x64 host (Fedora 33) and have no i686 near (Fedora not distributed in i686 version starting from F32). |
Hi, I am the reviewer in the linked Fedora bug. To clarify, this error occurs every time. I can easily reproduce it outside the Fedora build system, and even without any patches or extra build flags from the distribution. All it takes, on my x86_64 system, is The tests do occasionally die with “Aborted (core dumped)” in check-treedbm-perf on some platforms, but this is a different problem, and I do not know if it reflects a bug in tkrzw or a problem in the environment. That may be what caused the confusion. |
Hi Ben,
Thanks for the explanation.
I don't have an i686 environment either.
By design, Trkzw doesn't depend on specific CPU design, bus width, or
endian.
So, it's strange to me that the test fails on i686 only.
Do you know other environments where the test fails?
…On Fri, Jan 22, 2021 at 9:29 PM Ben Beasley ***@***.***> wrote:
Hi, I am the reviewer in the linked Fedora bug. To clarify, this error
occurs every time. I can easily reproduce it outside the Fedora build
system, and even without any patches or extra build flags from the
distribution. All it takes, on my x86_64 system, is ./configure && make
&& make check in a 32-bit (i386/i686) chroot.
The tests do occasionally die with “Aborted (core dumped)” in
check-treedbm-perf on some platforms, but this is a different problem, and
I do not know if it reflects a bug in tkrzw or a problem in the
environment. That may be what caused the confusion.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#8 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQGJVRGXAD35QWJVTXN4ED3S3FVQZANCNFSM4WNDDYYQ>
.
|
May be the problem is not in exactly i686 but in 32-bit. What about byte align? |
ARMv7 is 32-bit, so it is not just that. And ARM is stricter about alignment than x86, and has a weaker memory model. Passing I’m happy to try any tests that are helpful. |
I'd like to know this test fails or not. The difference is the number of threads is 1. ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks --iter 20000 --threads 1 --size 8 --step_unit 4 --max_level 10 If it doesn't fail, please increase the iteration (the value of --iter) to 100000. BTW, running the same test with "valgrind" doesn't report any abnormality on my x86_64 environment. |
You can see that it does fail. No segfault or anything; just a BROKEN_DATA_ERROR. |
The same test does not fail under valgrind:
Nor does it fail under valgrind with |
Thanks.
If the test with --threads 1 fails, the problem is not related to threading.
There should be some deterministic wrong behavior in Tkrzw's implementation.
But, one weird thing is that running on valgrind avoids the failure.
Anyway, I'll go over the code to locate the cause.
…On Sat, Jan 23, 2021 at 2:39 AM Ben Beasley ***@***.***> wrote:
The same test does not fail under valgrind:
<mock-chroot> sh-5.1# LD_LIBRARY_PATH=$PWD valgrind ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks --iter 20000 --threads 1 --size 8 --step_unit 4 --max_level 10
==18== Memcheck, a memory error detector
==18== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==18== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==18== Command: ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks --iter 20000 --threads 1 --size 8 --step_unit 4 --max_level 10
==18==
Setting: impl=skip num_iterations=20000 value_size=8 num_threads=1
.................................................. (00001000)
.................................................. (00002000)
.................................................. (00003000)
.................................................. (00004000)
.................................................. (00005000)
.................................................. (00006000)
.................................................. (00007000)
.................................................. (00008000)
.................................................. (00009000)
.................................................. (00010000)
.................................................. (00011000)
.................................................. (00012000)
.................................................. (00013000)
.................................................. (00014000)
.................................................. (00015000)
.................................................. (00016000)
.................................................. (00017000)
.................................................. (00018000)
.................................................. (00019000)
.................................................. (00020000)
Synchronizing: ... done (elapsed=7.996911)
Setting done: elapsed_time=10.236987 num_records=20000 qps=1954 mem=22440000
file_size=406816 eff_data_size=320000 efficiency=78.66%
Getting: impl=skip num_iterations=20000 value_size=8 num_threads=1
.................................................. (00001000)
.................................................. (00002000)
.................................................. (00003000)
.................................................. (00004000)
.................................................. (00005000)
.................................................. (00006000)
.................................................. (00007000)
.................................................. (00008000)
.................................................. (00009000)
.................................................. (00010000)
.................................................. (00011000)
.................................................. (00012000)
.................................................. (00013000)
.................................................. (00014000)
.................................................. (00015000)
.................................................. (00016000)
.................................................. (00017000)
.................................................. (00018000)
.................................................. (00019000)
.................................................. (00020000)
Getting done: elapsed_time=23.526716 num_records=20000 qps=850 mem=60320000
file_size=406816 eff_data_size=320000 efficiency=78.66%
Removing: impl=skip num_iterations=20000 value_size=8 num_threads=1
.................................................. (00001000)
.................................................. (00002000)
.................................................. (00003000)
.................................................. (00004000)
.................................................. (00005000)
.................................................. (00006000)
.................................................. (00007000)
.................................................. (00008000)
.................................................. (00009000)
.................................................. (00010000)
.................................................. (00011000)
.................................................. (00012000)
.................................................. (00013000)
.................................................. (00014000)
.................................................. (00015000)
.................................................. (00016000)
.................................................. (00017000)
.................................................. (00018000)
.................................................. (00019000)
.................................................. (00020000)
Synchronizing: ... done (elapsed=10.655468)
Removing done: elapsed_time=12.734186 num_records=0 qps=1571 mem=60664000
file_size=128 eff_data_size=0 efficiency=0.00%
==18==
==18== HEAP SUMMARY:
==18== in use at exit: 0 bytes in 0 blocks
==18== total heap usage: 596,566 allocs, 596,566 frees, 34,286,316 bytes allocated
==18==
==18== All heap blocks were freed -- no leaks are possible
==18==
==18== For lists of detected and suppressed errors, rerun with: -s
==18== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Nor does it fail under valgrind with --iter 100000, or --threads 5.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#8 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQGJVRHXCDCLUVV3KJTHNYTS3GZ4HANCNFSM4WNDDYYQ>
.
|
I'm now trying to verify this issue on VMware. |
Fedora Linux builds some “multilib” i686 packages as part of the x86_64 architecture, but doesn’t build a complete 32-bit release anymore. I expect you can best reproduce this by using the x86_64 DVD ISO for the just-released Fedora 34 from here, and adding |
OK, so I started the live DVD I linked, started “Terminal” ( sudo dnf install gcc-c++ libstdc++-devel.x86_64 libstdc++-devel.i686 libgcc.i686 glibc-devel.i686
wget https://github.com/estraier/tkrzw/archive/0d7badb65abcac72381ecc6fbd8f980ce6a53305.zip
unzip 0d7badb65abcac72381ecc6fbd8f980ce6a53305.zip
cd tkrzw-0d7badb65abcac72381ecc6fbd8f980ce6a53305/
CXXFLAGS=-m32 LDFLAGS=-m32 ./configure
# Change the number of parallel jobs as desired; my VM had 4 CPUs and adequate memory
make -j4
make check After a while (my workstation is an antique) I get:
Manually running the test we were looking at previously,
fails too. So it seems that’s a good way to reproduce it. |
Thanks. Reproduction is the greatest step to fix an issue!
BTW, the ISO image you wrote is for 64 bit. Is it intended?
…On Thu, Apr 29, 2021 at 5:03 AM Ben Beasley ***@***.***> wrote:
OK, so I started the live DVD I linked, started “Terminal” (gnome-terminal),
and ran:
sudo dnf install gcc-c++ libstdc++-devel.x86_64 libstdc++-devel.i686 libgcc.i686 glibc-devel.i686
wget https://github.com/estraier/tkrzw/archive/0d7badb65abcac72381ecc6fbd8f980ce6a53305.zip
unzip 0d7badb.zip
cd tkrzw-0d7badb65abcac72381ecc6fbd8f980ce6a53305/
CXXFLAGS=-m32 LDFLAGS=-m32 ./configure
# Change the number of parallel jobs as desired; my VM had 4 CPUs and adequate memory
make -j4
make check
After a while (my workstation is an antique) I get:
make check-skipdbm-perf
make[1]: Entering directory '/home/liveuser/tkrzw-0d7badb65abcac72381ecc6fbd8f980ce6a53305'
rm -Rf casket*
LD_LIBRARY_PATH=.:/usr/local/lib: ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks \
--iter 20000 --threads 5 --size 8 --step_unit 4 --max_level 10
Setting: impl=skip num_iterations=20000 value_size=8 num_threads=5
.................................................. (00001000)
.................................................. (00002000)
.................................................. (00003000)
.................................................. (00004000)
.................................................. (00005000)
.................................................. (00006000)
.................................................. (00007000)
.................................................. (00008000)
.................................................. (00009000)
.................................................. (00010000)
.................................................. (00011000)
.................................................. (00012000)
.................................................. (00013000)
.................................................. (00014000)
.................................................. (00015000)
.................................................. (00016000)
.................................................. (00017000)
.................................................. (00018000)
.................................................. (00019000)
.................................................. (00020000)
Synchronizing: ... done (elapsed=1.172894)
Setting done: elapsed_time=2.466667 num_records=100000 qps=40541 mem=8828000
file_size=2033480 eff_data_size=1600000 efficiency=78.68%
Getting: impl=skip num_iterations=20000 value_size=8 num_threads=5
Get failed: BROKEN_DATA_ERROR: invalid record magic number
Get failed: BROKEN_DATA_ERROR: invalid record magic number
Get failed: BROKEN_DATA_ERROR: invalid record magic number
Get failed: BROKEN_DATA_ERROR: invalid record magic numberGet failed: BROKEN_DATA_ERROR: invalid record magic number
Getting done: elapsed_time=0.020106 num_records=100000 qps=4973680 mem=7232000
file_size=2033480 eff_data_size=1600000 efficiency=78.68%
So it seems that’s a good way to reproduce it.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#8 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQGJVRE2UZZLPWYEL6Z3SW3TLBSYXANCNFSM4WNDDYYQ>
.
|
Ah, now I understand. Giving "-m32" is the key.
On Thu, Apr 29, 2021 at 11:47 AM Mikio Hirabayashi ***@***.***>
wrote:
… Thanks. Reproduction is the greatest step to fix an issue!
BTW, the ISO image you wrote is for 64 bit. Is it intended?
On Thu, Apr 29, 2021 at 5:03 AM Ben Beasley ***@***.***>
wrote:
> OK, so I started the live DVD I linked, started “Terminal” (
> gnome-terminal), and ran:
>
> sudo dnf install gcc-c++ libstdc++-devel.x86_64 libstdc++-devel.i686 libgcc.i686 glibc-devel.i686
>
>
>
> wget https://github.com/estraier/tkrzw/archive/0d7badb65abcac72381ecc6fbd8f980ce6a53305.zip
>
> unzip 0d7badb.zip
> cd tkrzw-0d7badb65abcac72381ecc6fbd8f980ce6a53305/
>
>
>
> CXXFLAGS=-m32 LDFLAGS=-m32 ./configure
> # Change the number of parallel jobs as desired; my VM had 4 CPUs and adequate memory
>
> make -j4
>
> make check
>
> After a while (my workstation is an antique) I get:
>
> make check-skipdbm-perf
>
> make[1]: Entering directory '/home/liveuser/tkrzw-0d7badb65abcac72381ecc6fbd8f980ce6a53305'
>
> rm -Rf casket*
>
> LD_LIBRARY_PATH=.:/usr/local/lib: ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks \
>
> --iter 20000 --threads 5 --size 8 --step_unit 4 --max_level 10
>
> Setting: impl=skip num_iterations=20000 value_size=8 num_threads=5
>
> .................................................. (00001000)
>
> .................................................. (00002000)
>
> .................................................. (00003000)
>
> .................................................. (00004000)
>
> .................................................. (00005000)
>
> .................................................. (00006000)
>
> .................................................. (00007000)
>
> .................................................. (00008000)
>
> .................................................. (00009000)
>
> .................................................. (00010000)
>
> .................................................. (00011000)
>
> .................................................. (00012000)
>
> .................................................. (00013000)
>
> .................................................. (00014000)
>
> .................................................. (00015000)
>
> .................................................. (00016000)
>
> .................................................. (00017000)
>
> .................................................. (00018000)
>
> .................................................. (00019000)
>
> .................................................. (00020000)
>
> Synchronizing: ... done (elapsed=1.172894)
>
> Setting done: elapsed_time=2.466667 num_records=100000 qps=40541 mem=8828000
>
> file_size=2033480 eff_data_size=1600000 efficiency=78.68%
>
>
>
> Getting: impl=skip num_iterations=20000 value_size=8 num_threads=5
>
> Get failed: BROKEN_DATA_ERROR: invalid record magic number
>
> Get failed: BROKEN_DATA_ERROR: invalid record magic number
>
> Get failed: BROKEN_DATA_ERROR: invalid record magic number
>
> Get failed: BROKEN_DATA_ERROR: invalid record magic numberGet failed: BROKEN_DATA_ERROR: invalid record magic number
>
>
>
> Getting done: elapsed_time=0.020106 num_records=100000 qps=4973680 mem=7232000
>
> file_size=2033480 eff_data_size=1600000 efficiency=78.68%
>
>
> So it seems that’s a good way to reproduce it.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#8 (comment)>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AQGJVRE2UZZLPWYEL6Z3SW3TLBSYXANCNFSM4WNDDYYQ>
> .
>
|
This issue could be reproducible on Ubuntu 20.01 x64 if -m32 is attached to the build command. Thanks so much. |
Fedora 32..35, RH8 - ok |
Seems nobody else will report this bug |
Due this log
make check
on x86 ends with:This is exactly i686 error. Other builds OK.
The text was updated successfully, but these errors were encountered: