Check not pass

[tieugene]

tkrzw-0.9.3.logs.tar.gz
Fedora 33 x64.
Log files:

• autoreconf.err.log - stderr of autoreconf -vif
• configure.diff - diff from original configure and autoreconf-made (not affects on result)
• configure.out.log - stderr of ./configure
• make.out.log - stdout of make
  make_check.err.log - stderr of make check
  make_check.out.log - stdout of make check
  make_check.log - stderr&stdout of make check

[tieugene]

Seems rebuild operation and --rebuild option cause errors.
As a --lock_mem_buckets option.

[estraier]

Hi, The error seems to come from the user limit of memory locking. Please check it by this command. $ ulimit -l You can loosen the limit by this, then please run "make check". $ ulimit -S -l 1024

…

On Tue, Jan 19, 2021 at 5:05 PM Eugene ***@***.***> wrote: Seems rebuild operation and --rebuild option cause errors. As a --lock_mem_buckets option. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQGJVRCXGFYZ7BM3ZVVLVJTS2U4MFANCNFSM4WHYLSCQ> .

[tieugene]

As for my home host it is 64. But anyway I cannot change this value in Fedora project’s packaging «build fabric». And in other’s users hosts.

…

19 янв. 2021 г., в 19:28, Mikio Hirabayashi ***@***.***> написал(а): Hi, The error seems to come from the user limit of memory locking. Please check it by this command. $ ulimit -l You can loosen the limit by this, then please run "make check". $ ulimit -S -l 1024 On Tue, Jan 19, 2021 at 5:05 PM Eugene ***@***.***> wrote: > Seems rebuild operation and --rebuild option cause errors. > As a --lock_mem_buckets option. > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#5 (comment)>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AQGJVRCXGFYZ7BM3ZVVLVJTS2U4MFANCNFSM4WHYLSCQ> > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALTYSYSBAABXWTGAH75QKTS2WXMBANCNFSM4WHYLSCQ>.

[estraier]

Is the hard limit 64?
The hard limit cannot be changed but the soft limit can.
Isn't it possible to write the test script like this?:
$ ulimit -S -l 1024 ; make check

[tieugene]

ulimit -S -l 1024; make check … LD_LIBRARY_PATH=.:/usr/local/lib: ./tkrzw_dbm_perf sequence --dbm hash --file mmap-para --path casket.tkh \ --iter 20000 --threads 5 --size 8 --buckets 100000 --lock_mem_buckets OpenAdvanced failed: SYSTEM_ERROR: mlock: no enough memory

…

19 янв. 2021 г., в 19:41, Mikio Hirabayashi ***@***.***> написал(а): Is the hard limit 64? The hard limit cannot be changed but the soft limit can. Isn't it possible to write the test script like this?: $ ulimit -S -l 1024 ; make check — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALTYS7UGIJRZ3OE4KCUXNTS2WYZ3ANCNFSM4WHYLSCQ>.

[estraier]

What is printed when you run this?
$ ulimit -S -l 1024 ; ulimit -l

[tieugene]

bash-5.0$ LC_ALL=C ulimit -S -l 1024 bash: ulimit: max locked memory: cannot modify limit: Invalid argument PS. BTW - this will be interesting for you: https://bugzilla.redhat.com/show_bug.cgi?id=1914292 <https://bugzilla.redhat.com/show_bug.cgi?id=1914292> Process is continuing, we make everything to bump tkrzw into RHEL, but as you can see there some stones on this way.

…

19 янв. 2021 г., в 20:34, Mikio Hirabayashi ***@***.***> написал(а): What is printed when you run this? $ ulimit -S -l 1024 ; ulimit -l — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALTYS6Z7JORAC3LCLOXKMTS2W7DHANCNFSM4WHYLSCQ>.

[estraier]

Then, there's no way to test the memory locking feature on your environment.
Skipping the whole make check or patching Makefile.in to omit tests related to memory lock seems a practical work around.

[tieugene]

patching Makefile.in to omit tests related to memory lock seems a practical work around.

May be good idea to switch those tests off by default and return back with special configure option?
Like --enable-havyload

[estraier]

I committed the CL to suppress the test flags for mlock if there's not enough allowed capacity.

[tieugene]

Now tkrzw is packaging for RHEL8/Fedora 32..35 without any patch, with all available tests and skipping unavailable (like this).
Thank you.

[tieugene]

0.9.14 - problems again.
Last lines:

Setting done: elapsed_time=0.593525 num_records=100000 qps=168485 mem=22812000
  file_size=15224832 eff_data_size=1600000 efficiency=10.51%
Getting: impl=tree num_iterations=20000 value_size=8 num_threads=5
.................................................. (00001000)
...
.................................................. (00020000)
Getting done: elapsed_time=0.114678 num_records=100000 qps=872008 mem=23272000
  file_size=15224832 eff_data_size=1600000 efficiency=10.51%
Removing: impl=tree num_iterations=20000 value_size=8 num_threads=5
make[1]: Leaving directory '/builddir/build/BUILD/tkrzw-0.9.14'
RPM build errors:
make[1]: *** [Makefile:279: check-treedbm-perf] Aborted (core dumped)

[estraier]

That seems a different problem from the --lock_mem_buckets option.
Could you tell me the command which crushed and the environment?

[tieugene]

Could you tell me the command which crushed and the environment?

I don't build manually - just 'autoreconf && configure && make && make check'.
It's ok in localhost but problems are at virtual hosts that build packages for Fedora/CentOS.
And error appears randomly from build to build without any changes in build process.
Two sequential builds from one source in one OS (Fedora 34):
https://koji.fedoraproject.org/koji/taskinfo?taskID=67707362
https://koji.fedoraproject.org/koji/taskinfo?taskID=67750933

[estraier]

One reason is that the direct I/O feature is not supported on every platform.
Thus, I enable the test cases for direct I/O only if --enable-devel is set.
I committed the change.

Another reason seems related to TreeDBM.
According to one of your log files, the following command crashed somehow.
./tkrzw_dbm_perf sequence --dbm tree --file mmap-para --path casket.tkt --iter 20000 --threads 5 --size 8 --max_page_size 200 --max_branches 8

It's not reproduced on my environment so I need to investigate more.