Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only dump the changed ColumnFamilyOptions to the LOG #770

Open
wants to merge 349 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
349 commits
Select commit Hold shift + click to select a range
3fe5bdb
build: rename artefacts from RocksDB to Speedb (#64)
isaac-io Jul 25, 2022
f1fd855
scripts: rename RocksDB references to Speedb (#64)
isaac-io Nov 24, 2022
ffe4885
general: replace RocksDB references in strings with Speedb (#64)
isaac-io Jul 25, 2022
3af146b
tools: replace RocksDB version information with the Speedb version (#64)
isaac-io Jul 25, 2022
fa900e2
Added questions section to the README file
bosmatt Oct 26, 2022
349e24e
ci: fix the release pipeline
isaac-io Oct 26, 2022
6eb9b38
release: publish version 2.1.0
Oct 26, 2022
9c3b26b
cmake: clean up on successful runs and randomise test scheduling (#202)
isaac-io Oct 24, 2022
8cba38e
build: add a version build-tag for non-release builds (#156)
isaac-io Nov 25, 2022
1b1a724
build: support ccache and sccache in the Makefile build (#170)
isaac-io Sep 21, 2022
58246bf
memtable_list: avoid rolling back memtable flush on CF drop (#144)
isaac-io Aug 30, 2022
ab702c5
deletefile_test: fix breakage caused by the compaction threads change…
isaac-io Nov 7, 2022
d3e982c
docs: Update README.md (#212)
Yuval-Ariel Nov 25, 2022
f01f543
docs: readme typo fix (#223)
azmisaquib Nov 9, 2022
cc59224
docs: fix instructions for building Speedb in README.md and INSTALL.md
isaac-io Nov 10, 2022
90514aa
ci: build a release without a debug tag (#228)
isaac-io Nov 10, 2022
cd89bda
build: correctly handle merge commits when calculating a build tag (#…
isaac-io Nov 9, 2022
d4f99cb
db_test2: fix BackgroundPurgeTest (#236)
Yuval-Ariel Nov 11, 2022
a5fe261
docs: Update HISTORY.md README.md (#239)
bosmatt Nov 25, 2022
3c323bb
db_bench: Fix a bug when destructing a Benchmark with multiple db-s (…
udi-speedb Nov 10, 2022
9f0f8cc
hash spd memtable (#98)
ayulas Nov 14, 2022
cf06aba
Rebase 8.1.1: fix for hash spd memtable (#98)
Yuval-Ariel May 1, 2023
d8cd9ea
db_bench: add benchmark - seektodeletedranges (#200)
Yuval-Ariel Nov 14, 2022
2d35a96
db_bench: Make -statistics_string behave as -statistics (#248)
udi-speedb Nov 17, 2022
eb245b9
docs: Add db_bench README
bosmatt Nov 27, 2022
388e7bb
QA tool to check artifacts (#120)
RoyBenMoshe Nov 20, 2022
e7f3b2a
ci: added db_bench_README.txt to db_bench-speedb-V.tar.gz (#253)
maxb-io Nov 25, 2022
b5ebaaa
cmake: Fix build for rebase 7.7.3 (#264)
Yuval-Ariel Nov 25, 2022
b5712a8
java makefile: fix dependencies
Yuval-Ariel Nov 25, 2022
5ebb3ce
Fix check sources to ignore GIF files
mrambacher Nov 15, 2022
bd23caa
rebase 7.7: fix merge issues
Yuval-Ariel Nov 26, 2022
9c58025
db_bench: Enhancement - Support multiple benchmarks groups, each with…
udi-speedb Nov 27, 2022
9c9942c
Rebase 8.1.1: fix for db_bench: Enhancement - Support multiple benchm…
Yuval-Ariel May 1, 2023
43beb26
db_bench: ErrorExit from static func compiler error (#277)
Yuval-Ariel Apr 25, 2023
6604be1
db_bench: Support control over WBM's allow_stall ctor parameter (#187)
udi-speedb Nov 24, 2022
6d3c92a
db_bench: Enhancement - Allow specifying the db-s to use in each group
udi-speedb Nov 27, 2022
2ef83ea
db_bench: fix compiler warnings (#263)
Yuval-Ariel Nov 28, 2022
edaa74c
build: Add gtest-parallel to project
AmnonHanuhov Nov 22, 2022
dca1875
Update README.md (#280)
bosmatt Dec 5, 2022
747e4f0
Update README.md (#291)
bosmatt Dec 7, 2022
272c6ab
db_stress: Support control over WBM's allow_stall (#287)
udi-speedb Dec 7, 2022
479f1f6
db_bench: Support '--groups' in addition to '-groups' (#283)
udi-speedb Dec 14, 2022
624fd45
unit tests: disable CompactionServiceTest.RemoteEventListener (#300)
Yuval-Ariel Dec 18, 2022
bba6a28
generic switch memtable allows the ability to pre create memtable tha…
ayulas Dec 14, 2022
0dfe58a
Smaller batch size in the paired bloom filter (#161)
udi-speedb Dec 14, 2022
ded3aa9
misc: add build properties info to LOG file (#262)
RoyBenMoshe Dec 18, 2022
5a6c9c2
Dynamic Delay: new feature - Dynamic Delay (#276)
Yuval-Ariel Dec 19, 2022
cf427c9
Rebase 8.1.1: WriteStallCause moved to types.h
Yuval-Ariel Apr 30, 2023
5a53078
Proactive Flushes (#155)
udi-speedb Sep 29, 2022
2df1aab
Rebase 8.1.1: fix for Proactive flushes
Yuval-Ariel May 1, 2023
114b857
Proactive Flushes: compilation warnings fix (#304)
Yuval-Ariel Dec 21, 2022
21805dd
unit tests: disable LoadCustomizableTest.LoadMemTableRepFactoryTest (…
ayulas Dec 21, 2022
8eafc2c
Update artifact-release.yml
maxb-io Dec 21, 2022
1a37466
release: publish version 2.2.0
Dec 22, 2022
1995c9e
Delayed Writes: fix L0 calc bug (#310)
Yuval-Ariel Dec 25, 2022
05fc753
Print WBM's cache info to log (#312)
udi-speedb Dec 25, 2022
e114378
artifacts check tool: update readme file (#293)
RoyBenMoshe Jan 1, 2023
88010da
unit tests: fix CompactionServiceTest.RemoteEventListener (#314)
Yuval-Ariel Jan 6, 2023
7b4b77b
Update README.md
bosmatt Jan 8, 2023
482dd17
build: remove the dependency on GNU Parallel for running unit tests (…
AmnonHanuhov Nov 15, 2022
e3508a5
db_bench: set db_bench defaults to speedbs (#61)
Yuval-Ariel Jan 1, 2023
ed8320d
Spdb memtable: improve performance (#298)
ayulas Dec 18, 2022
26768f9
add speedb is awesome example to support new getting started instruct…
RoyBenMoshe Feb 1, 2023
5668e8e
db_bench: align used tests (#325)
Yuval-Ariel Jan 10, 2023
ccc4e1a
windows build failed in speedb_paired_bloom_internal files due to bad…
ayulas Feb 2, 2023
e8ca52f
unit_tests: Generate new db dir per test (#353)
mrambacher Feb 7, 2023
6724bee
MAC_build: Add F_BARRIERFSYNC (#319)
mrambacher Feb 7, 2023
ce39679
db_bench: Fix compilation issues on Mac (#393)
mrambacher Feb 8, 2023
eb8d27e
Add Injection and NoSyncFS (#380)
mrambacher Feb 9, 2023
fbec567
live config changes: add RefreshOptions method (#294)
mrambacher Feb 11, 2023
af39698
db_bench: fix for issue (#290) (#370)
andy-byers Feb 11, 2023
49030a9
Paired-Bloom-Filter: Balancing rounding to batches between the bottom…
noamhaham Feb 13, 2023
9f7f68c
AXV512: don't use XXH3_accumulate_512 with ASAN (#398)
Yuval-Ariel Feb 14, 2023
32557b0
Adapt paired bloom filter & test to cache-related changes in RocksDB
udi-speedb May 1, 2023
8de66a0
release: publish version 2.3.0
Feb 15, 2023
973a309
fault_injection_fs: Add back accidental revert in DropRandomUnsyncedD…
mrambacher Feb 15, 2023
147a52e
Add speedb licenses to code (#409)
ofriedma Feb 23, 2023
717596f
CI: Enforce writing licenses inside source files (#410)
ofriedma Feb 23, 2023
35c78db
Makefile: Use speedb libs in `build_size` target (#399)
AmnonHanuhov Feb 28, 2023
87c350d
ci job test_fuzz - removed saving logs to s3, it is not needed as the…
maxb-io Jan 30, 2023
267dd76
removed AWS credentials as they are not required, were used for uploa…
maxb-io Mar 1, 2023
acdac96
Global Write Controller: share write controller across dbs (#346)
Yuval-Ariel Feb 23, 2023
df515d5
added ci_macos.yml file for automation of tests and builds on MacOS (…
maxb-io Mar 19, 2023
66b75c3
added ci_windows.yml for automation of tests and build on Windows (#435)
maxb-io Mar 19, 2023
22031b8
added a step that copies thirdparty.inc from external dir to the spee…
maxb-io Mar 19, 2023
23e8586
Global delay bug fix: use credits only under mutex (#438)
Yuval-Ariel Mar 22, 2023
79c1f87
crashtest: dont reroll skip_list or HashSpdRepFactory (#452)
Yuval-Ariel Mar 29, 2023
173b86c
options: Forward declare WriteBufferManager (#279)
AmnonHanuhov Mar 29, 2023
3f137d4
CI windows: switched VC 2019 to VC 2022 in the ci_windows.yml in orde…
maxb-io Mar 29, 2023
be2a194
dynamic delay: fix pending bytes rate calculation (#451)
Yuval-Ariel Apr 2, 2023
878a97d
Write Flow: Spdb WF (#445)
ayulas Apr 3, 2023
ff53bc4
Rebase 8.1.1: fix for WF
Yuval-Ariel May 2, 2023
fa5eddf
TTL: Skip expired object when using DBWithTtl (#403)
ofriedma Apr 4, 2023
5c574dc
added readline-dev package to the alpine container in the CI test (#463)
maxb-io Apr 4, 2023
57e540c
adding beezcli and ttl support (#427)
ofriedma Apr 5, 2023
aea4a01
466 artifact build fails due to beezcli is missing (#469)
maxb-io May 2, 2023
46474b6
release: publish version 2.4.0
Apr 5, 2023
083326c
Write Flow: Reduce debug logs (#472)
ayulas Apr 17, 2023
7124d9e
Java FilterPolicy: Add Filter::CreateFromString to Java (#387)
mrambacher Apr 19, 2023
b088644
Rebase 8.1.1: reintroduce preserve_unverified_changes
Yuval-Ariel May 2, 2023
835f46c
Rebase 8.1.1: fix for speedb_db_bloom_filter_test
Yuval-Ariel May 3, 2023
bcb244d
Rebase 8.1.1: Remove leftover ROCKSDB_LITE
Yuval-Ariel Apr 30, 2023
929f6b4
Rebase 8.1.1: make format
Yuval-Ariel Apr 30, 2023
21ff6ca
Rebase 8.1.1: disable failing unit tests and paired
Yuval-Ariel May 4, 2023
6c96280
Rebase 8.1.1: fix db_bench
Yuval-Ariel May 4, 2023
57686e6
Rebase 8.1.1: ‘begin_key_id’ may be used uninitialized
Yuval-Ariel May 4, 2023
11ff28a
Rebase 8.1.1: fix narrow crash test
Yuval-Ariel May 11, 2023
f28f0d7
Rebase 8.1.1: CI fix sanity check to use clang-format 10
Yuval-Ariel May 4, 2023
2ffee5b
CI sanity check: run sanity only once on PRs
Yuval-Ariel May 4, 2023
0960fa5
Makefile: Remove pycache artifacts after running gtest-parallel (#495)
AmnonHanuhov May 11, 2023
be87eba
AVX512: fix disabling other optimizations (#489)
Yuval-Ariel May 11, 2023
b2698cc
version: update Speedb patch version to 2.4.1 (#503)
Yuval-Ariel May 11, 2023
9cee48e
stress test: fix decoding error (#498)
Yuval-Ariel May 11, 2023
49bd79a
ci: add a workflow for building and publishing jar to maven central (…
maxb-io May 15, 2023
2fc98a8
Update HISTORY.md (#508)
bosmatt May 16, 2023
3a8d9b9
db_bench and stress: fix WBM initiation (#510)
udi-speedb May 18, 2023
040e688
LOG: Compaction job traces - report cf name and job id (#511)
udi-speedb May 21, 2023
e7e2de7
db_stress: Add cost_write_buffer_to_cache flag (#513)
udi-speedb May 22, 2023
5a4cba3
update HISTORY.md (#525)
Yuval-Ariel May 23, 2023
5ac0b2e
Proactive Flushes: Sanitize max_num_parallel_flushes in WBM if 0 (#46…
udi-speedb May 23, 2023
7904f10
Update README.md (#527)
bosmatt May 23, 2023
df8962b
Log Improvement: Report the name of cf-s whose options are skipped in…
udi-speedb May 24, 2023
39e3e99
Log Improvement: Display cf names in rolled logs with their options (…
udi-speedb May 24, 2023
5bcecf0
WC: fix for stop while shutting down (#499)
Yuval-Ariel May 24, 2023
d0cdbbd
unit tests: Transaction test UnlockWal fix (#514)
Yuval-Ariel May 24, 2023
6ede30d
Dirty Memory: Connect WBM to the global delay (#423)
Yuval-Ariel May 26, 2023
4c034e1
db_bench: set Write Flow as false by default (#532)
Yuval-Ariel May 30, 2023
1788bc7
unit tests: Enable SpdbDBBloomFilterTest::OptimizeFiltersForHits (#538)
udi-speedb Jun 6, 2023
fc24b77
Always assume optimize_filters_for_memory=false when creating a paire…
udi-speedb Jun 7, 2023
cd136e1
fix ldb headline to be ldb and not beezcli https://github.com/speedb-…
ofriedma Jun 7, 2023
f39f76f
CI: new workflow - license and HISTORY check (#526)
Yuval-Ariel Jun 11, 2023
3d4c9d1
LOG: Print optimize_filters_for_memory option to the log (#537)
udi-speedb Jun 11, 2023
594f177
Prevent flush entry that has a delete entry (#411)
ayulas Jun 12, 2023
f3bdb69
Hashspd memtable: Memory bug fix( #553)
ayulas Jun 12, 2023
1bc17c8
db_bench: Create a WBM once for all db-s regardless of their use in d…
udi-speedb Jun 12, 2023
72d4181
Dirty Memory: Revert allow_stall naming from allow_delays_and_stalls …
Yuval-Ariel Jun 12, 2023
6bf9458
CI: sanity_check replace wei/wget action with real wget (#559)
maxb-io Jun 13, 2023
201de3d
Skip Flush Entry: Tombstone test fix for mac (#561)
ayulas Jun 14, 2023
041da08
release: publish version 2.5.0
Jun 14, 2023
1286d7f
Version Release 2.5.0: Update HISTORY.md(#567)
Yuval-Ariel Jun 15, 2023
a9e051b
misc: Replace SpeeDB with Speedb for uniformity (#564)
udi-speedb Jun 15, 2023
ea3bdd5
Windows compilation fix (#568)
Yuval-Ariel Jun 18, 2023
3ee5632
db_bench: Add table-readers-mem benchmark (#575)
Yuval-Ariel Jun 26, 2023
a81e3ec
Logger: Fix log trace of Block Cache Stats dump (#578)
udi-speedb Jun 26, 2023
a785634
unit tests: Fix GlobalAndWBMSetupDelay (#589)
Yuval-Ariel Jul 3, 2023
0e1971c
590 new runner ubuntu 2004 on arm64 (#591)
maxb-io Jul 3, 2023
f66ae81
switch to rocksdbjavastatic and portable for building on Ubuntu u86 b…
maxb-io Jul 3, 2023
7d5f079
593 build of libspeedbjni linux64so is dynamic needs to be static (#595)
maxb-io Jul 4, 2023
404d794
build: Remove unused variables in unit tests (#582)
RoyBenMoshe Jul 5, 2023
45713bb
604 missing dependencies in jar (#605)
maxb-io Jul 13, 2023
964a9b4
spdb memtable: fix use_seek_parallel_threshold handleding (#603)
ayulas Jul 18, 2023
e887846
switch memtable: fix for stuck switch with plug (#610)
ayulas Jul 24, 2023
e0d7d07
WriteController: move it to public interface (#579)
Yuval-Ariel Jul 25, 2023
a680523
613 workflow with a status check not triggered when a pr initiated fr…
maxb-io Jul 31, 2023
ba1ce97
613 workflow with a status check not triggered when a pr initiated fr…
maxb-io Jul 31, 2023
6dd7279
Add a SerializeOptions method to Configurable
mrambacher Jul 31, 2023
983f42f
First Pass at serializing only options that are changed
mrambacher Aug 1, 2023
8ed9b68
unit tests: fix DisableMultiManualCompaction in db_compaction_test (#…
Yuval-Ariel Aug 2, 2023
6e05428
Clean Memory: Introduce a TablePinningPolicy to control how and when …
mrambacher Aug 2, 2023
ff1b086
Snapshot Optimization (#547)
ofriedma Aug 2, 2023
335c424
Unit tests: Enable DBIOFailureTest.DropWrites (#544)
mrambacher Aug 3, 2023
666ef17
removed check for github.event_name before starting each step, this w…
maxb-io Aug 6, 2023
7959837
621 check history and license on pr (#626)
maxb-io Aug 6, 2023
846dd2a
Revert "First Pass at serializing only options that are changed"
mrambacher Aug 8, 2023
8df931c
More use of SerializeOptions
mrambacher Aug 8, 2023
2075487
Enable Speedb features (#543)
RoyBenMoshe Aug 14, 2023
a53ad27
CI: fix check license by comparing the right sha (#638)
Yuval-Ariel Aug 16, 2023
eb8e466
Redo of Index/Filter/Data blocks sizes in Block (LRU) Block Cache per…
udi-speedb Aug 17, 2023
1751c4f
hashSpdb memtable: move it from plugin to main code (#640)
ayulas Aug 18, 2023
54bd545
Spdb memtable: Add support in Java and C (#612)
ayulas Aug 19, 2023
3e55c3d
Proactive Flushes: Decrement number of flushes to init on non-wbm-ini…
udi-speedb Aug 19, 2023
13af75a
Spdb memtable: Fix typo in memtableTest in java (#647)
ayulas Aug 20, 2023
a6e1ab2
Add on thread start callback (#629)
ofriedma Aug 20, 2023
944f9ba
Makefile: upgrade Zlib to 1.3 (#646)
ayulas Aug 21, 2023
ea70d40
release: publish version 2.6.0
Aug 22, 2023
8cec611
Add an OptionsFormatter class
mrambacher Aug 25, 2023
22f6398
Add ToMap function to OptionsFormatter
mrambacher Aug 28, 2023
51ab6a1
Add ToVector to OptionsFormatter
mrambacher Aug 28, 2023
b5e5c31
Eliminate StringToMap function
mrambacher Aug 28, 2023
8a8e3cd
Move NextToken into OptionsFormatter
mrambacher Aug 28, 2023
2357795
Update convenience.h
mrambacher Aug 28, 2023
a53332d
Update HISTORY.md for 2.6.0 (#654)
Yuval-Ariel Aug 29, 2023
4209cf5
db_bench: fix SeekRandomWriteRandom valid check (#652)
Yuval-Ariel Aug 29, 2023
b373e4f
Fix unnecessary (extra) call
mrambacher Sep 1, 2023
a7fb784
Merge remote-tracking branch 'upstream/main' into ConfigureFromVector
mrambacher Sep 4, 2023
9f60375
Fix repeatable threads not working with on thread start feature (#667)
ofriedma Sep 7, 2023
4125480
Build: Add support for ASAN builds on Mac (#608)
mrambacher Sep 18, 2023
19ad8a8
Build: Fix unused variable warnings in db_bench_tool.cc (#671)
git-hulk Sep 18, 2023
ea3da8e
Support Non-Blocking Manual Compactions (CompactRange) (#597) (#656)
udi-speedb Sep 18, 2023
b0ccf23
Java build: Fix for build issue introduced by #597 (#681)
udi-speedb Sep 21, 2023
fc2bee8
Unit tests: Expose the disallow_trivial_move flag in MoveFilesToLevel…
udi-speedb Sep 21, 2023
ffb63be
Add MacOS support for thread affinity tests (#661)
ofriedma Sep 28, 2023
40369db
unit tests: disable failing SeqnoToTimeMappingUniversal test (#694)
Yuval-Ariel Sep 28, 2023
61d89e2
700 switch to GitHub shared runner mac x86 (#703)
maxb-io Oct 4, 2023
e982951
695 mvn publish switch to new runner (#696)
maxb-io Oct 4, 2023
f909a3b
700 switch to GitHub shared runner mac and win x86 (#705)
maxb-io Oct 5, 2023
20a6593
docs: update the license for files we've updated (#698)
Yuval-Ariel Oct 5, 2023
1698f4e
Merge remote-tracking branch 'upstream/main' into DumpOptions
mrambacher Oct 6, 2023
c84e92a
Logger: Add reporting capabilities to the WBM (#556)
Yuval-Ariel Oct 10, 2023
5e4e147
706 move ci tests to gh shared runners (#710)
maxb-io Oct 11, 2023
858bb1a
Static Pinning: Report static pinning options to the log (#707)
udi-speedb Oct 12, 2023
1058aa6
CI: Verify updates to HISTORY.md on all file changes (#712)
Yuval-Ariel Oct 12, 2023
125dc8f
Add class comment
mrambacher Oct 13, 2023
ab1b292
Update license/copyright info
mrambacher Oct 13, 2023
70392c9
Merge remote-tracking branch 'upstream/main' into ConfigureFromVector
mrambacher Oct 13, 2023
7a80d1c
Add prefix to Serialize; Add Properties typedef
mrambacher Oct 14, 2023
bf004d9
Merge branch 'ConfigureFromVector' into DumpOptions
mrambacher Oct 14, 2023
ffa0162
Change to use Properties
mrambacher Oct 14, 2023
6511a7d
Add the offset and size to the dump SST footer details (#679)
git-hulk Oct 15, 2023
877f2a5
Fix some build issues (#664)
mrambacher Oct 15, 2023
bedf640
Unit tests: Disable stuck tests in db_compaction_test (#711)
Yuval-Ariel Oct 15, 2023
06b8e4c
Build: Fix db_bench unused variables (#717)
ofriedma Oct 15, 2023
4892a77
CI: Fix check history, add fetch-depth 0 for base sha (#718)
Yuval-Ariel Oct 15, 2023
24ea275
Add ability to print non-standard options to log and other files
mrambacher Oct 15, 2023
bbcc1c7
Remove opps
mrambacher Oct 15, 2023
d1a8508
Merge remote-tracking branch 'upstream/main' into LogOptions
mrambacher Oct 15, 2023
371c331
Remove backup files
mrambacher Oct 15, 2023
b1db8ae
Java RocksDBSample: Fix Speedb memtable check (#721)
mrambacher Oct 17, 2023
5bf9c60
Enable-Speedb-Feature: Use Scoped Pinning Policy instead of Default P…
udi-speedb Oct 17, 2023
ab9df1a
Make a static pinning decision based on the last level with data (not…
udi-speedb Oct 17, 2023
f39b8b7
Support hash spdb as part of enable speedb features (#653) (#655)
ayulas Oct 17, 2023
9e2b4d3
Fix a compilation error in scoped_pinning_policy.cc (#725)
udi-speedb Oct 18, 2023
dc39061
Change Configurable::SerializeOptions method to return a map (#619)
mrambacher Oct 19, 2023
8c68304
db bench: usability of the pinning policy parameter (#720)
udi-speedb Oct 19, 2023
563d8a3
Static Pinning: Set the default for last-level-with-data to 70 (#727)
udi-speedb Oct 21, 2023
c09402c
db_bench: Ignore pinning policy when enabling speedb features (#730)
udi-speedb Oct 22, 2023
0264ab9
release: publish version 2.7.0
Oct 25, 2023
34e1a7a
Serialize only options that have changed from the some default (#648)
mrambacher Oct 31, 2023
9f310e8
Add useBaseAddress to OptionTypeInfo (#397)
mrambacher Nov 1, 2023
4c5ce06
Stall deadlock consists small cfs (#737)
ayulas Nov 1, 2023
3babc5c
Update HISTORY.md for 2.7.0 (#746)
udi-speedb Nov 6, 2023
3e56585
747 ci check license and history allow manual workflow executoin (#748)
maxb-io Nov 6, 2023
e6cbe59
Documentation: Add the request to add Speedb's license when modifying…
udi-speedb Nov 6, 2023
f1f04db
Enable speedb features: Constrain the i/f of SharedOptions (make immu…
udi-speedb Nov 7, 2023
3600ba6
removed filter for .md files in history check (#754)
maxb-io Nov 7, 2023
d1a4a69
Expose Options::periodic_compaction_seconds via C API (#741)
zaidoon1 Nov 8, 2023
d6b7c34
Remove leftover references to ROCKSDB_LITE (#762)
udi-speedb Nov 10, 2023
f537bcd
Proactive Flushes: Fix a race in the ShouldInitiateAnotherFlushMemOnl…
udi-speedb Nov 10, 2023
b04d9eb
support-enable-speedb-features-in-ci:723 (#726)
ayulas Nov 12, 2023
1c6a506
Add OptionsFormatter class and update to allow auto-logging of Option…
mrambacher Nov 15, 2023
d47a747
Merge remote-tracking branch 'upstream/main' into LogOptions
mrambacher Nov 15, 2023
3ca75c4
Fix issues after merge
mrambacher Nov 15, 2023
5080859
More work on LogOptionsFormatter
mrambacher Nov 17, 2023
8e7e363
Make Cache use SerializeOptions et al
mrambacher Nov 18, 2023
cf23722
716 add build tests for macos and windows to ci pipeline (#764)
maxb-io Nov 19, 2023
0d12087
More formatter changes
mrambacher Nov 19, 2023
dfaefb3
Dump only changed ColumnFamilyOptions to the LOG
mrambacher Nov 19, 2023
1bb0283
Remove accidental checkin
mrambacher Nov 19, 2023
1455983
Merge remote-tracking branch 'upstream/main' into LogOptions
mrambacher Nov 19, 2023
00292e9
Merge branch 'LogOptions' into OnlyChanged
mrambacher Nov 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
28 changes: 28 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:
1.
2.
3.
4.

**Expected behavior**
A clear and concise description of what you expected to happen.

**System (please complete the following information):**
- OS: [e.g. RHEL8.6]
- Hardware [e.g. Intel Xeon Ice Lake, 64GB, NVMe]

**Additional context**
Add any other context about the problem here.
22 changes: 22 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''

---

Owner:

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.
Binary file added .github/new-bee-mascot-dark.gif
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/new-bee-mascot.gif
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/speedb-b.gif
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/speedb-logo-dark.gif
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/speedb-logo.gif
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
113 changes: 113 additions & 0 deletions .github/workflows/artifact-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
name: Create release artifacts

on:
push:
tags:
- 'speedb/v*'

permissions:
contents: write # Needed for release assets upload
id-token: write # Needed for AWS credentials setting

jobs:
build:
runs-on: [self-hosted, ubuntu, asrunner]

container:
image: centos:7.9.2009

steps:
- name: pre
run: |
yum install -y centos-release-scl epel-release
yum install -y make devtoolset-11-gcc-c++ \
coreutils wget unzip which git python3 openssl openssl-devel \
libzstd-devel lz4-devel snappy-devel zlib-devel readline-devel \
java-1.8.0-openjdk-devel
echo "PATH=/opt/rh/devtoolset-11/root/usr/bin:${PATH}" >> $GITHUB_ENV
echo "RELEASE_VERSION=${GITHUB_REF_NAME#speedb/v}" >> $GITHUB_ENV

- name: Install CMake
run: |
CMAKE_RELEASE=3.20.1
wget https://github.com/Kitware/CMake/releases/download/v${CMAKE_RELEASE}/cmake-${CMAKE_RELEASE}.tar.gz
tar xf cmake-${CMAKE_RELEASE}.tar.gz
cd cmake-${CMAKE_RELEASE}
./bootstrap
make -j$(nproc) && make install
cd .. && rm -rf cmake-${CMAKE_RELEASE}*

- name: Install awscli
run: |
wget "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -O "awscliv2.zip"
unzip awscliv2.zip
./aws/install
rm -rf aws awscliv2.zip

- uses: actions/checkout@v3

- run: mkdir "$GITHUB_WORKSPACE/out"

- name: Build and package release libraries
run: |
rm -rf build && mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DSPDB_RELEASE_BUILD=1 -DPORTABLE=1 -DWITH_GFLAGS=0 -DWITH_SNAPPY=1 -DWITH_LZ4=1 -DWITH_ZLIB=1 -DWITH_ZSTD=1
mkdir -p "$GITHUB_WORKSPACE/out/root"
DESTDIR="$GITHUB_WORKSPACE/out/root" make -j$(nproc) install
( cd "$GITHUB_WORKSPACE/out/root" && tar czf ../speedb-${RELEASE_VERSION}.tar.gz . )
rm -rf "$GITHUB_WORKSPACE/out/root"
cd .. && rm -rf build

- name: Build release Jar
run: |
make clean
SPDB_RELEASE_BUILD=1 LIB_MODE=static DEBUG_LEVEL=0 PORTABLE=1 JAVA_HOME=/usr/lib/jvm/java-openjdk make -j$(nproc) rocksdbjavastatic
cp "java/target/speedbjni-${RELEASE_VERSION}-linux64.jar" "$GITHUB_WORKSPACE/out"

- name: Build db_bench
run: |
yum install -y gflags-devel
rm -rf build && mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DSPDB_RELEASE_BUILD=1 -DPORTABLE=1 -DWITH_GFLAGS=1 \
-DWITH_SNAPPY=1 -DWITH_LZ4=1 -DWITH_ZLIB=1 -DWITH_ZSTD=1 \
-DWITH_BENCHMARK_TOOLS=1 -DROCKSDB_BUILD_SHARED=1
make -j$(nproc) db_bench
cp ../docs/db_bench_README.txt .
tar czf "$GITHUB_WORKSPACE/out/db_bench-speedb-${RELEASE_VERSION}.tar.gz" db_bench db_bench_README.txt
cd .. && rm -rf build

- name: Generate checksums
run: |
for f in $GITHUB_WORKSPACE/out/*; do
sha256sum "$f" > "$f.sha256"
done

- name: Get release date
run: |
echo "RELEASE_DATE=$(git for-each-ref "--format=%(creatordate:short)" "refs/tags/${GITHUB_REF_NAME}")" >> $GITHUB_ENV

- name: Release
uses: softprops/action-gh-release@v1
if: startsWith(github.ref, 'refs/tags/')
with:
generate_release_notes: false
name: Speedb ${{ env.RELEASE_VERSION }} (${{ env.RELEASE_DATE }})
files: |
out/db_bench-speedb-${{ env.RELEASE_VERSION }}.tar.gz
out/db_bench-speedb-${{ env.RELEASE_VERSION }}.tar.gz.sha256
out/speedb-${{ env.RELEASE_VERSION }}.tar.gz
out/speedb-${{ env.RELEASE_VERSION }}.tar.gz.sha256
out/speedbjni-${{ env.RELEASE_VERSION }}-linux64.jar
out/speedbjni-${{ env.RELEASE_VERSION }}-linux64.jar.sha256

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: eu-west-2

- name: Upload artifacts to S3
run: |
aws s3 cp "$GITHUB_WORKSPACE/out" "s3://spdb-github-artifacts/release-${RELEASE_VERSION}" --recursive
rm -rf "$GITHUB_WORKSPACE/out"
115 changes: 115 additions & 0 deletions .github/workflows/build_and_publish_jar.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# This workflow will build Speedb library on Mac i86 and ARM, Ubuntu i86 and Arm, Windows i86. Then build a jar and publish to Maven central
#

name: build all and publish jar

on:
workflow_dispatch:

jobs:
pre_build:
runs-on: ubu-mvn-g
env:
VERSION_FILE: speedb/version.h
outputs:
out1: ${{ steps.find_version.outputs.verSion }}

steps:
- name: 'Cleanup build folder'
run: |
ls -la ./
rm -rf ./* || true
rm -rf ./.??* || true
ls -la ./

- name: 'Checkout GitHub Action'
uses: actions/checkout@v3

- name: 'find_version'
id: 'find_version'
run: |
major=$(grep '_MAJOR\s\+[0-9]\+' "$VERSION_FILE" | sed 's/[^0-9]\+//') && echo $major
minor=$(grep '_MINOR\s\+[0-9]\+' "$VERSION_FILE" | sed 's/[^0-9]\+//') && echo $minor
#patch=$(( $(grep '_PATCH\s\+[0-9]\+' "$VERSION_FILE" | sed 's/[^0-9]\+//') + 1 )) && echo $patch
patch=$(( $(grep '_PATCH\s\+[0-9]\+' "$VERSION_FILE" | sed 's/[^0-9]\+//'))) && echo $patch
echo "verSion=$major.$minor.$patch" >> $GITHUB_OUTPUT

Mac_i86:
needs: pre_build
uses: ./.github/workflows/build_macos.yml
with:
verSion: ${{ needs.pre_build.outputs.out1 }}
secrets:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_BUCKET: ${{ secrets.AWS_BUCKET }}

Mac_ARM:
needs: pre_build
uses: ./.github/workflows/build_macos_ARM.yml
with:
verSion: ${{ needs.pre_build.outputs.out1 }}

Ubuntu_ARM:
needs: pre_build
uses: ./.github/workflows/build_ubuntu_arm.yml
with:
verSion: ${{ needs.pre_build.outputs.out1 }}

Windows:
needs: pre_build
uses: ./.github/workflows/build_windows.yml
with:
verSion: ${{ needs.pre_build.outputs.out1 }}
secrets:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_BUCKET: ${{ secrets.AWS_BUCKET }}

Build_and_upload:
needs: [pre_build, Mac_i86, Windows, Ubuntu_ARM]
runs-on: ubu-mvn-g
env:
VERSION_FILE: speedb/version.h
VERSION: ${{needs.pre_build.outputs.out1}}
outputs:
out1: ${{ steps.find_version.outputs.verSion }}

steps:
- name: 'Cleanup build folder'
run: |
ls -la ./
rm -rf ./* || true
rm -rf ./.??* || true
ls -la ./

- name: 'Checkout GitHub Action'
uses: actions/checkout@v3

- name: 'build'
run: |
export JAVA_HOME="$(jrunscript -e 'java.lang.System.out.println(java.lang.System.getProperty("java.home"));')"
export LIB_JAVA_VERSION=11.0.17
export the_version=${{ steps.find_version.outputs.verSion }}
export SPDB_LIB_DIR=~/spdb_lib && mkdir -p $SPDB_LIB_DIR
docker run --rm -v $(readlink -f ${SPDB_LIB_DIR}):/out -i speedb-centos-builder ${{ github.ref_name }}
cd java
mkdir src/main/resources
cp $SPDB_LIB_DIR/libspeedbjni-linux64.so src/main/resources
mv $SPDB_LIB_DIR/libspeedbjni-linux64.so{,_$(date '+%d_%m_%Y__%H_%M_%S')}
echo "aws s3 --profile nd7 cp --recursive s3://spdb-builder/jar_test/v$VERSION/ java/src/main/resources/"
sleep 180
aws s3 --profile nd7 cp --recursive s3://spdb-builder/jar_test/v$VERSION/ src/main/resources/
ls -l src/main/resources/
cp ../../../../../templ/pom.xml .
mvn versions:set -DnewVersion=$VERSION-SNAPSHOT
mvn deploy -X -e -DskipTests

#mvn versions:set -DnewVersion=$the_version
#mvn clean deploy -P release -X -e -DskipTests

- name: show next step
run: |
echo "Make sure the SNAPSHOT is fine and run these to publish the artifact"
echo "run: mvn versions:set -DnewVersion=$the_version"
echo "run: mvn clean deploy -P release -X -e -DskipTests"
50 changes: 50 additions & 0 deletions .github/workflows/build_macos.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: Build on Mac

on:
workflow_call:
inputs:
verSion:
required: true
type: string
secrets:
AWS_ACCESS_KEY_ID:
required: true
AWS_SECRET_ACCESS_KEY:
required: true
AWS_BUCKET:
required: true
workflow_dispatch:

jobs:
build-mac:
runs-on: macos-11

steps:
- name: 'Checkout GitHub Action'
uses: actions/checkout@v3
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
distribution: 'temurin' # See 'Supported distributions' for available options.
java-version: '8'
- name: build jar
run: |
echo $JAVA_HOME
export CPPFLAGS="-I$JAVA_HOME/include"
export CXXFLAGS="-I$JAVA_HOME/include"
brew install zlib
brew install bzip2 lz4 snappy
ROCKSDB_DISABLE_JEMALLOC=1 PORTABLE=1 DEBUG_LEVEL=0 make -j 2 rocksdbjavastatic #more CPU cores makes shared github runner to overload cpu and stop

- name: Upload artifacts to S3
if: inputs.verSion != ' '
uses: NotCoffee418/s3-zip-upload@v1
env:
AWS_SECRET_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_BUCKET: ${{ secrets.AWS_BUCKET }}
BUCKET_NAME: spdb-builder
AWS_REGION: us-east-1
SOURCE_MODE: FILE
SOURCE_PATH: java/target/libspeedbjni-osx-x86_64.jnilib
DEST_FILE: jar_test/v${{ inputs.verSion }}/libspeedbjni-osx-x86_64.jnilib
37 changes: 37 additions & 0 deletions .github/workflows/build_macos_ARM.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# This workflow will build Speedb on a Mac OS server
#
# This workflow assumes the self hosted runner on a Mac machine is ready.
# The Mac OS server must have all the tools and software required for Speedb building to be installed


name: ARM - Build on Mac

on:
workflow_call:
inputs:
verSion:
required: true
type: string
workflow_dispatch:

jobs:
build-and-deploy:
runs-on: ubuntu-latest # will be changed to a MAC when it will be available
steps:
- name: 'Checkout GitHub Action'
uses: actions/checkout@v3

- name: 'build'
run: |
echo "the built library java/target/libspeedbjni-osx-x86_64.jnilib needs to be uploaded to the folloing location"
echo "java/target/libspeedbjni-osx-arm64.jnilib s3://spdb-builder/jar_test/v${{ inputs.verSion }}/libspeedbjni-osx-arm64.jnilib"

# export JAVA_HOME=`/usr/libexec/java_home -v 11`
# export CPPFLAGS="-I$JAVA_HOME/include"
# export CXXFLAGS="-I$JAVA_HOME/include"
# ROCKSDB_DISABLE_JEMALLOC=1 PORTABLE=1 DEBUG_LEVEL=0 make -j 4 rocksdbjavastatic

- name: 'upload artifacts' #This step executed only when this workflow is called by another and a version is provided
if: inputs.verSion != ' '
run: echo "nothing to do here until the real environment is ready"
#run: aws s3 cp java/target/libspeedbjni-osx-x86_64.jnilib s3://spdb-builder/jar_test/v${{ inputs.verSion }}/libspeedbjni-osx-arm64.jnilib