cargo chainhooks-install failed due to rocksdb build error

[qustavo]

When trying to build chainhooks, I'm getting the following error:

Compiling librocksdb-sys v0.10.0+7.9.2
The following warnings were emitted during compilation:

warning: In file included from rocksdb/table/block_based/data_block_hash_index.cc:5:
warning: rocksdb/table/block_based/data_block_hash_index.h:65:7: error: ‘uint8_t’ does not name a type
warning:    65 | const uint8_t kNoEntry = 255;

[...]

 error occurred: Command "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.8.1/fused-src/" "-I" "snappy/" "-I" "/home/gchain/dev/chainhook/target/release/build/lz4-sys-5dba30cef2dbb318/out/include" "-I" "." "-Wall" "-Wextra" "-std=c++17" "-Wsign-compare" "-Wshadow" "-Wno-unused-parameter" "-Wno-unused-variable" "-Woverloaded-virtual" "-Wnon-virtual-dtor" "-Wno-missing-field-initializers" "-Wno-strict-aliasing" "-Wno-invalid-offsetof" "-msse2" "-std=c++17" "-DSNAPPY=1" "-DLZ4=1" "-DNDEBUG=1" "-DOS_LINUX" "-DROCKSDB_PLATFORM_POSIX" "-DROCKSDB_LIB_IO_POSIX" "-DROCKSDB_SUPPORT_THREAD_LOCAL" "-o" "/home/gchain/dev/chainhook/target/release/build/librocksdb-sys-3d8700161d2f94b8/out/rocksdb/table/block_based/data_block_hash_index.o" "-c" "rocksdb/table/block_based/data_block_hash_index.cc" with args "c++" did not execute successfully (status code exit status: 1).

error: failed to compile `chainhook v1.0.0 (/home/me/dev/chainhook/components/chainhook-cli)`, intermediate artifacts can be found at `/home/me/dev/chainhook/target`

I fixed it by bumping rocksdb dependency version:

diff --git a/Cargo.lock b/Cargo.lock
index 9338a06..ad5ef67 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -416,7 +416,7 @@ checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"
 
 [[package]]
 name = "chainhook"
-version = "0.20.0"
+version = "1.0.0"
 dependencies = [
  "ansi_term",
  "atty",
diff --git a/components/chainhook-cli/Cargo.toml b/components/chainhook-cli/Cargo.toml
index 3c10818..18b38ae 100644
--- a/components/chainhook-cli/Cargo.toml
+++ b/components/chainhook-cli/Cargo.toml
@@ -41,7 +41,7 @@ rocket_okapi = { version = "0.8.0-rc.3", git = "https://github.com/hirosystems/o
 rocket = { version = "=0.5.0-rc.3", features = ["json"] }
 
 [dependencies.rocksdb]
-version = "0.20.1"
+version = "0.21.0"
 default-features = false
 features = ["lz4", "snappy"]

I'd create a PR to bump the version but I've just seen that rocksdb has been downgraded from 0.21.0 to 0.20.1 (0e88b53).

Any chance to revert that commit?

[lgalabru]

hey @qustavo!
thanks for opening this PR. The last time I tried updating rocksdb, I did experience a light regression in terms of performance.
i was testing a few different things simultaneously so it could have been something else.
this is not something that I can personally investigate, if you can, any help would be welcome!

[qustavo]

To give a little bit more context, my issue has been previously described here due to gcc 13.x issue. The suggested fix is described here.
I managed to switch rocksdb versions setting a ROCKSDB_LIB_DIR and using precompiled .so.

I'd like to know how did you benchmarked and detected the performance regressions, do you have some tests I can run and measure?

[SrushtiHaryan]

after cargo chainhook-install, when i run the chainhook command, the output remains empty, can anyone tell what could have gone wrong while setting up the repo locally?

[MicaiahReid]

@SrushtiHaryan were there any errors when you ran cargo chainhook-install? And to clarify, when you now run chainhook, there is no result at all? No error, nothing?

[SrushtiHaryan]

@SrushtiHaryan were there any errors when you ran cargo chainhook-install? And to clarify, when you now run chainhook, there is no result at all? No error, nothing?

yes there were errors,

 process didn't exit successfully: `C:\Srushti\Chainhook\chainhook\target\release\build\librocksdb-sys-b75e8e5ac7d7572f\build-script-build` (exit code: 101)
  --- stderr
  thread 'main' panicked at C:\Users\manav\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bindgen-0.64.0\./lib.rs:2393:31:
  Unable to find libclang: "couldn't find any valid shared libraries matching: ['clang.dll', 'libclang.dll'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: failed to compile `chainhook v1.1.1 (C:\Srushti\Chainhook\chainhook\components\chainhook-cli)`, intermediate artifacts can be found at `C:\Srushti\Chainhook\chainhook\target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

i installed clang and followed this issue for changing the dependency version:
i tried to change the dependency versions of:

1. Chainhook from v1.1.1 to v1.0.0 in Cargo.lock
2. rocksdb from 0.20.1 to v0.21.0 in components/chainhook-cli/Cargo.toml

after which the packages got installed without an error (but a few warnings) on cargo chainhook-install

Finished release [optimized] target(s) in 11m 45s
warning: the following packages contain code that will be rejected by a future version of Rust: rstest v0.11.0
note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
  Replacing C:\Users\manav\.cargo\bin\chainhook.exe
   Replaced package `chainhook v1.1.1 (C:\Srushti\Chainhook\chainhook\components\chainhook-cli)` with `chainhook v1.0.0 

and just to clarify on the output, yes, it doesnt give errors too.

[radicleart]

Hey - I just tried cargo chainhook-install on Ubuntu 23.10 and end up with error ;

error: failed to run custom build command for `librocksdb-sys v0.10.0+7.9.2`

Caused by:
  process didn't exit successfully: `/mnt/bitcoin-mainnet/chainhook/target/release/build/librocksdb-sys-a85928e817e4688b/build-script-build` (exit status: 1)

followed by

  error occurred: Command "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.8.1/fused-src/" "-I" "snappy/" "-I" "/mnt/bitcoin-mainnet/chainhook/target/release/build/lz4-sys-2c47c50287c793be/out/include" "-I" "." "-Wall" "-Wextra" "-std=c++17" "-Wsign-compare" "-Wshadow" "-Wno-unused-parameter" "-Wno-unused-variable" "-Woverloaded-virtual" "-Wnon-virtual-dtor" "-Wno-missing-field-initializers" "-Wno-strict-aliasing" "-Wno-invalid-offsetof" "-msse2" "-std=c++17" "-DSNAPPY=1" "-DLZ4=1" "-DNDEBUG=1" "-DOS_LINUX" "-DROCKSDB_PLATFORM_POSIX" "-DROCKSDB_LIB_IO_POSIX" "-DROCKSDB_SUPPORT_THREAD_LOCAL" "-o" "/mnt/bitcoin-mainnet/chainhook/target/release/build/librocksdb-sys-2e1f7850601fc859/out/725dd32d7edce1c8-string_util.o" "-c" "rocksdb/util/string_util.cc" with args "c++" did not execute successfully (status code exit status: 1)

I'm running ordhook on same server and its indexing and building rocksdb.

I have clang, libssl-dev versions

chainhook$ sudo apt install make clang pkg-config libssl-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
make is already the newest version (4.3-4.1build1).
clang is already the newest version (1:16.0-57).
pkg-config is already the newest version (1.8.1-2).
libssl-dev is already the newest version (3.0.10-1ubuntu2.3).
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.

[MicaiahReid]

Thanks for the info @radicleart. That's interesting that building ordhook is working. I will investigate!

[ECBSJ]

I'm also getting similar issues when trying to install via cargo on Windows:

error: failed to run custom build command for `librocksdb-sys v0.10.0+7.9.2`

Caused by:
  process didn't exit successfully: `C:\Users\xxx\Documents\stuff\chainhook\target\release\build\librocksdb-sys-cf32fdf9d0c4e618\build-script-build` (exit code: 101)
  --- stderr
  thread 'main' panicked at C:\Users\xxx\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bindgen-0.64.0\./lib.rs:2393:31:
  Unable to find libclang: "couldn't find any valid shared libraries matching: ['clang.dll', 'libclang.dll'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: failed to compile `chainhook v1.6.0 (C:\Users\xxx\Documents\stuff\chainhook\components\chainhook-cli)`, intermediate artifacts can be found at `C:\Users\xxx\Documents\stuff\chainhook\target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

Do I need to change the librocksdb-sys dependency version?