Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 32-bit ARM build #754

Merged
merged 1 commit into from
Mar 8, 2023
Merged

Fix 32-bit ARM build #754

merged 1 commit into from
Mar 8, 2023

Conversation

EyeOfPython
Copy link
Contributor

Building this crate on arm-linux-gnueabihf is fine, HOWEVER, opening a rocksdb database using the built code fails:

On our CI, we get a "Value too large for defined data type" in our functional tests, see build output (you can login as guest if necessary).

After a bit of digging, it turns out this refers to a missing environment variable specifically for 32-bit systems that want to open files using 64-bit offsets.

Therefore, this PR fixes our build by adding those two env variables when CARGO_CFG_TARGET_POINTER_WIDTH is not 64.

@@ -226,6 +226,11 @@ fn build_rocksdb() {
config.define("ROCKSDB_IOURING_PRESENT", Some("1"));
}

if std::env::var("CARGO_CFG_TARGET_POINTER_WIDTH").unwrap() != "64" {
Copy link
Member

@aleksuss aleksuss Mar 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would propose to use matches! macro here rather than unwrap:

Suggested change
if std::env::var("CARGO_CFG_TARGET_POINTER_WIDTH").unwrap() != "64" {
if matches!(env::var("CARGO_CFG_TARGET_POINTER_WIDTH"), Ok(width) if width != "64") {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other places also use unwrap, so I thunk this would be more consistent

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it to use env::var though, I think this way it looks the cleanest. CARGO_CFG_TARGET_POINTER_WIDTH is guaranteed to be defined in build scripts, so IMO it's better to have it panic and let us know something went terribly wrong.

Building this crate on `arm-linux-gnueabihf` is fine, HOWEVER, opening a rocksdb database using the built code fails:

We get a "Value too large for defined data type" in our functional tests, see [build output](https://build.bitcoinabc.org/repository/download/BitcoinABC_BitcoinAbcStaging/515561:id/artifacts.tar.gz!/functional/test_runner_%E2%82%BF%E2%82%B5_%F0%9F%8F%83_20230302_193244/chronik_block_214/node0/regtest/debug.log).

After a bit of digging, it turns out this refers to a [missing environment variable specifically for 32-bit systems that want to open files using 64-bit offsets](https://stackoverflow.com/a/23372153).

Therefore, this PR [fixes our build](https://build.bitcoinabc.org/viewLog.html?buildId=515991&buildTypeId=BitcoinABC_BitcoinAbcStaging&tab=buildResultsDiv&branch_BitcoinAbcDiffs=refs%2Ftags%2Fphabricator%2Fdiff%2F38261) by adding those two env variables when CARGO_CFG_TARGET_POINTER_WIDTH is not 64.
@EyeOfPython
Copy link
Contributor Author

We're currently blocking on this fix @aleksuss @stanislav-tkach, do you guys think it will be deployed soon?

@aleksuss
Copy link
Member

aleksuss commented Mar 8, 2023

I will merge it into the master, but a new release with this fix will appear after two or three months, I guess.

@aleksuss aleksuss merged commit a6103ef into rust-rocksdb:master Mar 8, 2023
@EyeOfPython EyeOfPython deleted the build-32-bit branch March 8, 2023 10:24
@EyeOfPython
Copy link
Contributor Author

You can always release a minor version, like 0.20.2, or is there something preventing that?

@aleksuss
Copy link
Member

aleksuss commented Mar 8, 2023

You can always use a dependency directly from the master branch or even from a commit.

Congyuwang added a commit to Congyuwang/rust-rocksdb that referenced this pull request Mar 21, 2023
@Congyuwang
Copy link
Contributor

This change seems to break windows x86 (i686) compile.

@Congyuwang
Copy link
Contributor

Congyuwang commented Mar 21, 2023

running: "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.35.32215\\bin\\HostX64\\x86\\cl.exe" "-nologo" "-MD" "-O2" "-Brepro" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.8.1/fused-src/" "-I" "snappy/" "-I" "D:\\a\\RocksDict\\RocksDict\\target\\i686-pc-windows-msvc\\release\\build\\lz4-sys-2f2ef7caa05cef3e\\out\\include" "-I" "C:\\Users\\runneradmin\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\zstd-sys-2.0.7+zstd.1.5.4\\zstd/lib" "-I" "D:\\a\\RocksDict\\RocksDict\\target\\i686-pc-windows-msvc\\release\\build\\libz-sys-a3360bed1d7b33fd\\out/include" "-I" "D:\\a\\RocksDict\\RocksDict\\target\\i686-pc-windows-msvc\\release\\build\\bzip2-sys-9491d39ce54a7111\\out\\include" "-I" "." "-W4" "-EHsc" "-std:c++17" "-DSNAPPY=1" "-DLZ4=1" "-DZSTD=1" "-DZLIB=1" "-DBZIP2=1" "-DNDEBUG=1" "-DDWIN32" "-DOS_WIN" "-D_MBCS" "-DWIN64" "-DNOMINMAX" "-DROCKSDB_WINDOWS_UTF8_FILENAMES" "-DROCKSDB_SUPPORT_THREAD_LOCAL" "-D_FILE_OFFSET_BITS=64" "-D_LARGEFILE64_SOURCE=1" "-FoD:\\a\\RocksDict\\RocksDict\\target\\i686-pc-windows-msvc\\release\\build\\librocksdb-sys-32d9bb9bc3d80072\\out\\rocksdb/cache/sharded_cache.o" "-c" "rocksdb/cache/sharded_cache.cc"
  sharded_cache.cc
  rocksdb/cache/lru_cache.h(482): warning C4324: 'rocksdb::lru_cache::LRUCacheShard': structure was padded due to alignment specifier
  D:\a\RocksDict\RocksDict\target\i686-pc-windows-msvc\release\build\libz-sys-a3360bed1d7b33fd\out\include\zconf.h(475): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory
  rocksdb/util/math.h(143): warning C4127: conditional expression is constant
  rocksdb/util/math.h(143): note: consider using 'if constexpr' statement instead
  rocksdb/cache/sharded_cache.cc(95): note: see reference to function template instantiation 'int rocksdb::BitsSetToOne<uint32_t>(T)' being compiled
          with
          [
              T=uint32_t
          ]
  rocksdb/util/math.h(153): warning C4127: conditional expression is constant
  rocksdb/util/math.h(153): note: consider using 'if constexpr' statement instead
  exit code: 2

It says 'unistd.h' not found for libz-sys.

Reverting this commit fix compiling on windows i686. Moreover, it also runs correctly on win32.

FYI: comparing these two Github Action Runs: https://github.com/Congyuwang/RocksDict/actions/runs/4476861228/jobs/7867755946 (reverted)
and before:
https://github.com/Congyuwang/RocksDict/actions/runs/4476651838/jobs/7867232892.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants