Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to LevelDB 1.20 #10544

Merged
merged 4 commits into from Jun 13, 2017

Conversation

Projects
None yet
7 participants
@sipa
Copy link
Member

sipa commented Jun 6, 2017

@sipa sipa force-pushed the sipa:leveldb120 branch Jun 6, 2017

@gmaxwell
Copy link
Member

gmaxwell left a comment

utACK

@sipa

This comment has been minimized.

Copy link
Member Author

sipa commented Jun 6, 2017

Major improvement: adds the ability to use hardware-accelerated crc32, which this PR is not exploiting yet.

@gmaxwell

This comment has been minimized.

Copy link
Member

gmaxwell commented Jun 7, 2017

Travis is bombing with

leveldb/util/env_posix.cc:29:10: fatal error: 'util/env_posix_test_helper.h'

  file not found

#include "util/env_posix_test_helper.h"

@paveljanik

This comment has been minimized.

Copy link
Contributor

paveljanik commented Jun 7, 2017

On OS X:

Making all in src
  CXXLD    bitcoind
Undefined symbols for architecture x86_64:
  "leveldb::port::AcceleratedCRC32C(unsigned int, char const*, unsigned long)", referenced from:
      leveldb::crc32c::Extend(unsigned int, char const*, unsigned long) in libleveldb.a(leveldb_libleveldb_a-crc32c.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
@paveljanik

This comment has been minimized.

Copy link
Contributor

paveljanik commented Jun 7, 2017

But util/env_posix_test_helper.h is there:

94fe30a#diff-c439a67281cf7c414d51630edca309c9

Strange. This Travis issue does not happen locally.

@paveljanik

This comment has been minimized.

Copy link
Contributor

paveljanik commented Jun 7, 2017

My issue is this one 8) NuxiNL/cloudabi-ports#32 reported by @laanwj ;-)

@jonasschnelli

This comment has been minimized.

Copy link
Member

jonasschnelli commented Jun 8, 2017

Gitian complains:

  CXX      leveldb/util/leveldb_libleveldb_a-status.o
  CXX      leveldb/port/leveldb_libleveldb_a-port_posix.o
  CXX      leveldb/helpers/memenv/leveldb_libmemenv_a-memenv.o
  CXX      test/test_test_bitcoin-arith_uint256_tests.o
  AR       libbitcoin_cli.a
leveldb/util/env_posix.cc:29:10: fatal error: 'util/env_posix_test_helper.h' file not found
#include "util/env_posix_test_helper.h"
         ^
1 error generated.
make[2]: *** [leveldb/util/leveldb_libleveldb_a-env_posix.o] Error 1
make[2]: *** Waiting for unfinished jobs....
  CXX      test/test_test_bitcoin-addrman_tests
@laanwj

This comment has been minimized.

Copy link
Member

laanwj commented Jun 8, 2017

My issue is this one 8) NuxiNL/cloudabi-ports#32 reported by @laanwj ;-)

There the SSE-accelerated CRC function was referenced, but not linked in.

@sipa

This comment has been minimized.

Copy link
Member Author

sipa commented Jun 8, 2017

@theuni fixed it, yo

@paveljanik

This comment has been minimized.

Copy link
Contributor

paveljanik commented Jun 8, 2017

compiles OK here, only one new warning:

leveldb/db/version_set.cc:1039:21: warning: comparison of integers of different signs: 'uint64_t' (aka 'unsigned long long') and 'int' [-Wsign-compare]
      manifest_size >= TargetFileSize(options_)) {
      ~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
@MarcoFalke

This comment has been minimized.

Copy link
Member

MarcoFalke commented Jun 9, 2017

@sipa I suspect you need to fix the out-of-tree builds by something like

diff --git a/src/Makefile.leveldb.include b/src/Makefile.leveldb.include
index b61088e..ac38141 100644
--- a/src/Makefile.leveldb.include
+++ b/src/Makefile.leveldb.include
@@ -77,6 +77,7 @@ leveldb_libleveldb_a_SOURCES += leveldb/table/merger.h
 leveldb_libleveldb_a_SOURCES += leveldb/table/format.h
 leveldb_libleveldb_a_SOURCES += leveldb/table/iterator_wrapper.h
 leveldb_libleveldb_a_SOURCES += leveldb/util/crc32c.h
+leveldb_libleveldb_a_SOURCES += leveldb/util/env_posix_test_helper.h
 leveldb_libleveldb_a_SOURCES += leveldb/util/arena.h
 leveldb_libleveldb_a_SOURCES += leveldb/util/random.h
 leveldb_libleveldb_a_SOURCES += leveldb/util/posix_logger.h
@paveljanik

This comment has been minimized.

Copy link
Contributor

paveljanik commented Jun 9, 2017

@MarcoFalke Good catch!

@sipa

This comment has been minimized.

Copy link
Member Author

sipa commented Jun 9, 2017

@MarcoFalke Thanks! I've added a commit with you as author.

sipa and others added some commits Jun 10, 2017

Squashed 'src/leveldb/' changes from a31c8aa40..196962ff0
196962ff0 Add AcceleratedCRC32C to port_win.h
1bdf1c34c Merge upstream LevelDB v1.20
d31721eb0 Merge #17: Fixed file sharing errors
fecd44902 Fixed file sharing error in Win32Env::GetFileSize(), Win32SequentialFile::_Init(), Win32RandomAccessFile::_Init() Fixed error checking in Win32SequentialFile::_Init()
5b7510f1b Merge #14: Merge upstream LevelDB 1.19
0d969fd57 Merge #16: [LevelDB] Do no crash if filesystem can't fsync
c8c029b5b [LevelDB] Do no crash if filesystem can't fsync
a53934a3a Increase leveldb version to 1.20.
f3f139737 Separate Env tests from PosixEnv tests.
eb4f0972f leveldb: Fix compilation warnings in port_posix_sse.cc on x86 (32-bit).
d0883b600 Fixed path to doc file: index.md.
7fa20948d Convert documentation to markdown.
ea175e28f Implement support for Intel crc32 instruction (SSE 4.2)
95cd743e5 Including <limits> for std::numeric_limits.
646c3588d Limit the number of read-only files the POSIX Env will have open.
d40bc3fa5 Merge #13: Typo
ebbd772d3 Typo
a2fb086d0 Add option for max file size. The currend hard-coded value of 2M is inefficient in colossus.

git-subtree-dir: src/leveldb
git-subtree-split: 196962ff01c39b4705d8117df5c3f8c205349950

@sipa sipa force-pushed the sipa:leveldb120 branch to 3ee3d04 Jun 10, 2017

@gmaxwell
Copy link
Member

gmaxwell left a comment

ACK

@laanwj

This comment has been minimized.

Copy link
Member

laanwj commented Jun 13, 2017

After merging bitcoin-core/leveldb#1

src/leveldb in HEAD was last updated to upstream commit 196962ff01c39b4705d8117df5c3f8c205349950 (tree d5a89851da0a8ab07c40966ac137f4e299c80927)
src/leveldb in HEAD currently refers to tree d5a89851da0a8ab07c40966ac137f4e299c80927
GOOD
@laanwj

laanwj approved these changes Jun 13, 2017

@laanwj laanwj merged commit 3ee3d04 into bitcoin:master Jun 13, 2017

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

laanwj added a commit that referenced this pull request Jun 13, 2017

Merge #10544: Update to LevelDB 1.20
3ee3d04 Add extra LevelDB source to Makefile (MarcoFalke)
2424989 leveldb: enable runtime-detected crc32 instructions (Cory Fields)
cf44e4c Squashed 'src/leveldb/' changes from a31c8aa40..196962ff0 (Pieter Wuille)

Tree-SHA512: 19ade77e3f6265507b3ab7b9aa5150d378aa0751e24ac7a61567b0f720a566cedc6c3d3336da17a3bd2b5d068ee86600d96a15228f78bd20ccf98c8fc9041a91
@theuni

This comment has been minimized.

Copy link
Member

theuni commented Jun 13, 2017

post-merge utACK.

@fanquake fanquake referenced this pull request Jun 14, 2017

Closed

LevelDB 1.20 #10138

2 of 3 tasks complete

@sipa sipa deleted the sipa:leveldb120 branch Jun 23, 2017

sickpig referenced this pull request in sickpig/BitcoinUnlimited Aug 17, 2017

Backport of Core #10544: Update to LevelDB 1.20
3ee3d04 Add extra LevelDB source to Makefile (MarcoFalke)
2424989 leveldb: enable runtime-detected crc32 instructions (Cory Fields)
cf44e4c Squashed 'src/leveldb/' changes from a31c8aa40..196962ff0 (Pieter Wuille)

sickpig referenced this pull request in sickpig/BitcoinUnlimited Aug 17, 2017

Port Core #10544: Update to LevelDB 1.20
3ee3d04 Add extra LevelDB source to Makefile (MarcoFalke)
2424989 leveldb: enable runtime-detected crc32 instructions (Cory Fields)
cf44e4c Squashed 'src/leveldb/' changes from a31c8aa40..196962ff0 (Pieter Wuille)

Tree-SHA512: 19ade77e3f6265507b3ab7b9aa5150d378aa0751e24ac7a61567b0f720a566cedc6c3d3336da17a3bd2b5d068ee86600d96a15228f78bd20ccf98c8fc9041a91

karel-3d pushed a commit to karel-3d/bitcoin that referenced this pull request Oct 31, 2017

Update to LevelDB 1.20
Port of bitcoin core bitcoin#10544.

Summary:
3ee3d04 Add extra LevelDB source to Makefile (MarcoFalke)
2424989 leveldb: enable runtime-detected crc32 instructions (Cory Fields)
cf44e4c Squashed 'src/leveldb/' changes from a31c8aa40..196962ff0 (Pieter Wuille)

Test Plan: build, make check.  Binary runs fine (though I don't use the wallet)

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D580
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.