Merge upstream LevelDB 1.20 #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nefficient in colossus. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=134391640
Background compaction can create an unbounded number of leveldb::RandomAccessFile instances. On 64-bit systems mmap is used and file descriptors are only used beyond a certain number of mmap's. 32-bit systems to not use mmap at all. leveldb::RandomAccessFile does not observe Options.max_open_files so compaction could exhaust the file descriptor limit. This change uses getrlimit to determine the maximum number of open files and limits RandomAccessFile to approximately 20% of that value. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=143505556
------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=146841327
This change authored by vadimskipin and submitted via:
google#309
Changes made to support iOS builds and other architectures
without support for SSE 4.2.
db_bench reports original crc32 speed at:
crc32c : 3.610 micros/op; 1082.0 MB/s (4K per op)
with this change performance has increased to:
crc32c : 0.843 micros/op; 4633.6 MB/s (4K per op)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148694935
Markdown is more readable in a text editor and when hosted on GitHub is more readable than HTML. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=148830423
Prior index.html was using rawgit.com which doesn't process Markdown and therefore only serves the markdown source. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=148902180
LE_LOAD64 is only used when _mm_crc32_u64 is available, on 64-bit x86 processors. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=148906169
env_test.cc defines EnvPosixTest which tests the Env implementation returned by Env::Default(). The naming is a bit unfortunate, as the tests in env_test.cc are written against the Env contract, and therefore are applicable to any Env implementation. An instance of the confusion caused by the naming is [] which added a dependency from env_test.cc to EnvPosixTestHelper, which is closely coupled to EnvPosix. This change disentangles EnvPosix-specific test code into a env_posix_test.cc file. The code there uses EnvPosixTestHelper and specifically targets the EnvPosix implementation. env_test.cc now implements EnvTest, and contains tests that are also applicable to other ports, which may define their own Env implementation. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=148914642
------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=148937577
|
@tmagik I had to delete and recreate this repository in order to fix up the missing "Forked from" marker above (which made PRs complicated). I didn't mean to delete your comments on the earlier version of this PR, but perhaps they're more appropriate as a separate issue (here, or on the https://github.com/bitcoin/bitcoin/ repo)? |
|
Ah that’s what happened. The relevant comments can be condensed to:
@sipa so maybe for bitcoin core, then what I'd propose is:
- if -core keeps bitcoin-consensus-leveldb in the same build tree, that it at least be provided as a separate library, with libbitcoin-leveldb and libbitcoin-memenv available for other projects
- develop a testing system and risk mitigation strategy by which we can gain some confidence that alternative database backends (such as maybe hyperleveldb or rocksdb) work correctly
… On Apr 13, 2017, at 11:19 AM, Pieter Wuille ***@***.***> wrote:
@tmagik <https://github.com/tmagik> I had to delete and recreate this repository in order to fix up the missing "Forked from" marker above (which made PRs complicated). I didn't mean to delete your comments on the earlier version of this PR, but perhaps they're more appropriate as a separate issue (here, or on the https://github.com/bitcoin/bitcoin/ <https://github.com/bitcoin/bitcoin/> repo)?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGAQEDJx2b7HpWwyZ8bPnYAVkpP3kGkTks5rvkr_gaJpZM4M85EJ>.
|
|
@tmagik Those are comments that belong in an issue on https://github.com/bitcoin/bitcoin/. |
|
optimized crc32 is exciting, IIRC that's a pretty significant bottleneck. <cfields_> sipa: for leveldb merge, whenever it's needed: theuni/bitcoin@2cb7dda |
|
ACK. (tested previously) |
laanwj
added a commit
that referenced
this pull request
Jun 13, 2017
196962f Add AcceleratedCRC32C to port_win.h (Pieter Wuille) a53934a Increase leveldb version to 1.20. (costan) f3f1397 Separate Env tests from PosixEnv tests. (costan) eb4f097 leveldb: Fix compilation warnings in port_posix_sse.cc on x86 (32-bit). (costan) d0883b6 Fixed path to doc file: index.md. (cmumford) 7fa2094 Convert documentation to markdown. (cmumford) ea175e2 Implement support for Intel crc32 instruction (SSE 4.2) (costan) 95cd743 Including <limits> for std::numeric_limits. (cmumford) 646c358 Limit the number of read-only files the POSIX Env will have open. (cmumford) a2fb086 Add option for max file size. The currend hard-coded value of 2M is inefficient in colossus. (corrado) Tree-SHA512: 27df89e36313889cbe87d9e0e02e74c70c086a1f6b0d4f3fd98bc18d3b87aa94848864b31f39fe3c91870e3bba6ec7c355ec3209a7cfd3d16b117782c7c3245a
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.