Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Avoid one indirection in KeyFragment and use string interning #554

Merged
merged 2 commits into from May 17, 2021
Merged

Avoid one indirection in KeyFragment and use string interning #554

merged 2 commits into from May 17, 2021

Conversation

sebastiencs
Copy link
Contributor

  • KeyFragment(Rc<String>) becomes KeyFragment(Rc<str>) to remove 1 pointer indirection
  • Use string interning to reduce allocation

@sebastiencs sebastiencs marked this pull request as ready for review May 17, 2021 11:47
@tizoc tizoc merged commit 1310bf2 into tezedge:protocol-runner-storage May 17, 2021
tizoc pushed a commit that referenced this pull request May 17, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
44d7c98
tizoc pushed a commit that referenced this pull request May 19, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
e2d3dd5
tizoc pushed a commit that referenced this pull request May 21, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
a3670f0
tizoc pushed a commit that referenced this pull request May 25, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
55145b7
tizoc pushed a commit that referenced this pull request May 26, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
3a2d3dc
tizoc pushed a commit that referenced this pull request May 31, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
74ecb0f
tizoc pushed a commit that referenced this pull request Jun 2, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
b2deeba
tizoc pushed a commit that referenced this pull request Jun 6, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
6a7b505
tizoc pushed a commit that referenced this pull request Jun 13, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
53fd724
tizoc pushed a commit that referenced this pull request Jun 15, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
de50685
sebastiencs added a commit to sebastiencs/tezedge that referenced this pull request Jun 16, 2021
@sebastiencs
Squashed commit of the following:
a548a27 
commit f05aac3
Merge: b11e25f e17bd5b
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Wed Jun 16 11:48:12 2021 +0200

    Merge remote-tracking branch 'origin/protocol-runner-storage' into dedup

commit b11e25f
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Tue Jun 15 18:29:16 2021 +0200

    After rebase

commit 3e62581
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Sun Jun 6 18:30:33 2021 +0200

    Deduplicate hashes and add new GC

commit d6bd435
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Jun 13 19:45:31 2021 -0300

    When processing requests, if an error happens pass it up.

commit b431b17
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Jun 13 19:11:34 2021 -0300

    Move replay integration test at the end.

    It is not working right now because the blocks snapshot being used is from an older rocksdb.

commit 1ffd5f7
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Jun 13 18:54:32 2021 -0300

    Don't copy old context-actions-replayer in drone file, not there anymore

commit c45a67f
Merge: 888b874 0b7c16e
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Jun 13 18:44:15 2021 -0300

    Merge pull request tezedge#580 from tezedge/remote-new-context

    Readonly access to context through IPC

commit 0b7c16e
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Jun 13 18:25:16 2021 -0300

    Update libtezos-ffi-distribution-summary.json

commit 7120e19
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Jun 13 17:27:02 2021 -0300

    Remove tezos/interop_callback and context listener

commit 95bb81f
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Jun 13 17:26:38 2021 -0300

    Remove storage/src/context

commit aa71467
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Jun 13 15:19:06 2021 -0300

    Remove tezos/context and references to it

commit 0b23b49
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Jun 13 14:53:59 2021 -0300

    Address clippy warnings

commit c4c700d
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Jun 13 13:58:04 2021 -0300

    Implement IPC access to TezEdge context

commit 97a9bc0
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Fri Jun 11 11:23:48 2021 -0300

    TezedgeContextRemote->TezedgeContextClient and move to tezos_wrapper

commit d371bb5
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Fri Jun 11 10:35:57 2021 -0300

    Remove/adjust all code using the old context

commit 6b29af7
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Wed Jun 9 14:41:48 2021 -0300

    Move some of the working tree methods to the index. Remove unused stats code.

commit c7a9e61
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Mon Jun 7 12:36:38 2021 -0300

    On RPCs, access context through remote API

commit 18d585a
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Thu May 27 17:24:58 2021 -0300

    Add IPC access to context functionality required by node RPCs

commit 888b874
Author: Sebastien Chapuis <sebastiencs@users.noreply.github.com>
Date:   Wed Jun 9 00:20:34 2021 +0800

    Use sql transaction to insert actions in the timing database (tezedge#584)

commit b0fd1f7
Author: Sebastien Chapuis <sebastiencs@users.noreply.github.com>
Date:   Tue Jun 8 20:35:53 2021 +0800

    [rpc] Add CORS headers on 404 response (tezedge#582)

commit 06ba2b6
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Jun 6 15:08:23 2021 -0300

    Update libtezos-ffi-distribution-summary.json

commit 359533e
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Sun Jun 6 16:55:51 2021 +0200

    Pass EntryHash by value

commit 047ab32
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Sat Jun 5 09:37:51 2021 +0200

    Do not recompute hashes for the commit and root of the tree

commit 2247661
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Sat Jun 5 08:54:54 2021 +0200

    Fix duration of a block

    Call `Instant::now` at the hook call, not in the timing thread which
    might be lagging behind

commit fc19ac1
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Thu Jun 3 17:02:13 2021 +0200

    Add comment about protocol-runner path

commit c7d962e
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Mon May 24 10:36:37 2021 +0200

    Implement replayer

    viablesystems.atlassian.net/browse/TE-516

    This adds the subcommand replay to light-node with 4 new arguments
    --from-block, --to-block, --target-path and --fail-above.

    Usage example:

    cargo run --bin light-node replay --target-path=/tmp/replay
    --config-file ./light_node/etc/tezedge/tezedge.config
    --network=edo2net --to-block
    BMf2TQSuyJrsE7JQjEBj1ztfspoVaFChEmVg6DsUYHxinsEVEeW --one-context
    --fail-above 1000

commit 97d78a9
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Wed Jun 2 12:03:40 2021 -0300

    Cargo fmt

commit f312802
Author: Sebastien Chapuis <sebastiencs@users.noreply.github.com>
Date:   Tue Jun 1 23:19:09 2021 +0800

    Fix mem_tree timing in global statistics (tezedge#569)

commit 0b5d428
Author: Sebastien Chapuis <sebastiencs@users.noreply.github.com>
Date:   Mon May 31 20:58:49 2021 +0800

    Compute block times and fix single context (tezedge#565)

    Also:

    * Do not panic when a sql error occurs

    * Add mem_tree to the list of query

commit f36bd70
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Fri May 28 13:31:19 2021 -0300

    When taking disk usage measurements, default to 0 for missing files

commit a90e10b
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Thu May 13 10:18:17 2021 -0300

    Implement FOLD action for WorkingTree and context

commit be54cc2
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Wed May 19 16:16:02 2021 -0300

    Add settings for context stats db to distroless config

commit 7bc53a5
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Wed May 19 10:38:26 2021 -0300

    Include `context_stats_db_path` in the `TezosContextConfiguration` conversion to OCaml

commit f18857e
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Wed May 19 10:37:59 2021 -0300

    Add missing field to TezosContextConfiguration construction

commit 82dcbb1
Author: Sebastien Chapuis <sebastiencs@users.noreply.github.com>
Date:   Wed May 19 21:00:40 2021 +0800

    Make the timing database path configurable (tezedge#557)

    * Make the timing database path configurable

    * Return a 404 HTTP response when block is not found

commit e7df914
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Tue May 18 18:28:18 2021 -0300

    Do the marking phase of the GC as part of the commit for much better performance

commit 135610c
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Tue May 18 16:10:40 2021 -0300

    Avoid boxroot when patch_context is None.

    This avoids some potential issues when running tests that don't initialize the OCaml runtime.

commit 76795ad
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Tue May 18 16:10:00 2021 -0300

    Update ocaml-interop dependency to 0.8.4

commit 27b9120
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Tue May 18 15:21:55 2021 -0300

    Add hook on context for when a block is applied (needed for GC)

commit 47f12d3
Author: Sebastien Chapuis <sebastiencs@users.noreply.github.com>
Date:   Mon May 17 22:47:17 2021 +0800

    Fix irmin checkout & commit times (tezedge#555)

commit 53fd724
Author: Sebastien Chapuis <sebastiencs@users.noreply.github.com>
Date:   Mon May 17 21:12:55 2021 +0800

    Use interned strings for context path fragments (tezedge#554)

commit d00bdc5
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Mon May 17 13:24:49 2021 +0200

    Store keys in a different table to avoid duplicates

commit c64d6c6
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Mon May 17 09:58:08 2021 +0200

    Use in-memory database during timing tests

commit 12a4577
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Fri May 14 12:22:05 2021 -0300

    Override ocaml-boxroot-sys dependency with a version from git that includes OCaml 4.10 headers

commit d5d7b1d
Author: Sebastien Chapuis <sebastiencs@users.noreply.github.com>
Date:   Thu May 13 23:20:26 2021 +0800

    Avoid key-value allocations when performing lookups on the tree (tezedge#543)

commit 6a4c5d4
Author: Sebastien Chapuis <sebastiencs@users.noreply.github.com>
Date:   Thu May 13 21:16:54 2021 +0800

    Compute total times in global stats, add Irmin stats (tezedge#549)

commit 77a598d
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Wed May 12 10:06:00 2021 -0300

    Remove outdated comment

commit be4fbef
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Wed May 12 10:01:18 2021 -0300

    Add missing copyright notice

commit 96b42f5
Author: Sebastien Chapuis <sebastiencs@users.noreply.github.com>
Date:   Wed May 12 20:42:09 2021 +0800

    Compute context stats in the protocol runner process (tezedge#546)

commit 531d74c
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Tue May 11 15:46:22 2021 -0300

    Fix serialization of node hashes by avoiding clones when partitioning inodes

commit c1a5880
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun May 9 11:37:33 2021 -0300

    Calculate max_time too

commit 3190f38
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun May 9 10:57:13 2021 -0300

    Remove list and fold action metrics, not needed

commit 4a73f52
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sat May 8 16:57:45 2021 -0300

    Filter out actions where block_id is null

commit 1ebc162
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sat May 8 09:28:26 2021 -0300

    If there is a failure when processing a timing message, continue anyway

commit 803e87d
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Fri May 7 17:49:01 2021 -0300

    FIX on timings: ignore duplicate hashes for now

commit 317b92a
Author: Sebastien Chapuis <sebastiencs@users.noreply.github.com>
Date:   Fri May 7 20:41:03 2021 +0800

    Compute global stats for the context (tezedge#545)

commit 0df28b8
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Thu May 6 15:17:42 2021 -0300

    Add context storage settings to distroless docker image config

commit 4d04e5e
Author: Sebastien Chapuis <sebastiencs@users.noreply.github.com>
Date:   Thu May 6 21:52:46 2021 +0800

    Context timings recording + RPCs

commit 8a39908
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Mon May 3 18:11:15 2021 -0300

    Add GCed variant of InMem storage and make it the default

commit 0e23cd1
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Mon May 3 11:18:32 2021 -0300

    Add configuration structure for initialization of the protocol context

    Allows for both Irmin and Tezedge contexts now.

commit a897a2d
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Thu Apr 29 11:09:01 2021 -0300

    When preparing a commit, avoid recalculating entry hashes that we already have

commit d9717ce
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Mon Apr 26 16:12:51 2021 -0300

    Add context timing callbacks

commit 77da2a8
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Apr 25 19:17:40 2021 -0300

    Bump ocaml-interop dependency

    For now an OCaml install will be required when building.

commit 1c2d728
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Sun Apr 25 13:32:47 2021 -0300

    Rework custom blocks to fix segmentation faults, add patch_context

commit ecc6dfb
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Fri Apr 23 12:12:57 2021 +0200

    Avoid serializing more than once a node

commit 7f0d991
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Thu Apr 22 09:24:59 2021 -0300

    Add context/tree list function to FFI

commit 95438a4
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Thu Apr 22 08:37:04 2021 -0300

    Add FIXME comments about handling empty commits

commit b8fd1ba
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Thu Apr 22 10:12:35 2021 +0200

    Implement `list` method on the WorkingTree

commit 58b667d
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Wed Apr 21 17:05:28 2021 -0300

    Complete `tezedge_context_empty`.

commit 5819a8f
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Wed Apr 21 16:58:31 2021 -0300

    Add missing code and initialization for subtree API functions in FFI

commit e9898b3
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Wed Apr 21 14:32:45 2021 +0200

    Add subtree API to context

    https://viablesystems.atlassian.net/browse/TE-500

commit f9fda02
Author: Sebastien Chapuis <sebastien@chapu.is>
Date:   Tue Apr 20 20:12:10 2021 +0200

    Compute entry hash when necessary

commit 80f2e76
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Tue Apr 20 17:41:08 2021 -0300

    More work on adding missing API functions, make `get` return Option

    Update method names to map new OCaml API

commit d016d4d
Author: Bruno Deferrari <bruno.deferrari@viablesystems.io>
Date:   Tue Apr 20 11:32:31 2021 -0300

    Remove 'rocksdb' from new context aftion file replayer, set 'inmem' as the default backend.

commit 8f86144
Author: Bruno Deferrari <bruno.deferrari@simplestaking.com>
Date:   Mon Apr 12 16:19:40 2021 -0300

    Add storage index API, custom OCaml block support and complete FFI exposed API

commit 269f1ab
Author: Bruno Deferrari <bruno.deferrari@simplestaking.com>
Date:   Fri Apr 9 12:01:46 2021 -0300

    Finish moving repo logic out of WorkingTree and into TezedgeContext

commit 1cc4de8
Author: Bruno Deferrari <bruno.deferrari@simplestaking.com>
Date:   Thu Apr 8 22:46:00 2021 -0300

    Add TODOs for TE-261

commit 847b53f
Author: Bruno Deferrari <bruno.deferrari@simplestaking.com>
Date:   Thu Apr 8 22:44:03 2021 -0300

    New context implementation (WIP)
tizoc pushed a commit that referenced this pull request Jun 16, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
0706171
tizoc pushed a commit that referenced this pull request Jun 18, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
bf81b00
tizoc pushed a commit that referenced this pull request Jun 18, 2021
@sebastiencs @tizoc
Use interned strings for context path fragments (#554)
6326311
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@sebastiencs @tizoc