Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve memory handling of Deltas #1688

Merged
merged 3 commits into from Feb 6, 2024
Merged

Improve memory handling of Deltas #1688

merged 3 commits into from Feb 6, 2024

Conversation

Ignition
Copy link
Contributor

@Ignition Ignition commented Feb 2, 2024

  • Reduce delta from 104B to 80B
  • Hold and pass them around as in a deque
  • Detect and deleted deltas within commit if safe to do so

relates to #1685

profile without jemalloc

Before After
Before After

@Ignition Ignition added the memory label Feb 2, 2024
@Ignition Ignition self-assigned this Feb 2, 2024
@Ignition Ignition force-pushed the GL_2024_02_01 branch 3 times, most recently from 788c4ba to 821012e Compare February 2, 2024 17:44
- Reduce delta from 104B to 80B
- Hold and pass them around as in a deque
- Detect and deleted deltas within commit if safe to do so
@Ignition Ignition force-pushed the GL_2024_02_01 branch 4 times, most recently from 3f90def to 9e42b5c Compare February 5, 2024 20:10
Copy link
Contributor

@Josipmrden Josipmrden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few comments

src/storage/v2/delta.hpp Show resolved Hide resolved
src/storage/v2/id_types.hpp Show resolved Hide resolved
@Ignition Ignition enabled auto-merge (squash) February 6, 2024 17:02
@Ignition Ignition merged commit 4ef6a1f into master Feb 6, 2024
8 checks passed
@Ignition Ignition deleted the GL_2024_02_01 branch February 6, 2024 17:07
@imilinovic imilinovic added Docs needed Docs needed and removed Docs - changelog only Docs - changelog only labels Feb 21, 2024
Ignition added a commit to memgraph/documentation that referenced this pull request Feb 21, 2024
This is inline with a recent change
memgraph/memgraph#1688
kgolubic pushed a commit to memgraph/documentation that referenced this pull request Feb 27, 2024
This is inline with a recent change
memgraph/memgraph#1688
kgolubic added a commit to memgraph/documentation that referenced this pull request Feb 28, 2024
* Create Memgraph 2.15 section in Release notes

* Add placeholders to the release notes

* Update Release notes

* Update release notes

* Update Release notes

* Updated release notes

* Update Release notes

* Update Release notes

* Update release notes

* Update RN

* Update RN

* [memgraph-2-15<] Add propertySize function (#485)

* Update broken links (#448)

* [master < 450-update-default...] Update default flag values in the docs (#452)

* Update default values

* Update sitemap

* Accumulated path weight (#453)

* Add accumulated path and weight description

* Update pages/advanced-algorithms/built-in-graph-algorithms.mdx

* Update pages/advanced-algorithms/built-in-graph-algorithms.mdx

* Update pages/advanced-algorithms/built-in-graph-algorithms.mdx

* Update pages/advanced-algorithms/built-in-graph-algorithms.mdx

* Update pages/advanced-algorithms/built-in-graph-algorithms.mdx

* Update pages/advanced-algorithms/built-in-graph-algorithms.mdx

* Apply suggestions from code review

* Apply suggestions from code review

* Quick fix

---------

Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
Co-authored-by: Vlasta <95473291+vpavicic@users.noreply.github.com>

* Update instructions (#454)

* Change search behavior (#455)

* Fix menu link (#457)

* Update csv import (#449)

* update csv import

* fix examples and expand

* remove unused images

* Minor updates

---------

Co-authored-by: kgolubic <kgolubic@gmail.com>
Co-authored-by: Kruno Golubic <46486712+kgolubic@users.noreply.github.com>

* Add placeholders to the release notes

* Fix release notes (#460)

* Update broken link (#462)

* [master < 463-explain-serialization...] Add a new documentation page for Serialization errors  (#465)

* Create new error documentation page

* Update the instructions

* Add sitemap (#467)

* Update sitemap (#469)

* Update serialize error page (#470)

* Update page

* Update client pages

* Update image (#471)

* Update docker-compose files (#473)

* Update release notes and direct download links (#482)

* Added documentation for propertySize

---------

Co-authored-by: Kruno Golubic <46486712+kgolubic@users.noreply.github.com>
Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
Co-authored-by: Vlasta <95473291+vpavicic@users.noreply.github.com>
Co-authored-by: Matea Pesic <80577904+matea16@users.noreply.github.com>
Co-authored-by: kgolubic <kgolubic@gmail.com>

* Update release notes

* Update Release notes

* [memgraph-2-15<] Auth module changes (#475)

* Updates for the auth module PR

* Fix steps code

* Update auth.mdx

---------

Co-authored-by: kgolubic <kgolubic@gmail.com>
Co-authored-by: Kruno Golubic <46486712+kgolubic@users.noreply.github.com>

* Update Delta to be 80B (#488)

This is inline with a recent change
memgraph/memgraph#1688

* Update Release notes

* Update release notes

* [memgraph-2-15<] Expand parameters page (#507)

* update (and fix) parameters page

* Update expressions.mdx

---------

Co-authored-by: Kruno Golubic <46486712+kgolubic@users.noreply.github.com>

* Update RN

* [memgraph-2-15<] Update docker compose documentation (#480)

* Update broken links (#448)

* [master < 450-update-default...] Update default flag values in the docs (#452)

* Update default values

* Update sitemap

* Accumulated path weight (#453)

* Add accumulated path and weight description

* Update pages/advanced-algorithms/built-in-graph-algorithms.mdx

* Update pages/advanced-algorithms/built-in-graph-algorithms.mdx

* Update pages/advanced-algorithms/built-in-graph-algorithms.mdx

* Update pages/advanced-algorithms/built-in-graph-algorithms.mdx

* Update pages/advanced-algorithms/built-in-graph-algorithms.mdx

* Update pages/advanced-algorithms/built-in-graph-algorithms.mdx

* Apply suggestions from code review

* Apply suggestions from code review

* Quick fix

---------

Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
Co-authored-by: Vlasta <95473291+vpavicic@users.noreply.github.com>

* Update instructions (#454)

* Change search behavior (#455)

* Fix menu link (#457)

* Update csv import (#449)

* update csv import

* fix examples and expand

* remove unused images

* Minor updates

---------

Co-authored-by: kgolubic <kgolubic@gmail.com>
Co-authored-by: Kruno Golubic <46486712+kgolubic@users.noreply.github.com>

* Add placeholders to the release notes

* Fix release notes (#460)

* Update broken link (#462)

* [master < 463-explain-serialization...] Add a new documentation page for Serialization errors  (#465)

* Create new error documentation page

* Update the instructions

* Add sitemap (#467)

* Update sitemap (#469)

* Update serialize error page (#470)

* Update page

* Update client pages

* Update image (#471)

* Update docker-compose files (#473)

* Update docker compose document

* Update First steps with Memgraph

* Update Docker compose documentation

* Update Docker documentation

* Update Install Memgraph documentation

* Update Getting started

* Update pages/getting-started.mdx

Co-authored-by: Katarina Supe <61758502+katarinasupe@users.noreply.github.com>

* Update pages/getting-started/install-memgraph/docker.mdx

Co-authored-by: Katarina Supe <61758502+katarinasupe@users.noreply.github.com>

* Update getting-started.mdx

* Update first-steps-with-memgraph.mdx

* Update install-memgraph.mdx

* Update docker.mdx

* Update docker.mdx

* Update docker.mdx

* Update docker.mdx

* Add curl for Windows

* Update driver pages

* Update C-sharp documentation

* Update docker.mdx

* Update docker.mdx

* Update pages/getting-started.mdx

Co-authored-by: Katarina Supe <61758502+katarinasupe@users.noreply.github.com>

* Update pages/getting-started/first-steps-with-memgraph.mdx

Co-authored-by: Katarina Supe <61758502+katarinasupe@users.noreply.github.com>

* Update pages/getting-started/install-memgraph.mdx

Co-authored-by: Katarina Supe <61758502+katarinasupe@users.noreply.github.com>

* Add CodeSnippets

* Update

* Update

* Update code snippets

* Document flags

* Update release notes and direct download links (#482)

* Fix typos

* Update PowerShell command

* update analytics

* update copy button id

* Update docker-compose.mdx

---------

Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
Co-authored-by: Vlasta <95473291+vpavicic@users.noreply.github.com>
Co-authored-by: Matea Pesic <80577904+matea16@users.noreply.github.com>
Co-authored-by: Katarina Supe <61758502+katarinasupe@users.noreply.github.com>
Co-authored-by: katarinasupe <supe.katarina@gmail.com>

* Update RN

* [memgraph-2-15<] System replication (#476)

* Experimental replication first run

* Update pages/configuration/experimental_replication.mdx

Co-authored-by: Gareth Andrew Lloyd <gareth@ignition-web.co.uk>

* Update pages/help-center/faq.md

Co-authored-by: Ante Pušić <ante.pusic@memgraph.io>

* Update faq.md

* Rename experimental_replication.mdx to experimental-replication.mdx

* Update experimental-replication.mdx

* Update replication.mdx

---------

Co-authored-by: Kruno Golubic <46486712+kgolubic@users.noreply.github.com>
Co-authored-by: Gareth Andrew Lloyd <gareth@ignition-web.co.uk>
Co-authored-by: Ante Pušić <ante.pusic@memgraph.io>

* Update RN

* Update styling

* Update styling

* Stytling fix

* Update RN

* High availability docs (#491)

* Cluster management

* add initial docs for HA

* Added example

* Unregister from current main

* add idempotency and diverged from MAIN

* add missing parts last commit

* Edit example

* fix TODOs

* Update high-availability.mdx

* Update high-availability.mdx

* Update high-availability.mdx

* Update the formatting and language style

* remove comments

* change to ip address

* Add callout about IP addresses

* Update menu json file

---------

Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
Co-authored-by: Kruno Golubic <46486712+kgolubic@users.noreply.github.com>
Co-authored-by: kgolubic <kgolubic@gmail.com>

* Update rpm-package.mdx

* Update RN

---------

Co-authored-by: Josipmrden <josip.mrden@memgraph.io>
Co-authored-by: Aidar Samerkhanov <aidar.samerkhanov@memgraph.io>
Co-authored-by: Vlasta <95473291+vpavicic@users.noreply.github.com>
Co-authored-by: Matea Pesic <80577904+matea16@users.noreply.github.com>
Co-authored-by: andrejtonev <29177572+andrejtonev@users.noreply.github.com>
Co-authored-by: Gareth Andrew Lloyd <gareth.lloyd@memgraph.io>
Co-authored-by: DavIvek <david.ivekovic@memgraph.io>
Co-authored-by: Katarina Supe <61758502+katarinasupe@users.noreply.github.com>
Co-authored-by: katarinasupe <supe.katarina@gmail.com>
Co-authored-by: Gareth Andrew Lloyd <gareth@ignition-web.co.uk>
Co-authored-by: Ante Pušić <ante.pusic@memgraph.io>
Co-authored-by: Andi <andi8647@gmail.com>
Co-authored-by: antoniofilipovic <filipovicantonio1998@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants