Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GC doesn't work properly in the ANALYTICAL MODE #967

Closed
gvolfing opened this issue Jun 7, 2023 · 2 comments
Closed

GC doesn't work properly in the ANALYTICAL MODE #967

gvolfing opened this issue Jun 7, 2023 · 2 comments
Assignees
Labels
bug bug Importance - I3 Importance - I3

Comments

@gvolfing
Copy link
Contributor

gvolfing commented Jun 7, 2023

Memgraph version
2.8

Describe the bug
The issue happens when vertices and edges are created in IN_MEMORY_ANALYTICAL mode. The vertices and edges will be created and added to the skip list correctly, however when we try to delete these objects, they will be flagged for deletion as they should. Unfortunately the way analytical mode and garbage collection works at the moment, the GC will not be able to pick up the "deleted" objects and actually remove them from the skip list. This will clog up the skip list over time.

Under the hood, the way the deletion of skip list related objects is handled is that instead of actually deleting these object from memory, we just "mark" them as deleted objects. The garbage collector runs periodically, or we can force-call it by calling the FREE MEMORY query. Its job is to take care of the "marked" objects, un-link and delete them. The way this "marking" works is that the bool flag, deleted, is set to true for the given object, and it also involves the delta-chain of the transaction. This is problematic, because the way ANALYTICAL storage mode is implemented, these operations on transactions are almost always a no-op. When the garbage collector runs, the way it checks for the "marked" objects is by traversing the committed transactions, which will be an empty list in ANALYTICAL mode, so nothing is actually removed during garbage collection. There is an interesting occurrence here, in ANALYTICAL mode the delta-chain of the transaction is not modified, but the boolean flag of the vertex is set. So after deleting the vertices, running the query MATCH (n) RETURN COUNT (*) returns 0, but checking the number of vertices through SHOW STORAGE INFO is unchanged.

To Reproduce
Steps to reproduce the behavior:

  1. Set the database into analytical mode STORAGE MODE IN_MEMORY_ANALYTICAL;
  2. Create a couple of objects, CREATE (n:NODE{id:1}), (m:NODE{id:2}), (k:NODE{id:3});
  3. Delete the objects MATCH (n) DETACH DELETE (n);
  4. Force call the garbage collector FREE MEMORY;
  5. Check if the vertices have been removed or not with SHOW STORAGE INFO;
    after step 5 you should be seeing that the objects have not been removed in analytical mode. They are flaged as "deleted", but they are still in the skip list.
@gvolfing gvolfing added the bug bug label Jun 7, 2023
@gitbuda gitbuda added the Importance - I3 Importance - I3 label Jun 7, 2023
@gitbuda gitbuda changed the title [Vertices and edges created with analytical mode can not be picked up by the GC when deleted] GC doesn't work properly in the ANALYTICAL MODE Jun 7, 2023
@gitbuda
Copy link
Member

gitbuda commented Jun 28, 2023

This should be fixed under #1025 -> @gvolfing please double check 🙏

@gvolfing
Copy link
Contributor Author

Fixed with #1025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug bug Importance - I3 Importance - I3
Projects
None yet
Development

No branches or pull requests

2 participants