Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(server): Memory tracker #2501

Merged
merged 3 commits into from
Jan 30, 2024
Merged

feat(server): Memory tracker #2501

merged 3 commits into from
Jan 30, 2024

Conversation

chakaz
Copy link
Collaborator

@chakaz chakaz commented Jan 29, 2024

This PR adds AllocationTracker class, which can track (currently print) memory allocations via new and delete based on a given criteria (size & sampling fraction).

As a side effect, now we use mimalloc for new and delete also in debug builds (which wasn't the case before).

Here's the comment above Track(), provided here for completeness:

// Allow tracking of memory allocation via `new` and `delete` based on input criteria.
//
// MEMORY TRACK ADD <lower-bound> <upper-bound> <sample-odds>
// - Sets up tracking memory allocations in the (inclusive) range [lower, upper]
// - sample-odds indicates how many of the allocations will be logged, there 0 means none, 1 means
//   all, and everything in between is linear
// - There could be at most 4 tracking placed in parallel
//
// MEMORY TRACK REMOVE <lower-bound> <upper-bound>
// - Removes all memory tracking added which match bounds
// - Could remove 0, 1 or more
//
// MEMORY TRACK CLEAR
// - Removes all memory tracking
//
// MEMORY TRACK GET
// - Returns an array with all active tracking
//
// This command is not documented in `MEMORY HELP` because it's meant to be used internally.

@@ -270,6 +249,38 @@ void MemoryCmd::Stats() {
}
}

void MemoryCmd::MallocStats(CmdArgList args) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part is a literal cut-paste, no other changes

@chakaz chakaz requested a review from romange January 29, 2024 13:02
@romange
Copy link
Collaborator

romange commented Jan 29, 2024

Please add a commit description.

return;
}

inside_process_new = true;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it to prevent a recursive endless loop? care to add a comment?

// See LICENSE for licensing terms.
//

#include "allocation_tracker.h"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would put it under core/

#include "absl/container/inlined_vector.h"
#include "absl/strings/numbers.h"

#define INJECT_ALLOCATION_TRACKER
Copy link
Collaborator

@romange romange Jan 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please introduce a CMakeLists option DFLY_MEMORY_TRACKING in src/CMakelists.txt ?
Let's set it to ON by default and it will add the conditional define to dfly_main.cc.
In case it's disabled we fallback to #include <mimalloc-new-delete.h>

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, I see you add this define to every file but use it only here. ok

#include "absl/cleanup/cleanup.h"
#include "absl/container/inlined_vector.h"
#include "absl/strings/numbers.h"
#ifdef NDEBUG
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we now always allocate via mimalloc. worth adding this to commit description as well.

@@ -289,4 +300,73 @@ void MemoryCmd::Usage(std::string_view key) {
rb->SendLong(memory_usage);
}

void MemoryCmd::Track(CmdArgList args) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add comment above how the command looks like:
// MEMORY TRACK ...

Copy link
Collaborator

@romange romange left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@chakaz chakaz requested a review from romange January 29, 2024 19:42
//
// This command is not documented in `MEMORY HELP` because it's meant to be used internally.
void MemoryCmd::Track(CmdArgList args) {
#ifndef DFLY_ENABLE_MEMORY_TRACKING
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see.

@chakaz
Copy link
Collaborator Author

chakaz commented Jan 30, 2024

🤓

@chakaz chakaz merged commit b2bdb0f into main Jan 30, 2024
10 checks passed
@chakaz chakaz deleted the mem-tracker branch January 30, 2024 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants