Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added --aggregate option to attach #455

Merged
merged 1 commit into from
Sep 13, 2023

Conversation

ivonastojanovic
Copy link

Added --aggregate option which allows user to request aggregated mode for in-memory aggregation.

Issue number of the reported bug or feature request: #

Describe your changes
A clear and concise description of the changes you have made.

Testing performed
Describe the testing you have performed to ensure that the bug has been addressed, or that the new feature works as planned.

Additional context
Add any other context about your contribution here.

@codecov-commenter
Copy link

codecov-commenter commented Sep 8, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.01% 🎉

Comparison is base (bb9c0db) 91.88% compared to head (5a2cdef) 91.90%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #455      +/-   ##
==========================================
+ Coverage   91.88%   91.90%   +0.01%     
==========================================
  Files          90       91       +1     
  Lines       10766    10803      +37     
  Branches     1473     1485      +12     
==========================================
+ Hits         9892     9928      +36     
- Misses        872      873       +1     
  Partials        2        2              
Flag Coverage Δ
cpp 85.01% <ø> (-0.22%) ⬇️
python_and_cython 95.41% <100.00%> (+0.12%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
src/memray/commands/attach.py 59.17% <100.00%> (+5.23%) ⬆️
tests/integration/test_attach.py 91.22% <100.00%> (+5.11%) ⬆️
tests/unit/test_attach.py 100.00% <100.00%> (ø)

... and 5 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

if args.aggregate and not hasattr(args, "output"):
parser.error("Can't use aggregated mode without an output file.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This parser.error() never fires, because hasattr(args, "output") is always True.

Sometimes the attribute is None, but it always exists. If it didn't, we'd get an AttributeError on this line above:

if args.output:

We should add a unit test that exercises this case, too, so that we're not introducing a new, uncovered line of code.

Comment on lines 346 to 349
file_format = (
f"file_format={FileFormat.AGGREGATED_ALLOCATIONS}" if args.aggregate else ""
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oooh, this is interesting. This winds up passing file_format=1, because:

>>> FileFormat.AGGREGATED_ALLOCATIONS
<FileFormat.AGGREGATED_ALLOCATIONS: 1>
>>> f"file_format={FileFormat.AGGREGATED_ALLOCATIONS}"
'file_format=1'

I'm surprised that works. I think it would be more robust to do:

        file_format = (
            "file_format=memray.FileFormat.AGGREGATED_ALLOCATIONS" if args.aggregate else ""
        )

I'd rather pass around the enum by name than by value, in case a future version of the API changes the value associated with the enum, or changes the Tracker constructor to reject ints and only accept enum members.

I'd like to see an integration test that exercises this, too - that should be as easy as parametrizing our current attach test with a new flag for whether or not to use aggregated mode, passing --aggregate in the attach_cmd when the flag is set, and asserting that get_allocation_records() fails when --aggregate was passed but succeeds when it wasn't, and that get_high_watermark_allocation_records() works in both cases.

Copy link
Contributor

@godlygeek godlygeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we discussed offline I've modified the integration tests a bit, but otherwise this looks great to me! I'm going to squash and land this.

Add --aggregate option which allows user to request aggregated mode for
in-memory aggregation.

Signed-off-by: Ivona Stojanovic <istojanovic2@bloomberg.net>
@godlygeek godlygeek merged commit f1e2488 into bloomberg:main Sep 13, 2023
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants