Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add --debug option for printing mimalloc memory stats #822

Merged
merged 5 commits into from
Nov 22, 2022

Conversation

fukusuket
Copy link
Collaborator

What Changed

Evidence

Enviroment

  • Windows 10 Home edtion

Test1

If -h is specified, --debug description will be printed.

PS C:\tmp\hayabusa-1.9.0-dev-win-64-bit> .\hayabusa.exe -h

Hayabusa 1.8.1
Yamato Security (https://github.com/Yamato-Security/hayabusa) @SecurityYamato)

USAGE:
    hayabusa.exe <INPUT> [OTHER-ACTIONS] [OPTIONS]

OPTIONS:
    -h, --help       Print help information
        --version    Print version information
...

DISPLAY-SETTINGS:
        --debug                 Print debug information (memory usage, etc...)
        --no-color              Disable color output
        --no-summary            Do not display result summary
    -q, --quiet                 Quiet mode: do not display the launch banner
    -v, --verbose               Output verbose information
    -V, --visualize-timeline    Output event frequency timeline

Test2

If --debug is specified, memory statistics will be printed.

PS C:\tmp\hayabusa-1.9.0-dev-win-64-bit> .\hayabusa.exe -d C:\tmp\hayabusa-sample-evtx\ -o out.csv --debug -q
Start time: 2022/11/22 21:26

Analyzing event files: 580
Total file size: 136.9 MB
...

Results Summary:

Events with hits / Total events: 19,566 / 47,458 (Data reduction: 27,892 events (58.77%))

Total | Unique detections: 32,745 | 590
Total | Unique critical detections: 47 (0.14%) | 19 (3.22%)
Total | Unique high detections: 6,239 (19.05%) | 267 (45.25%)
Total | Unique medium detections: 1,585 (4.84%) | 176 (29.83%)
Total | Unique low detections: 6,626 (20.24%) | 75 (12.71%)
Total | Unique informational detections: 18,248 (55.73%) | 53 (8.98%)
...

Elapsed time: 00:00:17.320
Saved file: out.csv (16.3 MB)

Memory usage stats:
heap stats:    peak      total      freed    current       unit      count
  reserved:    2.6 GiB    2.6 GiB      0        2.6 GiB                        not all freed!
 committed:    1.7 GiB    2.0 GiB  406.6 MiB    1.6 GiB                        not all freed!
     reset:      0          0          0          0                            ok
   touched:  128.5 KiB    3.1 MiB    4.1 GiB   -4.0 GiB                        ok
  segments:     18         25         21          4                            not all freed!
-abandoned:      0          0          0          0                            ok
   -cached:      0          0          0          0                            ok
     pages:      0          0       47.2 Ki   -47.2 Ki                         ok
-abandoned:      0          0          0          0                            ok
 -extended:      0
 -noretire:      0
     mmaps:      0
   commits:    471
   threads:     16         16          0         16                            not all freed!
  searches:     0.0 avg
numa nodes:       1
   elapsed:      17.408 s
   process: user: 84.437 s, system: 1.843 s, faults: 441765, rss: 1.6 GiB, commit: 1.7 GiB

Test3

If --debug is not specified, memory statistics is not printed.

PS C:\tmp\hayabusa-1.9.0-dev-win-64-bit> .\hayabusa.exe -d C:\tmp\hayabusa-sample-evtx\ -o out.csv -q
Start time: 2022/11/22 21:29

Analyzing event files: 580
Total file size: 136.9 MB
...

Results Summary:

Events with hits / Total events: 19,566 / 47,458 (Data reduction: 27,892 events (58.77%))

Total | Unique detections: 32,745 | 590
Total | Unique critical detections: 47 (0.14%) | 19 (3.22%)
Total | Unique high detections: 6,239 (19.05%) | 267 (45.25%)
Total | Unique medium detections: 1,585 (4.84%) | 176 (29.83%)
Total | Unique low detections: 6,626 (20.24%) | 75 (12.71%)
Total | Unique informational detections: 18,248 (55.73%) | 53 (8.98%)

...

Elapsed time: 00:00:17.062
Saved file: out.csv (16.3 MB)

PS C:\tmp\hayabusa-1.9.0-dev-win-64-bit>

I would appreciate it if you could review🙏

@fukusuket
Copy link
Collaborator Author

fukusuket commented Nov 22, 2022

Test4

Memory stats can be output on macOS Ventura(Apple M1).

スクリーンショット 2022-11-22 21 44 47

@hitenkoku hitenkoku self-requested a review November 22, 2022 12:50
@fukusuket
Copy link
Collaborator Author

Test5

Memory stats can be output on Ubuntu 22.04(ARM64).

スクリーンショット 2022-11-22 21 49 26

Copy link
Collaborator

@hitenkoku hitenkoku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fukusuket Thank you for your pull request.

LGTM.

I will merge after I wrote changelog and CI test passed.

@hitenkoku hitenkoku linked an issue Nov 22, 2022 that may be closed by this pull request
@hitenkoku hitenkoku added the enhancement New feature or request label Nov 22, 2022
@hitenkoku hitenkoku merged commit 02e1b05 into main Nov 22, 2022
@fukusuket
Copy link
Collaborator Author

Thanks for the quick review and update changelog😁

@fukusuket fukusuket deleted the 788-print-mimalloc-stats-info branch November 22, 2022 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enhancement: Print Maximum memory usage after scan
2 participants