Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: benchmarking code improvements #630

Merged
merged 29 commits into from
Dec 31, 2021
Merged

perf: benchmarking code improvements #630

merged 29 commits into from
Dec 31, 2021

Conversation

petethepig
Copy link
Member

@petethepig petethepig commented Dec 18, 2021

improves existing benchmarking code, adds pull mode benchmarks.


To run pull benchmarks run:

# 100 is number of clients that are being simulated
./start 100

You should be able to see:

/cc @abeaumont @kolesnikovae you all might be interested in pull mode benchmark

@github-actions
Copy link
Contributor

github-actions bot commented Dec 18, 2021

size-limit report 📦

Path Size Loading time (3g) Running time (snapdragon) Total time
webapp/public/assets/app.js 291.94 KB (0%) 5.9 s (0%) 752 ms (+2.19% 🔺) 6.6 s

@codecov
Copy link

codecov bot commented Dec 18, 2021

Codecov Report

Merging #630 (8172b64) into main (35ce0b5) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #630   +/-   ##
=======================================
  Coverage   75.76%   75.76%           
=======================================
  Files          45       45           
  Lines        1588     1588           
  Branches      292      292           
=======================================
  Hits         1203     1203           
  Misses        356      356           
  Partials       29       29           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 35ce0b5...8172b64. Read the comment docs.

@pyroscopebot
Copy link
Collaborator

pyroscopebot commented Dec 18, 2021

Server Benchmark

Details
Name Value
BENCH_RUN_FOR 10m
PYROBENCH_RAND_SEED 2306912
PYROBENCH_PROFILE_WIDTH 20
PYROBENCH_PROFILE_DEPTH 20
PYROBENCH_PROFILE_SYMBOL_LENGTH 30
PYROBENCH_APPS 20
PYROBENCH_CLIENTS 20
PYROBENCH_REQUESTS 10000

Result

main pr diff threshold
throughput 131.10 131.00 -0.10 (-0.08%) 5%
total items processed 95505.00 95074.00 -431.00 (-0.45%) 5%
Details
Name Description Query for main Query for pr
throughput rate(pyroscope_http_request_duration_seconds_count{handler="/ingest", instance="pyroscope:4040"}[5m]) rate(pyroscope_http_request_duration_seconds_count{handler="/ingest", instance="pyroscope_main:4040"}[5m])
total items processed pyroscope_http_request_duration_seconds_count{handler="/ingest", instance="pyroscope:4040"} pyroscope_http_request_duration_seconds_count{handler="/ingest", instance="pyroscope_main:4040"}

Screenshots

Throughput Throughput
Disk Usage Disk Usage
Memory Memory
Upload Errors (Total) Upload Errors (Total)
Successful Uploads (Total) Successful Uploads (Total)
CPU Utilization CPU Utilization

Generated by 🚫 dangerJS against e90c2e9

@pyroscopebot
Copy link
Collaborator

pyroscopebot commented Dec 21, 2021

Server Pull Benchmark

Details
Name Value
BENCH_RUN_FOR 10m
PYROBENCH_RAND_SEED 2306912
PYROBENCH_PROFILE_WIDTH 20
PYROBENCH_PROFILE_DEPTH 20
PYROBENCH_PROFILE_SYMBOL_LENGTH 30
PYROBENCH_APPS 20
PYROBENCH_CLIENTS 20
PYROBENCH_REQUESTS 10000

Result

main pr diff threshold
memory usage 1953376952.00 2015114952.00 61738000.00 (3.11%) 5%
Details
Name Description Query for main Query for pr
memory usage go_memstats_sys_bytes{instance="pyroscope:4040"} go_memstats_sys_bytes{instance="pyroscope_main:4040"}

Screenshots

Throughput Throughput
Disk Usage Disk Usage
Memory Memory
Upload Errors (Total) Upload Errors (Total)
Successful Uploads (Total) Successful Uploads (Total)
CPU Utilization CPU Utilization

Generated by 🚫 dangerJS against e90c2e9

Comment on lines 1 to 11
---
# log-level: debug
scrape-configs:
- job-name: testing
enabled-profiles: [cpu, mem]
static-configs:
- application: pull-target
targets:
- pull-target:4042
labels:
pod: pod-0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could even create a fake discovery mechanism that would produce targets (with unique scrape url).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that would be useful here, although generating these configs a is also pretty easy.

What do you mean by unique scraping URL? Is it important for them to have different ones?

@abeaumont
Copy link
Contributor

This is a great help when working on performance improvements!

@petethepig petethepig merged commit 3aa460c into main Dec 31, 2021
@petethepig petethepig deleted the bench-capacity branch December 31, 2021 11:54
kolesnikovae pushed a commit that referenced this pull request Jan 7, 2022
* adds benchmarking code

* wip

* pull benchmarks

* adds more info to readme

* benchmark improvements

* no extra logging

* fix

* improvements

* fix

* changes

* lint fix

* initial version of pr-pull report

* fixes

* fix

* fix

* fix

* fix

* fix

* report improvements

* fix

* fix

* wip: unifying benchmark scripts

* more benchmark improvements

* updates to benchmark

* cleanup

* fixes

* fix
korniltsev pushed a commit that referenced this pull request Jul 18, 2023
Features:
* Adds an Application Selector to enable users to make more targeted queries by default, similar to OG Pyroscope.
* Users can still write their own complex queries if desired.
* Supports both "pyroscope_app" and "service_name" as indexes.

Caveats:
* Utilizes the /querier.v1.QuerierService/Series to create a list of apps, which returns more data than necessary.
* Only returns data that is currently in memory, specifically recently ingested apps. (Related to [ui] 'ProfileID/Applications' only shows data that has been ingested recently #630)
* Parsing an "App" from a non-trivial query (e.g., using !~) does not function correctly. In this PR, it primarily affects the dropdown population, which should not match a query like "cpu{app="myapp.*"}" to an "App" accurately.
* Does not preserve tags when switching between apps. For example, if the current query is "cpu{mytag="foo", pyroscope_app="myapp"}" and the user clicks on "myapp2", even if "myapp2" shares the exact tags, the new query will completely remove "mytag="foo"": "cpu{pyroscope_app="myapp2"}".
* The "pyroscope_app/service_name" tag is present in the app selector.
* There is no filtering mechanism similar to OG Pyroscope.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants