Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement telemetry for shell git commands #6665

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Feb 12, 2024

What Does This Do

Implements sending telemetry metrics for shell Git commands executed in CI Visibility module.

Motivation

Monitoring CI Visibility behaviour to detect unexpected errors/problems.

Jira ticket: CIVIS-2427

@nikita-tkachenko-datadog nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Feb 12, 2024
@nikita-tkachenko-datadog nikita-tkachenko-datadog force-pushed the nikita-tkachenko/ci-vis-telemetry branch 2 times, most recently from 5001f17 to 8936a3f Compare February 12, 2024 10:32
@nikita-tkachenko-datadog nikita-tkachenko-datadog force-pushed the nikita-tkachenko/ci-vis-git-commands-telemetry branch 4 times, most recently from f7b8297 to 4d417ef Compare February 12, 2024 13:30
@pr-commenter
Copy link

pr-commenter bot commented Feb 12, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/ci-vis-git-commands-telemetry
git_commit_date 1707845538 1707845617
git_commit_sha c471269 1321c26
release_version 1.31.0-SNAPSHOT~c471269d77 1.31.0-SNAPSHOT~1321c2606c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1707848541 1707848541
ci_job_id 433064026 433064026
ci_pipeline_id 28225380 28225380
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 10 unstable metrics.

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-02-13T17:58:28 2024-02-13T18:17:20
git_branch master nikita-tkachenko/ci-vis-git-commands-telemetry
git_commit_date 1707845538 1707845617
git_commit_sha c471269 1321c26
release_version 1.31.0-SNAPSHOT~c471269d77 1.31.0-SNAPSHOT~1321c2606c
start_time 2024-02-13T17:58:15 2024-02-13T18:17:07
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1707848541 1707848541
ci_job_id 433064026 433064026
ci_pipeline_id 28225380 28225380
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 16 unstable metrics.

Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~1321c2606c, baseline=1.31.0-SNAPSHOT~c471269d77
    dateFormat X
    axisFormat %s
section baseline
no_agent (359.48 µs) : 340, 379
.   : milestone, 359,
iast (464.66 µs) : 444, 485
.   : milestone, 465,
iast_FULL (531.248 µs) : 511, 552
.   : milestone, 531,
iast_GLOBAL (490.434 µs) : 469, 512
.   : milestone, 490,
iast_HARDCODED_SECRET_DISABLED (467.305 µs) : 447, 488
.   : milestone, 467,
iast_INACTIVE (442.249 µs) : 421, 464
.   : milestone, 442,
iast_TELEMETRY_OFF (474.005 µs) : 452, 496
.   : milestone, 474,
tracing (444.294 µs) : 423, 465
.   : milestone, 444,
section candidate
no_agent (364.109 µs) : 345, 384
.   : milestone, 364,
iast (478.283 µs) : 457, 499
.   : milestone, 478,
iast_FULL (528.253 µs) : 508, 549
.   : milestone, 528,
iast_GLOBAL (496.181 µs) : 474, 518
.   : milestone, 496,
iast_HARDCODED_SECRET_DISABLED (469.625 µs) : 449, 491
.   : milestone, 470,
iast_INACTIVE (442.282 µs) : 422, 463
.   : milestone, 442,
iast_TELEMETRY_OFF (472.326 µs) : 452, 493
.   : milestone, 472,
tracing (442.29 µs) : 421, 464
.   : milestone, 442,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 359.48 µs [339.676 µs, 379.284 µs] -
iast 464.66 µs [444.168 µs, 485.152 µs] 105.18 µs (29.3%)
iast_FULL 531.248 µs [510.581 µs, 551.914 µs] 171.768 µs (47.8%)
iast_GLOBAL 490.434 µs [468.848 µs, 512.02 µs] 130.954 µs (36.4%)
iast_HARDCODED_SECRET_DISABLED 467.305 µs [446.599 µs, 488.011 µs] 107.825 µs (30.0%)
iast_INACTIVE 442.249 µs [420.979 µs, 463.52 µs] 82.769 µs (23.0%)
iast_TELEMETRY_OFF 474.005 µs [452.41 µs, 495.6 µs] 114.525 µs (31.9%)
tracing 444.294 µs [423.241 µs, 465.348 µs] 84.814 µs (23.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.109 µs [344.503 µs, 383.715 µs] -
iast 478.283 µs [457.114 µs, 499.453 µs] 114.174 µs (31.4%)
iast_FULL 528.253 µs [507.847 µs, 548.659 µs] 164.144 µs (45.1%)
iast_GLOBAL 496.181 µs [474.176 µs, 518.185 µs] 132.072 µs (36.3%)
iast_HARDCODED_SECRET_DISABLED 469.625 µs [448.697 µs, 490.553 µs] 105.516 µs (29.0%)
iast_INACTIVE 442.282 µs [421.553 µs, 463.011 µs] 78.173 µs (21.5%)
iast_TELEMETRY_OFF 472.326 µs [451.714 µs, 492.939 µs] 108.218 µs (29.7%)
tracing 442.29 µs [420.505 µs, 464.076 µs] 78.182 µs (21.5%)
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~1321c2606c, baseline=1.31.0-SNAPSHOT~c471269d77
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.341 ms) : 1323, 1360
.   : milestone, 1341,
appsec (1.752 ms) : 1726, 1777
.   : milestone, 1752,
iast (1.505 ms) : 1480, 1529
.   : milestone, 1505,
profiling (1.53 ms) : 1505, 1555
.   : milestone, 1530,
tracing (1.475 ms) : 1450, 1499
.   : milestone, 1475,
section candidate
no_agent (1.355 ms) : 1336, 1374
.   : milestone, 1355,
appsec (1.774 ms) : 1748, 1799
.   : milestone, 1774,
iast (1.504 ms) : 1479, 1528
.   : milestone, 1504,
profiling (1.522 ms) : 1496, 1548
.   : milestone, 1522,
tracing (1.48 ms) : 1455, 1505
.   : milestone, 1480,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.341 ms [1.323 ms, 1.36 ms] -
appsec 1.752 ms [1.726 ms, 1.777 ms] 410.023 µs (30.6%)
iast 1.505 ms [1.48 ms, 1.529 ms] 163.058 µs (12.2%)
profiling 1.53 ms [1.505 ms, 1.555 ms] 188.732 µs (14.1%)
tracing 1.475 ms [1.45 ms, 1.499 ms] 133.226 µs (9.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.336 ms, 1.374 ms] -
appsec 1.774 ms [1.748 ms, 1.799 ms] 418.73 µs (30.9%)
iast 1.504 ms [1.479 ms, 1.528 ms] 148.453 µs (11.0%)
profiling 1.522 ms [1.496 ms, 1.548 ms] 166.832 µs (12.3%)
tracing 1.48 ms [1.455 ms, 1.505 ms] 124.744 µs (9.2%)

Base automatically changed from nikita-tkachenko/ci-vis-telemetry to master February 13, 2024 17:32
@nikita-tkachenko-datadog nikita-tkachenko-datadog force-pushed the nikita-tkachenko/ci-vis-git-commands-telemetry branch from 4d417ef to 1321c26 Compare February 13, 2024 17:33
@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit c96efd6 into master Feb 13, 2024
79 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/ci-vis-git-commands-telemetry branch February 13, 2024 19:21
@github-actions github-actions bot added this to the 1.30.0 milestone Feb 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp: ci visibility Continuous Integration Visibility
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants