Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adjust log level or error messages written by repo indexer #6556

Merged
merged 1 commit into from
Jan 25, 2024
Merged

Adjust log level or error messages written by repo indexer #6556

merged 1 commit into from
Jan 25, 2024

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Jan 25, 2024 •
edited by jira bot
Loading

What Does This Do

Changes log level of repo indexing error messages from ERROR to DEBUG.

Motivation

Logging something as error implies that something is wrong with the application/build - ie. either the result can't be trusted or the system will eventually fail.
In the case of repo indexing errors, the issue breaks neither the tests, nor the tracer. Moreover, it is something that the users have no influence over, so we're effectively spamming their logs with errors they cannot remove.

Jira ticket: CIVIS-8777

@nikita-tkachenko-datadog nikita-tkachenko-datadog added tag: no release notes Changes to exclude from release notes comp: ci visibility Continuous Integration Visibility labels Jan 25, 2024
@pr-commenter
Copy link

pr-commenter bot commented Jan 25, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/adjust-repo-indexer-log-level
git_commit_date 1706187092 1706188038
git_commit_sha e2c012d f9bc124
release_version 1.29.0-SNAPSHOT~e2c012df9c 1.29.0-SNAPSHOT~f9bc124cbf
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1706190959 1706190959
ci_job_id 418461262 418461262
ci_pipeline_id 27174582 27174582
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 9 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.29.0-SNAPSHOT~f9bc124cbf, baseline=1.29.0-SNAPSHOT~e2c012df9c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1064435
Total [baseline] (9.39 s) : 0, 9389773
Agent [candidate] (1.064 s) : 0, 1063883
Total [candidate] (9.351 s) : 0, 9350776
section appsec
Agent [baseline] (1.153 s) : 0, 1152933
Total [baseline] (9.468 s) : 0, 9467607
Agent [candidate] (1.155 s) : 0, 1154729
Total [candidate] (9.502 s) : 0, 9501991
section iast
Agent [baseline] (1.187 s) : 0, 1186945
Total [baseline] (9.73 s) : 0, 9730268
Agent [candidate] (1.177 s) : 0, 1176992
Total [candidate] (9.688 s) : 0, 9687750
section profiling
Agent [baseline] (1.279 s) : 0, 1278533
Total [baseline] (9.596 s) : 0, 9595945
Agent [candidate] (1.285 s) : 0, 1284576
Total [candidate] (9.636 s) : 0, 9636181
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent appsec 1.153 s 88.498 ms (8.3%)
Agent iast 1.187 s 122.51 ms (11.5%)
Agent profiling 1.279 s 214.098 ms (20.1%)
Total tracing 9.39 s -
Total appsec 9.468 s 77.834 ms (0.8%)
Total iast 9.73 s 340.495 ms (3.6%)
Total profiling 9.596 s 206.172 ms (2.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent appsec 1.155 s 90.846 ms (8.5%)
Agent iast 1.177 s 113.109 ms (10.6%)
Agent profiling 1.285 s 220.693 ms (20.7%)
Total tracing 9.351 s -
Total appsec 9.502 s 151.214 ms (1.6%)
Total iast 9.688 s 336.974 ms (3.6%)
Total profiling 9.636 s 285.405 ms (3.1%) 
gantt
    title petclinic - break down per module: candidate=1.29.0-SNAPSHOT~f9bc124cbf, baseline=1.29.0-SNAPSHOT~e2c012df9c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (671.337 ms) : 0, 671337
BytebuddyAgent [candidate] (670.427 ms) : 0, 670427
GlobalTracer [baseline] (297.568 ms) : 0, 297568
GlobalTracer [candidate] (298.667 ms) : 0, 298667
AppSec [baseline] (52.689 ms) : 0, 52689
AppSec [candidate] (52.061 ms) : 0, 52061
Remote Config [baseline] (694.999 µs) : 0, 695
Remote Config [candidate] (701.739 µs) : 0, 702
Telemetry [baseline] (7.599 ms) : 0, 7599
Telemetry [candidate] (7.55 ms) : 0, 7550
section appsec
BytebuddyAgent [baseline] (665.392 ms) : 0, 665392
BytebuddyAgent [candidate] (666.184 ms) : 0, 666184
GlobalTracer [baseline] (295.784 ms) : 0, 295784
GlobalTracer [candidate] (296.435 ms) : 0, 296435
AppSec [baseline] (149.936 ms) : 0, 149936
AppSec [candidate] (150.442 ms) : 0, 150442
Remote Config [baseline] (689.801 µs) : 0, 690
Remote Config [candidate] (692.243 µs) : 0, 692
Telemetry [baseline] (6.778 ms) : 0, 6778
Telemetry [candidate] (6.759 ms) : 0, 6759
section iast
BytebuddyAgent [baseline] (781.241 ms) : 0, 781241
BytebuddyAgent [candidate] (774.422 ms) : 0, 774422
GlobalTracer [baseline] (288.034 ms) : 0, 288034
GlobalTracer [candidate] (286.042 ms) : 0, 286042
AppSec [baseline] (53.737 ms) : 0, 53737
AppSec [candidate] (55.074 ms) : 0, 55074
Remote Config [baseline] (618.863 µs) : 0, 619
Remote Config [candidate] (585.786 µs) : 0, 586
Telemetry [baseline] (6.778 ms) : 0, 6778
Telemetry [candidate] (6.579 ms) : 0, 6579
IAST [baseline] (21.98 ms) : 0, 21980
IAST [candidate] (20.023 ms) : 0, 20023
section profiling
ProfilingAgent [baseline] (123.355 ms) : 0, 123355
ProfilingAgent [candidate] (124.046 ms) : 0, 124046
BytebuddyAgent [baseline] (663.53 ms) : 0, 663530
BytebuddyAgent [candidate] (666.728 ms) : 0, 666728
GlobalTracer [baseline] (376.685 ms) : 0, 376685
GlobalTracer [candidate] (378.523 ms) : 0, 378523
AppSec [baseline] (52.281 ms) : 0, 52281
AppSec [candidate] (52.473 ms) : 0, 52473
Remote Config [baseline] (668.679 µs) : 0, 669
Remote Config [candidate] (674.734 µs) : 0, 675
Telemetry [baseline] (7.492 ms) : 0, 7492
Telemetry [candidate] (7.537 ms) : 0, 7537
Profiling [baseline] (123.379 ms) : 0, 123379
Profiling [candidate] (124.071 ms) : 0, 124071
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-01-25T13:31:55 2024-01-25T13:50:53
git_branch master nikita-tkachenko/adjust-repo-indexer-log-level
git_commit_date 1706187092 1706188038
git_commit_sha e2c012d f9bc124
release_version 1.29.0-SNAPSHOT~e2c012df9c 1.29.0-SNAPSHOT~f9bc124cbf
start_time 2024-01-25T13:31:41 2024-01-25T13:50:40
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1706190959 1706190959
ci_job_id 418461262 418461262
ci_pipeline_id 27174582 27174582
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 10 metrics, 15 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling worse
[+35.894µs; +91.407µs] or [+2.355%; +5.997%]		 unstable
[-650.310op/s; +422.390op/s] or [-21.135%; +13.728%]		 1.588ms 2962.963op/s 1.524ms 3076.923op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.29.0-SNAPSHOT~f9bc124cbf, baseline=1.29.0-SNAPSHOT~e2c012df9c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.367 ms) : 1348, 1386
.   : milestone, 1367,
appsec (1.774 ms) : 1749, 1800
.   : milestone, 1774,
iast (1.548 ms) : 1523, 1572
.   : milestone, 1548,
profiling (1.524 ms) : 1499, 1550
.   : milestone, 1524,
tracing (1.519 ms) : 1493, 1545
.   : milestone, 1519,
section candidate
no_agent (1.386 ms) : 1366, 1405
.   : milestone, 1386,
appsec (1.767 ms) : 1741, 1794
.   : milestone, 1767,
iast (1.506 ms) : 1482, 1530
.   : milestone, 1506,
profiling (1.588 ms) : 1562, 1614
.   : milestone, 1588,
tracing (1.516 ms) : 1491, 1541
.   : milestone, 1516,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.367 ms [1.348 ms, 1.386 ms] -
appsec 1.774 ms [1.749 ms, 1.8 ms] 407.708 µs (29.8%)
iast 1.548 ms [1.523 ms, 1.572 ms] 180.845 µs (13.2%)
profiling 1.524 ms [1.499 ms, 1.55 ms] 157.492 µs (11.5%)
tracing 1.519 ms [1.493 ms, 1.545 ms] 152.323 µs (11.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.386 ms [1.366 ms, 1.405 ms] -
appsec 1.767 ms [1.741 ms, 1.794 ms] 381.708 µs (27.5%)
iast 1.506 ms [1.482 ms, 1.53 ms] 120.4 µs (8.7%)
profiling 1.588 ms [1.562 ms, 1.614 ms] 202.195 µs (14.6%)
tracing 1.516 ms [1.491 ms, 1.541 ms] 130.722 µs (9.4%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.29.0-SNAPSHOT~f9bc124cbf, baseline=1.29.0-SNAPSHOT~e2c012df9c
    dateFormat X
    axisFormat %s
section baseline
no_agent (376.095 µs) : 354, 398
.   : milestone, 376,
iast (484.175 µs) : 463, 505
.   : milestone, 484,
iast_FULL (546.349 µs) : 526, 567
.   : milestone, 546,
iast_GLOBAL (511.457 µs) : 491, 532
.   : milestone, 511,
iast_HARDCODED_SECRET_DISABLED (476.18 µs) : 456, 497
.   : milestone, 476,
iast_INACTIVE (458.465 µs) : 438, 479
.   : milestone, 458,
iast_TELEMETRY_OFF (477.059 µs) : 457, 497
.   : milestone, 477,
tracing (451.568 µs) : 430, 473
.   : milestone, 452,
section candidate
no_agent (367.452 µs) : 347, 388
.   : milestone, 367,
iast (492.825 µs) : 472, 513
.   : milestone, 493,
iast_FULL (540.91 µs) : 520, 562
.   : milestone, 541,
iast_GLOBAL (509.463 µs) : 488, 531
.   : milestone, 509,
iast_HARDCODED_SECRET_DISABLED (472.551 µs) : 452, 493
.   : milestone, 473,
iast_INACTIVE (453.884 µs) : 433, 475
.   : milestone, 454,
iast_TELEMETRY_OFF (478.903 µs) : 458, 500
.   : milestone, 479,
tracing (454.722 µs) : 433, 476
.   : milestone, 455,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 376.095 µs [354.16 µs, 398.031 µs] -
iast 484.175 µs [463.073 µs, 505.277 µs] 108.08 µs (28.7%)
iast_FULL 546.349 µs [525.63 µs, 567.067 µs] 170.254 µs (45.3%)
iast_GLOBAL 511.457 µs [490.66 µs, 532.255 µs] 135.362 µs (36.0%)
iast_HARDCODED_SECRET_DISABLED 476.18 µs [455.719 µs, 496.641 µs] 100.085 µs (26.6%)
iast_INACTIVE 458.465 µs [437.505 µs, 479.426 µs] 82.37 µs (21.9%)
iast_TELEMETRY_OFF 477.059 µs [456.81 µs, 497.307 µs] 100.963 µs (26.8%)
tracing 451.568 µs [430.283 µs, 472.853 µs] 75.473 µs (20.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.452 µs [347.05 µs, 387.853 µs] -
iast 492.825 µs [472.281 µs, 513.368 µs] 125.373 µs (34.1%)
iast_FULL 540.91 µs [520.265 µs, 561.554 µs] 173.458 µs (47.2%)
iast_GLOBAL 509.463 µs [488.051 µs, 530.875 µs] 142.011 µs (38.6%)
iast_HARDCODED_SECRET_DISABLED 472.551 µs [451.845 µs, 493.257 µs] 105.099 µs (28.6%)
iast_INACTIVE 453.884 µs [433.083 µs, 474.686 µs] 86.433 µs (23.5%)
iast_TELEMETRY_OFF 478.903 µs [458.033 µs, 499.773 µs] 111.451 µs (30.3%)
tracing 454.722 µs [433.127 µs, 476.318 µs] 87.27 µs (23.8%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review January 25, 2024 16:54
@nikita-tkachenko-datadog nikita-tkachenko-datadog removed the tag: no release notes Changes to exclude from release notes label Jan 25, 2024
@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 399faee into master Jan 25, 2024
82 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/adjust-repo-indexer-log-level branch January 25, 2024 16:57
@github-actions github-actions bot added this to the 1.29.0 milestone Jan 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@romainkomorndatadog romainkomorndatadog romainkomorndatadog approved these changes

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility
Projects
None yet
Milestone
1.29.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@nikita-tkachenko-datadog @romainkomorndatadog