[fleet installer] Split the `install` command in two #6766

andrewlock · 2025-03-13T18:16:00Z

Summary of changes

Splits the FleetInstaller.exe install command in two, and rename some commands

Reason for change

Currently "installing" the library and "instrumenting IIS" are tightly coupled; You have to do both. With this change, we split those responsibilities between different commands, which provides us additional flexibility in the future.

Implementation details

install has been split into two commands
- install-version creates the log file, installs into the GAC, and adds the crash tracking key. It has no dependencies on IIS
- enable-iis-instrumentation runs after install-version, and enables the IIS instrumentation.
reinstall command has been deleted, as it was unused.
uninstall-product was renamed to remove-iis-instrumentation (for symmetry)
uninstall-version is unchanged.

Test coverage

The split is covered by fleet installer smoke tests in this repo. The larger behaviour will be tracked by e2e tests

Other details

I considered having enable-instrumentation and a --type flag, but this seemed a bit easier, and I don't think we get much benefit from the alternative way, but happy to change it if others feel differently

datadog-datadog-prod-us1 · 2025-03-13T18:48:46Z

Datadog Report

Branch report: andrew/win-ssi/split-commands
Commit report: 068d18d
Test service: dd-trace-dotnet

✅ 0 Failed, 251953 Passed, 2412 Skipped, 21h 51m 4.91s Total Time
❄️ 2 New Flaky

New Flaky Tests (2)

OnSupportedFrameworkInSsi_CallsForwarderWithExpectedTelemetry - Datadog.Trace.ClrProfiler.IntegrationTests.InstrumentationTests - Last Failure
Expand for error
```
 The sample did not exit in 30000ms. Memory dump taken: True. Killing process.
```
OnSupportedFrameworkInSsi_CallsForwarderWithExpectedTelemetry - Datadog.Trace.ClrProfiler.IntegrationTests.InstrumentationTests - Last Failure
Expand for error
```
 The sample did not exit in 30000ms. Memory dump taken: True. Killing process.
```

andrewlock · 2025-03-13T18:50:39Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6766) - mean (69ms)  : 67, 72
     .   : milestone, 69,
    master - mean (70ms)  : 66, 73
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (6766) - mean (1,012ms)  : 986, 1038
     .   : milestone, 1012,
    master - mean (1,015ms)  : 992, 1037
     .   : milestone, 1015,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6766) - mean (103ms)  : 101, 105
     .   : milestone, 103,
    master - mean (103ms)  : 101, 105
     .   : milestone, 103,

    section CallTarget+Inlining+NGEN
    This PR (6766) - mean (694ms)  : 679, 708
     .   : milestone, 694,
    master - mean (688ms)  : 668, 709
     .   : milestone, 688,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6766) - mean (90ms)  : 88, 92
     .   : milestone, 90,
    master - mean (90ms)  : 88, 91
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (6766) - mean (645ms)  : 631, 659
     .   : milestone, 645,
    master - mean (645ms)  : 629, 661
     .   : milestone, 645,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6766) - mean (190ms)  : 186, 194
     .   : milestone, 190,
    master - mean (190ms)  : 185, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6766) - mean (1,101ms)  : 1071, 1130
     .   : milestone, 1101,
    master - mean (1,104ms)  : 1080, 1128
     .   : milestone, 1104,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6766) - mean (269ms)  : 264, 274
     .   : milestone, 269,
    master - mean (269ms)  : 265, 273
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (6766) - mean (875ms)  : 842, 908
     .   : milestone, 875,
    master - mean (875ms)  : 845, 906
     .   : milestone, 875,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6766) - mean (261ms)  : 257, 265
     .   : milestone, 261,
    master - mean (261ms)  : 258, 265
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (6766) - mean (858ms)  : 819, 897
     .   : milestone, 858,
    master - mean (857ms)  : 822, 893
     .   : milestone, 857,

andrewlock · 2025-03-13T19:10:16Z

Benchmarks Report for tracer 🐌

Benchmarks for #6766 compared to master:

4 benchmarks are faster, with geometric mean 1.146
1 benchmarks are slower, with geometric mean 1.114
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.08μs	45.8ns	318ns	0.0116	0.00387	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.1μs	54ns	275ns	0.0147	0.0049	0	5.8 KB
master	`StartStopWithChild`	net472	16μs	32.4ns	125ns	1.04	0.302	0.0953	6.21 KB
#6766	`StartStopWithChild`	net6.0	7.92μs	40.2ns	213ns	0.0156	0.0078	0	5.61 KB
#6766	`StartStopWithChild`	netcoreapp3.1	10.1μs	52.6ns	268ns	0.0298	0.0149	0	5.8 KB
#6766	`StartStopWithChild`	net472	16.2μs	37.4ns	145ns	1.04	0.314	0.0966	6.2 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	502μs	532ns	2.06μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	651μs	706ns	2.73μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	845μs	877ns	3.4μs	0.422	3.3 KB
#6766	`WriteAndFlushEnrichedTraces`	net6.0	503μs	494ns	1.91μs	0	2.7 KB
#6766	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	668μs	570ns	2.21μs	0	2.7 KB
#6766	`WriteAndFlushEnrichedTraces`	net472	834μs	1.41μs	5.29μs	0.417	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	135μs	287ns	1.07μs	0.134	14.47 KB
master	`SendRequest`	netcoreapp3.1	149μs	258ns	997ns	0.223	17.27 KB
master	`SendRequest`	net472	0.00108ns	0.000391ns	0.00146ns	0	0 b
#6766	`SendRequest`	net6.0	133μs	359ns	1.39μs	0.132	14.47 KB
#6766	`SendRequest`	netcoreapp3.1	152μs	144ns	559ns	0.226	17.27 KB
#6766	`SendRequest`	net472	0ns	0ns	0ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	567μs	2.67μs	10.3μs	0.566	0	0	41.66 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	693μs	3.73μs	21.7μs	0.361	0	0	41.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	845μs	3.86μs	14.9μs	8.81	2.52	0.419	53.31 KB
#6766	`WriteAndFlushEnrichedTraces`	net6.0	570μs	3.26μs	25.6μs	0.587	0	0	41.69 KB
#6766	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	722μs	4.2μs	38.7μs	0.329	0	0	41.76 KB
#6766	`WriteAndFlushEnrichedTraces`	net472	849μs	3.45μs	12.9μs	8.39	2.52	0.419	53.32 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.41μs	2.29ns	8.88ns	0.0139	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.8μs	1.48ns	5.52ns	0.0135	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.2μs	2.86ns	11.1ns	0.156	0.00109	987 B
#6766	`ExecuteNonQuery`	net6.0	1.35μs	1.54ns	5.95ns	0.0143	0	1.02 KB
#6766	`ExecuteNonQuery`	netcoreapp3.1	1.77μs	2.47ns	9.25ns	0.0134	0	1.02 KB
#6766	`ExecuteNonQuery`	net472	2.08μs	2.74ns	10.6ns	0.157	0.00104	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.25μs	1.05ns	3.77ns	0.0138	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.58μs	1.17ns	4.51ns	0.0125	976 B
master	`CallElasticsearch`	net472	2.62μs	2.15ns	8.32ns	0.158	995 B
master	`CallElasticsearchAsync`	net6.0	1.32μs	0.874ns	3.39ns	0.0133	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.7μs	0.724ns	2.8ns	0.0136	1.02 KB
master	`CallElasticsearchAsync`	net472	2.64μs	3.54ns	13.7ns	0.166	1.05 KB
#6766	`CallElasticsearch`	net6.0	1.25μs	0.469ns	1.82ns	0.0136	976 B
#6766	`CallElasticsearch`	netcoreapp3.1	1.67μs	0.796ns	2.98ns	0.0134	976 B
#6766	`CallElasticsearch`	net472	2.56μs	1.99ns	7.71ns	0.157	995 B
#6766	`CallElasticsearchAsync`	net6.0	1.36μs	0.568ns	2.13ns	0.0136	952 B
#6766	`CallElasticsearchAsync`	netcoreapp3.1	1.69μs	0.661ns	2.56ns	0.0135	1.02 KB
#6766	`CallElasticsearchAsync`	net472	2.74μs	1.14ns	4.42ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.36μs	3.04ns	11.8ns	0.0136	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.66μs	0.994ns	3.72ns	0.0124	952 B
master	`ExecuteAsync`	net472	1.81μs	0.574ns	2.15ns	0.145	915 B
#6766	`ExecuteAsync`	net6.0	1.41μs	2.25ns	8.4ns	0.0133	952 B
#6766	`ExecuteAsync`	netcoreapp3.1	1.6μs	0.639ns	2.39ns	0.0128	952 B
#6766	`ExecuteAsync`	net472	1.92μs	0.335ns	1.3ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.35μs	1.86ns	6.95ns	0.0325	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.5μs	1.92ns	7.44ns	0.0384	2.85 KB
master	`SendAsync`	net472	7.48μs	1.81ns	7ns	0.494	3.12 KB
#6766	`SendAsync`	net6.0	4.4μs	1.09ns	4.07ns	0.031	2.31 KB
#6766	`SendAsync`	netcoreapp3.1	5.26μs	3.97ns	14.8ns	0.0367	2.85 KB
#6766	`SendAsync`	net472	7.56μs	1.77ns	6.38ns	0.496	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6766

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1	1.143	2,423.58	2,120.59

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.5μs	0.728ns	2.73ns	0.0229	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.42μs	1.2ns	4.63ns	0.0218	1.64 KB
master	`EnrichedLog`	net472	2.42μs	0.772ns	2.99ns	0.249	1.57 KB
#6766	`EnrichedLog`	net6.0	1.48μs	0.886ns	3.2ns	0.0228	1.64 KB
#6766	`EnrichedLog`	netcoreapp3.1	2.12μs	1.05ns	3.79ns	0.0229	1.64 KB
#6766	`EnrichedLog`	net472	2.59μs	1.14ns	4.41ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	111μs	158ns	593ns	0.055	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	117μs	198ns	767ns	0	0	4.28 KB
master	`EnrichedLog`	net472	149μs	153ns	591ns	0.67	0.223	4.46 KB
#6766	`EnrichedLog`	net6.0	112μs	90.1ns	349ns	0.0558	0	4.28 KB
#6766	`EnrichedLog`	netcoreapp3.1	116μs	82.7ns	320ns	0.0575	0	4.28 KB
#6766	`EnrichedLog`	net472	149μs	73.1ns	283ns	0.67	0.223	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3μs	0.605ns	2.26ns	0.03	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.29μs	3.24ns	12.5ns	0.03	2.2 KB
master	`EnrichedLog`	net472	4.83μs	1.47ns	5.71ns	0.321	2.02 KB
#6766	`EnrichedLog`	net6.0	2.93μs	1.13ns	4.24ns	0.0307	2.2 KB
#6766	`EnrichedLog`	netcoreapp3.1	4.13μs	1.81ns	7ns	0.0289	2.2 KB
#6766	`EnrichedLog`	net472	4.68μs	2.29ns	8.57ns	0.321	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.35μs	0.731ns	2.83ns	0.0156	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.79μs	1.35ns	5.23ns	0.0152	1.14 KB
master	`SendReceive`	net472	2.09μs	1ns	3.88ns	0.183	1.16 KB
#6766	`SendReceive`	net6.0	1.4μs	0.535ns	2ns	0.016	1.14 KB
#6766	`SendReceive`	netcoreapp3.1	1.69μs	1.04ns	4.05ns	0.0152	1.14 KB
#6766	`SendReceive`	net472	2.16μs	0.845ns	3.27ns	0.184	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.77μs	1.32ns	4.95ns	0.0222	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.89μs	4.18ns	16.2ns	0.0213	1.65 KB
master	`EnrichedLog`	net472	4.43μs	1.91ns	7.4ns	0.323	2.04 KB
#6766	`EnrichedLog`	net6.0	2.81μs	0.759ns	2.94ns	0.0225	1.6 KB
#6766	`EnrichedLog`	netcoreapp3.1	4.06μs	1.42ns	4.92ns	0.0204	1.65 KB
#6766	`EnrichedLog`	net472	4.24μs	2.84ns	11ns	0.324	2.04 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6766

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1	1.182	775.94	656.23
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472	1.139	849.95	745.91
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472	1.120	690.11	615.94

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	455ns	0.131ns	0.506ns	0.00803	576 B
master	`StartFinishSpan`	netcoreapp3.1	595ns	0.381ns	1.48ns	0.00772	576 B
master	`StartFinishSpan`	net472	688ns	1.76ns	6.8ns	0.0918	578 B
master	`StartFinishScope`	net6.0	484ns	0.0714ns	0.277ns	0.00988	696 B
master	`StartFinishScope`	netcoreapp3.1	776ns	0.455ns	1.76ns	0.00936	696 B
master	`StartFinishScope`	net472	850ns	0.291ns	1.13ns	0.104	658 B
#6766	`StartFinishSpan`	net6.0	426ns	0.341ns	1.32ns	0.0081	576 B
#6766	`StartFinishSpan`	netcoreapp3.1	571ns	0.268ns	1.04ns	0.00761	576 B
#6766	`StartFinishSpan`	net472	616ns	0.397ns	1.49ns	0.0917	578 B
#6766	`StartFinishScope`	net6.0	480ns	0.179ns	0.693ns	0.00982	696 B
#6766	`StartFinishScope`	netcoreapp3.1	656ns	0.189ns	0.732ns	0.00953	696 B
#6766	`StartFinishScope`	net472	747ns	0.696ns	2.7ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6766

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0	1.114	653.37	727.97

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	654ns	0.411ns	1.48ns	0.0098	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	943ns	1.32ns	5.12ns	0.00899	696 B
master	`RunOnMethodBegin`	net472	1.12μs	0.417ns	1.62ns	0.104	658 B
#6766	`RunOnMethodBegin`	net6.0	728ns	0.301ns	1.17ns	0.00978	696 B
#6766	`RunOnMethodBegin`	netcoreapp3.1	943ns	0.64ns	2.48ns	0.00939	696 B
#6766	`RunOnMethodBegin`	net472	1.09μs	0.345ns	1.34ns	0.104	658 B

bouwkast

LGTM 👍

Split the install command in two

068d18d

andrewlock added the area:ssi label Mar 13, 2025

andrewlock requested review from a team as code owners March 13, 2025 18:16

bouwkast approved these changes Mar 14, 2025

View reviewed changes

andrewlock merged commit af5a1fa into master Mar 14, 2025
131 of 134 checks passed

andrewlock deleted the andrew/win-ssi/split-commands branch March 14, 2025 14:37

github-actions bot added this to the vNext-v3 milestone Mar 14, 2025

bmermet mentioned this pull request Mar 17, 2025

(fleet) split installation and instrumentation for dotnet tracer oci DataDog/datadog-agent#35152

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fleet installer] Split the `install` command in two #6766

[fleet installer] Split the `install` command in two #6766

andrewlock commented Mar 13, 2025

datadog-datadog-prod-us1 bot commented Mar 13, 2025 •

edited

Loading

andrewlock commented Mar 13, 2025

andrewlock commented Mar 13, 2025

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6766

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6766

Raw results

Slower ⚠️ in #6766

Raw results

bouwkast left a comment

[fleet installer] Split the install command in two #6766

[fleet installer] Split the install command in two #6766

Conversation

andrewlock commented Mar 13, 2025

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-datadog-prod-us1 bot commented Mar 13, 2025 • edited Loading

Datadog Report

New Flaky Tests (2)

andrewlock commented Mar 13, 2025

Execution-Time Benchmarks Report ⏱️

andrewlock commented Mar 13, 2025

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6766

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6766

Raw results

Slower ⚠️ in #6766

Raw results

bouwkast left a comment

Choose a reason for hiding this comment

[fleet installer] Split the `install` command in two #6766

[fleet installer] Split the `install` command in two #6766

datadog-datadog-prod-us1 bot commented Mar 13, 2025 •

edited

Loading