@@ -288,27 +223,27 @@ As an example, the CollectTracing command to EventPipe (explained below) encodes
command
reserved
circularBufferMB
-
outputPath Length
-
outputPath String
+
format
n Providers
Keywords
logLevel
provider_name length
-
provider_name string
+
provider_name string
+
arguments length
"DOTNET_IPC_V1"
-
78
+
80
0x0202
0x0000
250
-
16
-
"/tmp/foo.nettrace"
+
1
1
100
2
14
-
"MyEventSource"
+
"MyEventSource"
+
0
@@ -351,6 +286,7 @@ enum class EventPipeCommandId : uint8_t
CollectTracing2 = 0x03, // create/start a given session with/without rundown
CollectTracing3 = 0x04, // create/start a given session with/without collecting stacks
CollectTracing4 = 0x05, // create/start a given session with specific rundown keyword
+ CollectTracing5 = 0x06, // create/start a given session with/without user_events
}
```
See: [EventPipe Commands](#EventPipe-Commands)
@@ -359,7 +295,9 @@ See: [EventPipe Commands](#EventPipe-Commands)
enum class DumpCommandId : uint8_t
{
// reserved = 0x00,
- CreateCoreDump = 0x01,
+ GenerateCoreDump = 0x01,
+ GenerateCoreDump2 = 0x02,
+ GenerateCoreDump3 = 0x03,
// future
}
```
@@ -370,6 +308,7 @@ enum class ProfilerCommandId : uint8_t
{
// reserved = 0x00,
AttachProfiler = 0x01,
+ StartupProfiler = 0x02,
// future
}
```
@@ -378,14 +317,15 @@ See: [Profiler Commands](#Profiler-Commands)
```c++
enum class ProcessCommandId : uint8_t
{
- ProcessInfo = 0x00,
- ResumeRuntime = 0x01,
- ProcessEnvironment = 0x02,
- ProcessInfo2 = 0x04,
- EnablePerfMap = 0x05,
- DisablePerfMap = 0x06,
- ApplyStartupHook = 0x07
- ProcessInfo3 = 0x08,
+ ProcessInfo = 0x00,
+ ResumeRuntime = 0x01,
+ ProcessEnvironment = 0x02,
+ SetEnvironmentVariable = 0x03,
+ ProcessInfo2 = 0x04,
+ EnablePerfMap = 0x05,
+ DisablePerfMap = 0x06,
+ ApplyStartupHook = 0x07
+ ProcessInfo3 = 0x08,
// future
}
```
@@ -397,6 +337,7 @@ For example, the Command to start a stream session with EventPipe would be `0x02
## EventPipe Commands
+### EventPipe Command IDs
```c++
enum class EventPipeCommandId : uint8_t
{
@@ -406,6 +347,7 @@ enum class EventPipeCommandId : uint8_t
CollectTracing2 = 0x03, // create/start a given session with/without rundown
CollectTracing3 = 0x04, // create/start a given session with/without collecting stacks
CollectTracing4 = 0x05, // create/start a given session with specific rundown keyword
+ CollectTracing5 = 0x06, // create/start a given session with/without user_events
}
```
EventPipe Payloads are encoded with the following rules:
@@ -414,10 +356,50 @@ EventPipe Payloads are encoded with the following rules:
* `uint` = 4 little endian bytes
* `ulong` = 8 little endian bytes
* `wchar` = 2 little endian bytes, UTF16 encoding
-* `byte` = 1 unsigned little endian byte
+* `byte` = 1 unsigned byte
+* `bool` = 1 unsigned byte
* `array` = uint length, length # of `T`s
* `string` = (`array` where the last `wchar` must = `0`) or (length = `0`)
+### `StopTracing`
+
+Command Code: `0x0201`
+
+The `StopTracing` command is used to stop a specific EventPipe session. Clients are expected to use this command to stop EventPipe sessions started with [`CollectStreaming`](#CollectStreaming).
+
+#### Inputs:
+
+Header: `{ Magic; 28; 0x0201; 0x0000 }`
+
+Payload:
+* `ulong sessionId`: The ID for the EventPipe session to stop
+
+#### Returns:
+
+Header: `{ Magic; 28; 0xFF00; 0x0000 }`
+
+Payload:
+* `ulong sessionId`: the ID for the EventPipe session that was stopped
+
+
+##### Details:
+
+Inputs:
+```c
+Payload
+{
+ ulong sessionId
+}
+```
+
+Returns:
+```c
+Payload
+{
+ ulong sessionId
+}
+```
+
### `CollectTracing`
Command Code: `0x0202`
@@ -432,17 +414,18 @@ If the stream is stopped prematurely due to a client or server error, the `nettr
#### Inputs:
-Header: `{ Magic; Size; 0x0202; 0x0000 }`
+Header: `{ Magic; 20 + Payload Size; 0x0202; 0x0000 }`
+Payload:
* `uint circularBufferMB`: The size of the circular buffer used for buffering event data while streaming
-* `uint format`: 0 for the legacy NetPerf format and 1 for the NetTrace format
+* `uint format`: 0 for the legacy NetPerf format and 1 for the NetTrace V4 format
* `array providers`: The providers to turn on for the streaming session
A `provider_config` is composed of the following data:
* `ulong keywords`: The keywords to turn on with this providers
* `uint logLevel`: The level of information to turn on
* `string provider_name`: The name of the provider
-* `string filter_data` (optional): Filter information
+* `string arguments`: (Key-value pairs string to pass to the provider) or (length = 0)
> see ETW documentation for a more detailed explanation of Keywords, Filters, and Log Level.
@@ -469,7 +452,7 @@ provider_config
ulong keywords,
uint logLevel,
string provider_name,
- string filter_data (optional)
+ string arguments
}
```
@@ -490,10 +473,10 @@ The `CollectTracing2` command is an extension of the `CollectTracing` command -
#### Inputs:
-Header: `{ Magic; Size; 0x0203; 0x0000 }`
+Header: `{ Magic; 20 + Payload Size; 0x0203; 0x0000 }`
* `uint circularBufferMB`: The size of the circular buffer used for buffering event data while streaming
-* `uint format`: 0 for the legacy NetPerf format and 1 for the NetTrace format
+* `uint format`: 0 for the legacy NetPerf format and 1 for the NetTrace V4 format
* `bool requestRundown`: Indicates whether rundown should be fired by the runtime.
* `array providers`: The providers to turn on for the streaming session
@@ -501,7 +484,7 @@ A `provider_config` is composed of the following data:
* `ulong keywords`: The keywords to turn on with this providers
* `uint logLevel`: The level of information to turn on
* `string provider_name`: The name of the provider
-* `string filter_data` (optional): Filter information
+* `string arguments`: (Key-value pairs string to pass to the provider) or (length = 0)
> see ETW documentation for a more detailed explanation of Keywords, Filters, and Log Level.
>
@@ -529,7 +512,7 @@ provider_config
ulong keywords,
uint logLevel,
string provider_name,
- string filter_data (optional)
+ string arguments
}
```
@@ -550,10 +533,10 @@ The `CollectTracing3` command is an extension of the `CollectTracing2` command -
#### Inputs:
-Header: `{ Magic; Size; 0x0203; 0x0000 }`
+Header: `{ Magic; 20 + Payload Size; 0x0203; 0x0000 }`
* `uint circularBufferMB`: The size of the circular buffer used for buffering event data while streaming
-* `uint format`: 0 for the legacy NetPerf format and 1 for the NetTrace format
+* `uint format`: 0 for the legacy NetPerf format and 1 for the NetTrace V4 format
* `bool requestRundown`: Indicates whether rundown should be fired by the runtime.
* `bool requestStackwalk`: Indicates whether stacktrace information should be recorded.
* `array providers`: The providers to turn on for the streaming session
@@ -562,7 +545,7 @@ A `provider_config` is composed of the following data:
* `ulong keywords`: The keywords to turn on with this providers
* `uint logLevel`: The level of information to turn on
* `string provider_name`: The name of the provider
-* `string filter_data` (optional): Filter information
+* `string arguments`: (Key-value pairs string to pass to the provider) or (length = 0)
> see ETW documentation for a more detailed explanation of Keywords, Filters, and Log Level.
>
@@ -591,7 +574,7 @@ provider_config
ulong keywords,
uint logLevel,
string provider_name,
- string filter_data (optional)
+ string arguments
}
```
@@ -608,7 +591,7 @@ Followed by an Optional Continuation of a `nettrace` format stream of events.
Command Code: `0x0205`
-The `CollectTracing4` command is an extension of the `CollectTracing3` command - its behavior is the same as `CollectTracing3` command, except the requestRundown field is replaced by the rundownKeyword field to allow customizing the set of rundown events to be fired.
+The `CollectTracing4` command is an extension of the `CollectTracing3` command - its behavior is the same as `CollectTracing3` command, except the requestRundown field is replaced by the rundownKeyword field to allow customizing the set of rundown events to be fired.
A rundown keyword of `0x80020139` has the equivalent behavior as `CollectTracing3` with `requestRundown=true` and rundown keyword of `0` has the equivalent behavior as `requestRundown=false`.
@@ -617,18 +600,20 @@ A rundown keyword of `0x80020139` has the equivalent behavior as `CollectTracing
#### Inputs:
-Header: `{ Magic; Size; 0x0205; 0x0000 }`
+Header: `{ Magic; 20 + Payload Size; 0x0205; 0x0000 }`
+Payload:
* `uint circularBufferMB`: The size of the circular buffer used for buffering event data while streaming
-* `uint format`: 0 for the legacy NetPerf format and 1 for the NetTrace format
+* `uint format`: 0 for the legacy NetPerf format and 1 for the NetTrace V4 format
* `ulong rundownKeyword`: Indicates the keyword for the rundown provider
+* `bool requestStackwalk`: Indicates whether stacktrace information should be recorded.
* `array providers`: The providers to turn on for the streaming session
A `provider_config` is composed of the following data:
* `ulong keywords`: The keywords to turn on with this providers
* `uint logLevel`: The level of information to turn on
* `string provider_name`: The name of the provider
-* `string filter_data` (optional): Filter information
+* `string arguments`: (Key-value pairs string to pass to the provider) or (length = 0)
> see ETW documentation for a more detailed explanation of Keywords, Filters, and Log Level.
>
@@ -656,7 +641,7 @@ provider_config
ulong keywords,
uint logLevel,
string provider_name,
- string filter_data (optional)
+ string arguments
}
```
@@ -669,43 +654,459 @@ Payload
```
Followed by an Optional Continuation of a `nettrace` format stream of events.
-### `StopTracing`
+### `CollectTracing5`
-Command Code: `0x0201`
+Command Code: `0x0206`
-The `StopTracing` command is used to stop a specific streaming session. Clients are expected to use this command to stop streaming sessions started with [`CollectStreaming`](#CollectStreaming).
+The `CollectTracing5` command is an extension of the `CollectTracing4` command. It has all the capabilities of `CollectTracing4` and introduces new fields to enable a Linux-only user_events-based eventpipe session and to prescribe an enable/disable list for Event IDs. When the user_events-based eventpipe session is enabled, the file descriptor and SCM_RIGHTS of the `user_events_data` file must be sent through the optional continuation stream as [described](#passing_file_descriptor). The runtime will register tracepoints based on the provider configurations passed in, and runtime events will be written directly to the `user_events_data` file descriptor. The enable/disable list of Event IDs will apply after the keyword/level filter to determine whether or not that provider's event will be written.
+
+> Note available for .NET 10.0 and later.
#### Inputs:
-Header: `{ Magic; 28; 0x0201; 0x0000 }`
+Header: `{ Magic; 20 + Payload Size; 0x0206; 0x0000 }`
-* `ulong sessionId`: The ID for the streaming session to stop
+#### Streaming Session Payload:
+* `uint session_type`: 0
+* `uint streaming_circularBufferMB`: Specifies the size of the Streaming session's circular buffer used for buffering event data.
+* `uint streaming_format`: 0 for the legacy NetPerf format and 1 for the NetTrace V4 format. Specifies the format in which event data will be serialized into the IPC Stream
+* `ulong rundownKeyword`: Indicates the keyword for the rundown provider
+* `bool requestStackwalk`: Indicates whether stacktrace information should be recorded.
+* `array providers`: The providers to turn on for the session
-#### Returns:
+The `streaming_provider_config` is composed of the following data:
+* `ulong keywords`: The keywords to turn on with this provider
+* `uint logLevel`: The level of information to turn on
+* `string provider_name`: The name of the provider
+* `string arguments`: (Key-value pairs string to pass to the provider) or (length = 0)
+* `event_filter filter`: Rules for filtering this provider's Event IDs, applied after `keyword`/`logLevel`, using an enable/disable list or (length = `0`).
-Header: `{ Magic; 28; 0xFF00; 0x0000 }`
+An `event_filter` is comprised of the following data:
+* `bool enable`: 0 to disable events, 1 to enable events
+* `array event_ids`: List of Event IDs to disable or enable.
+
+See [event_filter serialization examples](#event_filter)
-* `ulong sessionId`: the ID for the streaming session that was stopped
+#### User_events Session Payload:
+* `uint session_type`: 1
+* `ulong rundownKeyword`: Indicates the keyword for the rundown provider
+* `array providers`: The providers to turn on for the session
+The `user_events_provider_config` is composed of the following data:
+* `ulong keywords`: The keywords to turn on with this provider
+* `uint logLevel`: The level of information to turn on
+* `string provider_name`: The name of the provider
+* `string arguments`: (Key-value pairs string to pass to the provider) or (length = 0)
+* `event_filter filter`: Rules for filtering this provider's Event IDs, applied after `keyword`/`logLevel`, using an enable/disable list or (length = `0`).
+* `tracepoint_config config`: Maps Event IDs to tracepoints. If an Event ID is excluded by `event_filter`, it will not be written to any tracepoint.
-##### Details:
+An `event_filter` is comprised of the following data:
+* `bool enable`: 0 to disable events, 1 to enable events
+* `array event_ids`: List of Event IDs to disable or enable.
-Inputs:
-```c
-Payload
-{
- ulong sessionId
-}
+See [event_filter serialization examples](#event_filter)
+
+A `tracepoint_config` is comprised of the following data:
+* `string default_tracepoint_name`: (The default tracepoint filtered Event IDs will be written to unless otherwise specified by `tracepoints`) or (length = `0` to only write to tracepoints specified in `tracepoints`)
+* `array tracepoints`: Specifies alternate tracepoints for a set of Event IDs to be written to instead of the default tracepoint or (length = `0`).
+
+A `tracepoint_set` is comprised of the following data:
+* `string tracepoint_name`: The tracepoint that the following subset of Event IDs should be written to.
+* `array event_ids`: The Event IDs to be written to `tracepoint_name`.
+
+With a user_events session, atleast one of `default_tracepoint_name` and `tracepoints` must be specified. An error will be returned through the stream if both are length = `0`.
+Event IDs specified in `tracepoint_set`s must be exclusive. If an Event ID is detected in different `tracepoint_set`s of the provider, an error will be returned through the stream.
+
+See [tracepoint_config serialization examples](#tracepoint_config)
+
+> See ETW documentation for a more detailed explanation of Keywords, Filters, and Log Level.
+
+#### Returns (as an IPC Message Payload):
+
+Header: `{ Magic; 28; 0xFF00; 0x0000; }`
+
+`CollectTracing5` returns:
+* `ulong sessionId`: the ID for the EventPipe Session started
+
+A Streaming Session started with `CollectTracing5` is followed by an Optional Continuation of a `nettrace` format stream of events.
+
+A User_events Session started with `CollectTracing5` expects the Optional Continuation to contain another message passing along the SCM_RIGHTS `user_events_data` file descriptor. See [details](#passing_file_descriptor)
+
+## EventPipe Payload Serialization Examples
+
+### Event_filter
+Example `event_filter` serialization. Serializing
+```
+enable=0, event_ids=[]
+Disable Nothing === Enable all events.
+```
+
+
+
1
+
2-5
+
+
+
+
bool
+
array<uint>
+
+
+
+
enable
+
event_ids
+
+
+
0
+
0
+
+
+
+```
+enable=0, event_ids=[4, 5]
+Disable only Event IDs 4 and 5 === Enable all Event IDs except 4 and 5
```
-Returns:
-```c
-Payload
-{
- ulong sessionId
-}
+
+
+```
+enable=1, event_ids=[1, 2, 3]
+Enable only EventIDs 1, 2, and 3 === Disable all EventIDs except 1, 2, and 3.
+```
+
+
+
+
1
+
2
+
6
+
10
+
14-17
+
+
+
+
bool
+
array<uint>
+
+
+
+
enable
+
event_ids
+
+
+
1
+
3
+
1
+
2
+
3
+
+
+
+### Tracepoint_config
+Example `tracepoint_config` serialization
+```
+session_type=0, Streaming Sessions DO NOT encode bytes for tracepoint_config
+session_type=1, encode bytes for tracepoint_config
+```
+
+```
+All enabled Event IDs will be written to a default "MyTracepoint" tracepoint
+```
+
+
+
1
+
5
+
33-36
+
+
+
+
string (array<wchar>)
+
array<uint>
+
+
+
default_tracepoint_name
+
tracepoints
+
+
+
14
+
"MyTracepoint"
+
0
+
+
+
+```
+Enabled Event IDs 1 - 9 will be written to tracepoint "LowEvents".
+All other enabled Event IDs will be written to "MyTracepoint"
+```
+
+
+
1
+
5
+
33
+
37
+
41
+
61
+
65
+
69
+
73
+
77
+
81
+
85
+
89
+
93
+
97-100
+
+
+
+
string (array<wchar>)
+
uint
+
string (array<wchar>)
+
array<uint>
+
+
+
+
+
default_tracepoint_name
+
tracepoints
+
tracepoint_name
+
event_ids
+
+
+
14
+
"MyTracepoint"
+
1
+
10
+
"LowEvents"
+
9
+
1
+
2
+
3
+
4
+
5
+
6
+
7
+
8
+
9
+
+
+
+```
+Enabled Event IDs 1 - 9 will be written to tracepoint "LowEvents".
+No default tracepoint needed, don't write any other enabled Event IDs
+```
+
+
+
1
+
5
+
9
+
13
+
33
+
37
+
41
+
45
+
49
+
53
+
57
+
61
+
65
+
69-72
+
+
+
+
string (array<wchar>)
+
uint
+
string (array<wchar>)
+
array<uint>
+
+
+
+
+
default_tracepoint_name
+
tracepoints
+
tracepoint_name
+
event_ids
+
+
+
0
+
1
+
10
+
"LowEvents"
+
9
+
1
+
2
+
3
+
4
+
5
+
6
+
7
+
8
+
9
+
+
+
+### passing_file_descriptor
+
+> Note: This only applies to enabling an user_event-based EventPipe session, which is specifically a Linux feature
+
+To register [user_event](https://docs.kernel.org/trace/user_events.html) tracepoints and write events, access to the root protected `user_events_data` file is required. Once the .NET Runtime's Diagnostic Server processes a [CollectTracing5](#collecttracing5) command specifying the `user_events` format (`session_type=1`), it expects that the client will send a file descriptor to the [continuation stream](#general-flow) via SCM_RIGHTS.
+
+```C
+#include
+#include
+
+struct msghdr {
+ void * msg_name; /* ignored by runtime */
+ unsigned int msg_namelen; /* ignored by runtime */
+ struct iovec * msg_iov; /* runtime will "parse" 1 byte */
+ unsigned int msg_iovlen; /* runtime will "parse" one msg_iov */
+ void * msg_control; /* ancillary data */
+ unsigned int msg_controllen; /* ancillary data buffer len */
+ int msg_flags; /* ignored by runtime */
+};
+
+struct cmsghdr {
+ unsigned int cmsg_len; /* length of control message */
+ int cmsg_level; /* SOL_SOCKET */
+ int cmsg_type; /* SCM_RIGHTS */
+ int cmsg_data[0]; /* file descriptor */
+};
+```
+
+For parsing the file descriptor passed with SCM_RIGHTS, the runtime will `recvmsg` the message and only care about the control message containing ancillary data. It will read one byte from the `msg_iov` buffer just to receive the ancillary data, but it will disregard the contents of the `msg_iov` buffers.
+
+### User_events format
+
+#### User_events Registration
+
+Once the runtime has received the configured tracepoint names as detailed under [tracepoint_config](#user_events-session-payload), it uses the [file descriptor passed in the continuation stream](#passing_file_descriptor) to register the prescribed tracepoint names following the [user_events registering protocol](https://docs.kernel.org/trace/user_events.html#registering). The runtime will construct a `user_reg` struct for every tracepoint name, defaulting to using none of the `user_reg` flags, so the resulting command format will be as follows:
+
+##### Tracepoint Format V1
+
+` u8 version; u16 event_id; __rel_loc u8[] extension; __rel_loc u8[] payload`
+
+See [user_events writing](#user_events-writing) below for field details`.
+
+#### User_events Writing
+
+When writing events to their mapped user_events tracepoints prescribed by the `tracepoint_config` in the [User_events session payload](#user_events-session-payload), the runtime will adapt the [user_events writing protocol](https://docs.kernel.org/trace/user_events.html#writing-data) to write the event as:
+
+```
+struct iovec io[7];
+
+io[0].iov_base = &write_index; // __u32 tracepoint write index from registration
+io[0].iov_len = sizeof(write_index);
+io[1].iov_base = &version; // __u8 tracepoint format version
+io[1].iov_len = sizeof(version);
+io[2].iov_base = &event_id; // __u16 EventID defined by EventSource/native manifest
+io[2].iov_len = sizeof(event_id);
+io[3].iov_base = &extension; // __rel_loc u8[] optional event information
+io[3].iov_base = sizeof(extension);
+io[4].iov_base = &payload; // __rel_loc u8[] event payload
+io[4].iov_len = sizeof(payload);
+io[6].iov_base = &data; // __u8[] data
+io[6].iov_len = data_len;
+
+writev(ep_session->data_fd, (const struct iovec *)io, 7);
+```
+
+The `__rel_loc` is the relative dynamic array attribute described [here](https://lwn.net/Articles/876682/).
+
+The `write_index` is the tracepoint's write index determined during tracepoint registration.
+
+The `version` is the version of the tracepoint format, which in this case is [version 1](#tracepoint-format-v1).
+
+The `event_id` is the ID of the event, defined by the EventSource/native manifest.
+
+#### Extension Blob Format
+
+The `extension` field is an optional data blob that can provide additional information about an event. Its structure is as follows:
+
+1. **Label** (`byte`): Indicates the type of data that follows.
+2. **Data**: The content, whose format depends on the label.
+
+**Label Values and Corresponding Data:**
+
+| Label | Meaning | Data Format | Description |
+|--------|------------------------|----------------------------|-----------------------------------------------------------------------------|
+| 0x01 | Event Metadata | `array metadata` | Contains event metadata, formatted per NetTrace v5. |
+| 0x02 | ActivityId | `uint16 guid` | Contains the GUID for the ActivityId. |
+| 0x03 | RelatedActivityId | `uint16 guid` | Contains the GUID for the RelatedActivityId. |
+
+**Details:**
+- The extension blob may be empty if no extra information is present.
+- Multiple extension blobs can be concatenated if more than one piece of information is needed. Each blob starts with its own label byte.
+- For Event Metadata (`0x01`), the `metadata` array matches the NetTrace v5 PayloadBytes format.
+- For ActivityId and RelatedActivityId (`0x02`, `0x03`), the `guid` is a 16-byte value representing the GUID.
+- The size of the entire extension blob can be inferred from the extension `__rel_loc` field. See the [__rel_loc documentation](https://lwn.net/Articles/876682/) for more details.
+
+**Example Layout:**
+
+```
+[Label][Data][Label][Data]...
+```
+
+For example, an extension blob containing both Event Metadata and ActivityId would look like:
+- `[0x01][metadata][0x02][guid]`
+
+**Notes:**
+- The runtime includes Event Metadata only the first time an event is sent in a session.
+- Native runtime events do not include metadata.
+
+The `payload` points at a blob of data with the same format as an EventPipe payload – the concatenated encoded values for all the parameters.
+
+The `metadata` either points at nothing if the event doesn’t have metadata, or it points at a metadata blob matching the NetTrace version 5 formatting convention. Specifically it is the data that would be stored inside the PayloadBytes area of an event blob within a MetadataBlock described [here](https://github.com/microsoft/perfview/blob/main/src/TraceEvent/EventPipe/NetTraceFormat_v5.md#metadata-event-encoding).
+
+> NOTE: V5 and V6 metadata formats have the same info, but they aren’t formatted identically. Parsing and reserialization is required to convert between the two.
+
+### Which events have metadata?
+
+The runtime will keep track per-session whether it has sent a particular event before. The first time each event is sent during a session, metadata will be included, and otherwise, it will be left empty. As a special case, runtime events currently implemented in native code will never send metadata.
+
## Dump Commands
### `CreateCoreDump`
diff --git a/eng/Build-Native.cmd b/eng/Build-Native.cmd
index 28f101cf53..c6349242c7 100644
--- a/eng/Build-Native.cmd
+++ b/eng/Build-Native.cmd
@@ -9,24 +9,6 @@ echo %__MsgPrefix%Starting Build at %TIME%
set __ThisScriptFull="%~f0"
set __ThisScriptDir="%~dp0"
-call "%__ThisScriptDir%"\native\init-vs-env.cmd
-if NOT '%ERRORLEVEL%' == '0' goto ExitWithError
-
-if defined VS170COMNTOOLS (
- set "__VSToolsRoot=%VS170COMNTOOLS%"
- set "__VCToolsRoot=%VS170COMNTOOLS%\..\..\VC\Auxiliary\Build"
- set __VSVersion=vs2022
-)
-if defined VS160COMNTOOLS (
- set "__VSToolsRoot=%VS160COMNTOOLS%"
- set "__VCToolsRoot=%VS160COMNTOOLS%\..\..\VC\Auxiliary\Build"
- set __VSVersion=vs2019
-) else if defined VS150COMNTOOLS (
- set "__VSToolsRoot=%VS150COMNTOOLS%"
- set "__VCToolsRoot=%VS150COMNTOOLS%\..\..\VC\Auxiliary\Build"
- set __VSVersion=vs2017
-)
-
:: Set the default arguments for build
set __TargetArch=x64
@@ -39,14 +21,14 @@ set __TargetOS=Windows_NT
set __BuildNative=1
set __CI=0
set __Verbosity=minimal
-set __Ninja=0
+set __Ninja=1
:: Set the various build properties here so that CMake and MSBuild can pick them up
-set "__ProjectDir=%~dp0"
-:: remove trailing slash
-if %__ProjectDir:~-1%==\ set "__ProjectDir=%__ProjectDir:~0,-1%"
-set "__ProjectDir=%__ProjectDir%\.."
-set "__SourceDir=%__ProjectDir%\src"
+set "__RepoRootDir=%~dp0"
+:: remove trailing slash
+if %__RepoRootDir:~-1%==\ set "__RepoRootDir=%__RepoRootDir:~0,-1%"
+set "__RepoRootDir=%__RepoRootDir%\.."
+set "__SourceDir=%__RepoRootDir%\src"
:: __UnprocessedBuildArgs are args that we pass to msbuild (e.g. /p:OfficialBuildId=xxxxxx)
set "__args=%*"
@@ -64,6 +46,8 @@ if /i "%1" == "--help" goto Usage
if /i "%1" == "-configuration" (set __BuildType=%2&set processedArgs=!processedArgs! %1 %2&shift&shift&goto Arg_Loop)
if /i "%1" == "-architecture" (set __TargetArch=%2&set processedArgs=!processedArgs! %1 %2&shift&shift&goto Arg_Loop)
if /i "%1" == "-verbosity" (set __Verbosity=%2&set processedArgs=!processedArgs! %1 %2&shift&shift&goto Arg_Loop)
+if /i "%1" == "-msbuild" (set __Ninja=0&set processedArgs=!processedArgs! %1&shift&goto Arg_Loop)
+if /i "%1" == "-ninja" (set __Ninja=1&set processedArgs=!processedArgs! %1&shift&goto Arg_Loop)
if /i "%1" == "-ci" (set __CI=1&set processedArgs=!processedArgs! %1&shift&goto Arg_Loop)
:: These options are ignored for a native build
@@ -89,13 +73,20 @@ if [!processedArgs!] == [] (
:ArgsDone
+call "%__RepoRootDir%"\eng\native\init-vs-env.cmd
+if NOT '%ERRORLEVEL%' == '0' goto ExitWithError
+
+if defined VCINSTALLDIR (
+ set "__VCToolsRoot=%VCINSTALLDIR%Auxiliary\Build"
+)
+
if "%__HostArch%" == "" set __HostArch=%__TargetArch%
if /i "%__BuildType%" == "debug" set __BuildType=Debug
if /i "%__BuildType%" == "release" set __BuildType=Release
if "%NUGET_PACKAGES%" == "" (
if %__CI% EQU 1 (
- set "NUGET_PACKAGES=%__ProjectDir%\.packages"
+ set "NUGET_PACKAGES=%__RepoRootDir%\.packages"
) else (
set "NUGET_PACKAGES=%UserProfile%\.nuget\packages"
)
@@ -104,7 +95,7 @@ if "%NUGET_PACKAGES%" == "" (
echo %NUGET_PACKAGES%
:: Set the remaining variables based upon the determined build configuration
-set "__RootBinDir=%__ProjectDir%\artifacts"
+set "__RootBinDir=%__RepoRootDir%\artifacts"
set "__BinDir=%__RootBinDir%\bin\%__TargetOS%.%__TargetArch%.%__BuildType%"
set "__LogDir=%__RootBinDir%\log\%__TargetOS%.%__TargetArch%.%__BuildType%"
set "__ArtifactsIntermediatesDir=%__RootBinDir%\obj"
@@ -116,7 +107,7 @@ set "__CMakeBinDir=%__BinDir%"
set "__CMakeBinDir=%__CMakeBinDir:\=/%"
:: Common msbuild arguments
-set "__CommonBuildArgs=/v:!__Verbosity! /p:Configuration=%__BuildType% /p:BuildArch=%__TargetArch% %__UnprocessedBuildArgs%"
+set "__CommonBuildArgs=/v:!__Verbosity! /p:Configuration=%__BuildType% /p:TargetOS=%__TargetOS% /p:TargetArch=%__TargetArch% %__UnprocessedBuildArgs%"
if not exist "%__BinDir%" md "%__BinDir%"
if not exist "%__IntermediatesDir%" md "%__IntermediatesDir%"
@@ -128,7 +119,7 @@ echo %__MsgPrefix%Commencing diagnostics repo build
echo %__MsgPrefix%Checking prerequisites
:: Eval the output from probe-win1.ps1
-for /f "delims=" %%a in ('powershell -NoProfile -ExecutionPolicy ByPass "& ""%__ProjectDir%\eng\native\set-cmake-path.ps1"""') do %%a
+for /f "delims=" %%a in ('powershell -NoProfile -ExecutionPolicy ByPass "& ""%__RepoRootDir%\eng\native\set-cmake-path.ps1"""') do %%a
REM =========================================================================================
REM ===
@@ -139,7 +130,7 @@ REM ============================================================================
@if defined _echo @echo on
:: Parse the optdata package versions out of msbuild so that we can pass them on to CMake
-set __DotNetCli=%__ProjectDir%\dotnet.cmd
+set __DotNetCli=%__RepoRootDir%\dotnet.cmd
REM =========================================================================================
REM ===
@@ -173,13 +164,9 @@ if %__BuildNative% EQU 1 (
goto ExitWithError
)
- if %__Ninja% EQU 1 (
- set __ExtraCmakeArgs="-DCMAKE_BUILD_TYPE=!__BuildType!"
- )
-
echo Generating Version Header
set __GenerateVersionLog="%__LogDir%\GenerateVersion.binlog"
- powershell -NoProfile -ExecutionPolicy ByPass -NoLogo -File "%__ProjectDir%\eng\common\msbuild.ps1" "%__ProjectDir%\eng\native-prereqs.proj" /bl:!__GenerateVersionLog! /t:BuildPrereqs /restore %__CommonBuildArgs%
+ powershell -NoProfile -ExecutionPolicy ByPass -NoLogo -File "%__RepoRootDir%\eng\common\msbuild.ps1" "%__RepoRootDir%\eng\native-prereqs.proj" /bl:!__GenerateVersionLog! /t:Build /restore %__CommonBuildArgs%
if not !errorlevel! == 0 (
echo Generate Version Header FAILED
goto ExitWithError
@@ -188,12 +175,14 @@ if %__BuildNative% EQU 1 (
echo %__MsgPrefix%Regenerating the Visual Studio solution
- set "__ManagedBinaryDir=%__RootBinDir%\bin"
- set "__ManagedBinaryDir=!__ManagedBinaryDir:\=/!"
- set __ExtraCmakeArgs=!__ExtraCmakeArgs! "-DCMAKE_SYSTEM_VERSION=10.0" "-DCLR_MANAGED_BINARY_DIR=!__ManagedBinaryDir!" "-DCLR_BUILD_TYPE=%__BuildType%" "-DCLR_CMAKE_TARGET_ARCH=%__TargetArch%" "-DNUGET_PACKAGES=%NUGET_PACKAGES:\=/%"
+ if %__Ninja% EQU 1 (
+ set __ExtraCmakeArgs="-DCMAKE_BUILD_TYPE=!__BuildType!"
+ )
+
+ set __ExtraCmakeArgs=!__ExtraCmakeArgs! "-DCMAKE_SYSTEM_VERSION=10.0" "-DCLR_BUILD_TYPE=%__BuildType%" "-DCLR_CMAKE_TARGET_ARCH=%__TargetArch%"
pushd "%__IntermediatesDir%"
- call "%__ProjectDir%\eng\native\gen-buildsys.cmd" "%__ProjectDir%" "%__IntermediatesDir%" %__VSVersion% %__HostArch% %__TargetOS% !__ExtraCmakeArgs!
+ call "%__RepoRootDir%\eng\native\gen-buildsys.cmd" "%__RepoRootDir%" "%__IntermediatesDir%" %VisualStudioVersion% %__HostArch% %__TargetOS% !__ExtraCmakeArgs!
@if defined _echo @echo on
popd
@@ -205,9 +194,16 @@ if %__BuildNative% EQU 1 (
goto ExitWithError
)
set __BuildLog="%__LogDir%\Native.Build.binlog"
+ set __CmakeBuildToolArgs=
+ if %__Ninja% EQU 1 (
+ set __CmakeBuildToolArgs=
+ ) else (
+ REM We pass the /m flag directly to MSBuild so that we can get both MSBuild and CL parallelism, which is fastest for our builds.
+ set __CmakeBuildToolArgs=/bl:!__BuildLog! !__CommonBuildArgs!
+ )
- echo running "%CMakePath%" --build %__IntermediatesDir% --target install --config %__BuildType% -- /bl:!__BuildLog! !__CommonBuildArgs!
- "%CMakePath%" --build %__IntermediatesDir% --target install --config %__BuildType% -- /bl:!__BuildLog! !__CommonBuildArgs!
+ echo running "%CMakePath%" --build %__IntermediatesDir% --target install --config %__BuildType% -- !__CmakeBuildToolArgs!
+ "%CMakePath%" --build %__IntermediatesDir% --target install --config %__BuildType% -- !__CmakeBuildToolArgs!
if not !ERRORLEVEL! == 0 (
echo %__MsgPrefix%Error: native component build failed. Refer to the build log files for details:
@@ -220,20 +216,6 @@ if %__BuildNative% EQU 1 (
endlocal
)
-REM Copy the native SOS binaries to where these tools expect for CI & VS testing
-
-set "__targetRid=net8.0"
-set "__dotnet_sos=%__RootBinDir%\bin\dotnet-sos\%__BuildType%\%__targetRid%"
-set "__dotnet_dump=%__RootBinDir%\bin\dotnet-dump\%__BuildType%\%__targetRid%"
-mkdir %__dotnet_sos%\win-%__TargetArch%
-mkdir %__dotnet_sos%\publish\win-%__TargetArch%
-mkdir %__dotnet_dump%\win-%__TargetArch%
-mkdir %__dotnet_dump%\publish\win-%__TargetArch%
-xcopy /y /q /i %__BinDir% %__dotnet_sos%\win-%__TargetArch%
-xcopy /y /q /i %__BinDir% %__dotnet_sos%\publish\win-%__TargetArch%
-xcopy /y /q /i %__BinDir% %__dotnet_dump%\win-%__TargetArch%
-xcopy /y /q /i %__BinDir% %__dotnet_dump%\publish\win-%__TargetArch%
-
REM =========================================================================================
REM ===
REM === All builds complete!
diff --git a/eng/Build.props b/eng/Build.props
index 370baace2d..61f52242a2 100644
--- a/eng/Build.props
+++ b/eng/Build.props
@@ -1,5 +1,12 @@
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/eng/CdacPackageItems.props b/eng/CdacPackageItems.props
new file mode 100644
index 0000000000..65e548d057
--- /dev/null
+++ b/eng/CdacPackageItems.props
@@ -0,0 +1,24 @@
+
+
+
+ <_cdacPackageVersion Condition="'$(TargetRid)' == 'win-x64'">$(runtimewinx64MicrosoftDotNetCdacTransportVersion)
+ <_cdacPackageVersion Condition="'$(TargetRid)' == 'win-arm64'">$(runtimewinarm64MicrosoftDotNetCdacTransportVersion)
+ <_cdacPackageVersion Condition="'$(TargetRid)' == 'linux-x64'">$(runtimelinuxx64MicrosoftDotNetCdacTransportVersion)
+ <_cdacPackageVersion Condition="'$(TargetRid)' == 'linux-arm64'">$(runtimelinuxarm64MicrosoftDotNetCdacTransportVersion)
+ <_cdacPackageVersion Condition="'$(TargetRid)' == 'osx-x64'">$(runtimeosxx64MicrosoftDotNetCdacTransportVersion)
+ <_cdacPackageVersion Condition="'$(TargetRid)' == 'osx-arm64'">$(runtimeosxarm64MicrosoftDotNetCdacTransportVersion)
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/eng/CodeAnalysis.src.globalconfig b/eng/CodeAnalysis.src.globalconfig
index c5f1ea2287..cbb4bbc276 100644
--- a/eng/CodeAnalysis.src.globalconfig
+++ b/eng/CodeAnalysis.src.globalconfig
@@ -524,6 +524,11 @@ dotnet_diagnostic.CA2020.severity = warning
# CA2021: Do not call Enumerable.Cast or Enumerable.OfType with incompatible types
dotnet_diagnostic.CA2021.severity = warning
+
+# CA2025: Ensure tasks using 'IDisposable' instances complete before the instances are disposed
+# This is having a fair bit of false positives - downgrade to suggestion.
+dotnet_diagnostic.CA2025.severity = suggestion
+
# CA2100: Review SQL queries for security vulnerabilities
dotnet_diagnostic.CA2100.severity = none
diff --git a/eng/Directory.Build.props b/eng/Directory.Build.props
index fc1ff6e1d1..0f8214b1b2 100644
--- a/eng/Directory.Build.props
+++ b/eng/Directory.Build.props
@@ -1,4 +1,3 @@
-
-
\ No newline at end of file
+
diff --git a/eng/DotNetBuild.props b/eng/DotNetBuild.props
deleted file mode 100644
index bf6e4287ba..0000000000
--- a/eng/DotNetBuild.props
+++ /dev/null
@@ -1,14 +0,0 @@
-
-
-
-
-
- diagnostics
- true
-
-
-
-
-
-
-
diff --git a/eng/InstallNativePackages.targets b/eng/InstallNativePackages.targets
new file mode 100644
index 0000000000..f8804fd7ab
--- /dev/null
+++ b/eng/InstallNativePackages.targets
@@ -0,0 +1,38 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/eng/InstallRuntimes.proj b/eng/InstallRuntimes.proj
index e7615549eb..d817ad01f9 100644
--- a/eng/InstallRuntimes.proj
+++ b/eng/InstallRuntimes.proj
@@ -1,7 +1,8 @@
- $(Platform)
- $([System.Runtime.InteropServices.RuntimeInformation]::ProcessArchitecture.ToString().ToLowerInvariant)
+ $([MSBuild]::NormalizePath('$(LiveRuntimeDir)'))
-
+ $(RepoRoot).dotnet-test\HKEY_LOCAL_MACHINE\SOFTWARE
-
+ $(RepoRoot).dotnet-test\x86\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node
- -NoPath -SkipNonVersionedFiles -Architecture $(BuildArch) -InstallDir $(DotNetInstallRoot)
- $([MSBuild]::NormalizeDirectory('$(DotNetInstallRoot)', 'shared', 'Microsoft.NETCore.App', '$(MicrosoftNETCoreAppRuntimewinx64Version)'))
+ -NoPath -SkipNonVersionedFiles -Architecture $(TargetArch) -InstallDir $(DotNetInstallRoot)
+ $([MSBuild]::NormalizeDirectory('$(DotNetInstallRoot)', 'shared', 'Microsoft.NETCore.App', '$(MicrosoftNETCoreAppRefVersion)'))$(DotNetInstallRoot)Debugger.Tests.Versions.txt
@@ -57,7 +57,7 @@
-->
+ DependsOnTargets="CleanupVersionManifest;InstallRuntimesWindows;InstallRuntimesUnix;OverrideLatestRuntime;WriteTestVersionManifest" />
+
+
+
+ <_LiveRuntimeFiles Include="$(LiveRuntimeDir)\**\*.*" />
+
+
+
-
- 3
- $(ArtifactsDir)bundledtools/
@@ -13,8 +10,13 @@
+
+
+
+
+
-
-
-
-
-
-
-
diff --git a/eng/Version.Details.xml b/eng/Version.Details.xml
index f1d10cae95..260c35a22a 100644
--- a/eng/Version.Details.xml
+++ b/eng/Version.Details.xml
@@ -1,74 +1,87 @@
+
-
+ https://github.com/microsoft/clrmd
- 3bbceb5b74c7a30a6a61581721e44b1380a96faa
+ d724947392626b66e39b525998a8817447d50380
-
+ https://github.com/microsoft/clrmd
- 3bbceb5b74c7a30a6a61581721e44b1380a96faa
+ d724947392626b66e39b525998a8817447d50380
-
- https://github.com/dotnet/arcade
- 1ec6078c26e4a60b77d8fe64881491cd28335a08
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
-
-
- https://github.com/dotnet/arcade
- 1ec6078c26e4a60b77d8fe64881491cd28335a08
-
-
-
- https://github.com/dotnet/arcade
- 1ec6078c26e4a60b77d8fe64881491cd28335a08
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4fhttps://github.com/dotnet/arcadeccfe6da198c5f05534863bbb1bff66e830e0c6ab
-
- https://github.com/dotnet/sdk
- bf19cbbae5fdf176cd4f13bca559133021779846
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
+
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
+
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
+
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
+
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
+
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
-
- https://github.com/dotnet/aspnetcore
- f3555640d3b0d049856947c4f2bd0b869adf5c5e
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
-
- https://github.com/dotnet/aspnetcore
- f3555640d3b0d049856947c4f2bd0b869adf5c5e
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
-
- https://github.com/dotnet/runtime
- 3cf22b78d833dbac3d7e32f32406419bcb7511c0
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
-
- https://github.com/dotnet/runtime
- 3cf22b78d833dbac3d7e32f32406419bcb7511c0
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
-
-
- https://github.com/dotnet/source-build-reference-packages
- cac4d021768f34079c30570518ece6af317cbab8
-
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
-
- https://github.com/dotnet/roslyn
- 40e6b96cad11400acb5b8999057ac8ba748df940
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
-
- https://github.com/dotnet/roslyn
- 40e6b96cad11400acb5b8999057ac8ba748df940
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
-
- https://github.com/dotnet/roslyn-analyzers
- 7f449a5d6f05c6aed77e4abf85aac2ce19f6a2d6
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
-
- https://github.com/dotnet/roslyn-analyzers
- 7f449a5d6f05c6aed77e4abf85aac2ce19f6a2d6
+
+ https://github.com/dotnet/dotnet
+ eaa19c281d34580a8168cff9ce1e7337da8bfe4f
diff --git a/eng/Versions.props b/eng/Versions.props
index 55e330f46f..8e96ea5ed8 100644
--- a/eng/Versions.props
+++ b/eng/Versions.props
@@ -16,13 +16,24 @@
- 10.0.0-preview.3.25153.8
- 10.0.0-preview.3.25153.8
+
+ 10.0.0-preview.7.25351.106
+ 10.0.0-preview.7.25351.106
- 10.0.0-preview.3.25163.10
- 10.0.0-preview.3.25163.10
+ 10.0.0-preview.7.25351.106
+ 10.0.0-preview.7.25351.106
- 10.0.100-preview.3.25159.6
+ 10.0.100-preview.7.25351.106
+
+
+ 10.0.0-preview.7.25351.106
+ 10.0.0-preview.7.25351.106
+ 10.0.0-preview.7.25351.106
+ 10.0.0-preview.7.25351.106
+ 10.0.0-preview.7.25351.106
+ 10.0.0-preview.7.25351.106
@@ -37,15 +48,15 @@
8.0.06.0.0
- 4.0.0-beta.25162.1
+ 4.0.0-beta.25359.117.10.0-beta1.24272.1
- 3.1.16
+ 3.1.236.0.06.0.46.0.05.0.1
- 2.0.0-beta4.25153.1
+ 2.0.0-beta5.25210.15.0.04.5.18.0.1
@@ -55,12 +66,12 @@
8.0.08.0.52.0.3
- 10.0.0-beta.25157.1
+ 10.0.0-beta.25351.1061.2.0-beta.5567.0.0-beta.22316.210.0.26100.113.0.1
- 10.0.616304
+ 10.0.622304
- 4.11.0-2.24271.11
- 4.11.0-2.24271.11
+ 5.0.0-1.25351.106
+ 5.0.0-1.25351.1064.11.0-2.24271.113.11.0
@@ -83,8 +94,8 @@
4.4.04.8.0$(MicrosoftCodeAnalysisVersion)
- 3.12.0-beta1.24605.1
- 10.0.0-preview.24605.1
+ 5.0.0-1.25351.106
+ 10.0.0-preview.25351.106
@@ -102,8 +113,8 @@
-->
- 9.0.0
- 8.0.8
+ 9.0.4
+ 8.0.15default
@@ -114,8 +125,8 @@
- $(VSRedistCommonNetCoreSharedFrameworkx64100Version)
- $(MicrosoftNETCoreAppRuntimewinx64Version)
+ $(MicrosoftNETCorePlatformsVersion)
+ $(MicrosoftNETCoreAppRefVersion)$(MicrosoftAspNetCoreAppRefInternalVersion)$(MicrosoftAspNetCoreAppRefVersion)net10.0
@@ -138,8 +149,8 @@
- $(VSRedistCommonNetCoreSharedFrameworkx64100Version)
- $(MicrosoftNETCoreAppRuntimewinx64Version)
+ $(MicrosoftNETCorePlatformsVersion)
+ $(MicrosoftNETCoreAppRefVersion)$(MicrosoftAspNetCoreAppRefInternalVersion)$(MicrosoftAspNetCoreAppRefVersion)net10.0
diff --git a/eng/build.ps1 b/eng/build.ps1
index 72886ed021..15f2d14f7e 100644
--- a/eng/build.ps1
+++ b/eng/build.ps1
@@ -7,13 +7,16 @@ Param(
[switch] $installruntimes,
[switch] $privatebuild,
[switch] $ci,
+ [switch][Alias('bl')]$binaryLog,
[switch] $skipmanaged,
[switch] $skipnative,
[switch] $bundletools,
+ [switch] $useCdac,
[ValidatePattern("(default|\d+\.\d+.\d+(-[a-z0-9\.]+)?)")][string] $dotnetruntimeversion = 'default',
[ValidatePattern("(default|\d+\.\d+.\d+(-[a-z0-9\.]+)?)")][string] $dotnetruntimedownloadversion= 'default',
[string] $runtimesourcefeed = '',
[string] $runtimesourcefeedkey = '',
+ [string] $liveRuntimeDir = '',
[Parameter(ValueFromRemainingArguments=$true)][String[]] $remainingargs
)
@@ -36,9 +39,12 @@ switch ($configuration.ToLower()) {
$reporoot = Join-Path $PSScriptRoot ".."
$engroot = Join-Path $reporoot "eng"
$artifactsdir = Join-Path $reporoot "artifacts"
+$os = "Windows_NT"
$logdir = Join-Path $artifactsdir "log"
$logdir = Join-Path $logdir Windows_NT.$architecture.$configuration
+$bl = if ($binaryLog) { '-binaryLog' } else { '' }
+
if ($ci) {
$remainingargs = "-ci " + $remainingargs
}
@@ -51,17 +57,18 @@ if ($bundletools) {
$test = $False
}
-# Install sdk for building, restore and build managed components.
-if (-not $skipmanaged) {
- Invoke-Expression "& `"$engroot\common\build.ps1`" -configuration $configuration -verbosity $verbosity /p:BuildArch=$architecture /p:TestArchitectures=$architecture $remainingargs"
+# Build native components
+if (-not $skipnative) {
+ Invoke-Expression "& `"$engroot\Build-Native.cmd`" -architecture $architecture -configuration $configuration -verbosity $verbosity $remainingargs"
if ($lastExitCode -ne 0) {
exit $lastExitCode
}
}
-# Build native components
-if (-not $skipnative) {
- Invoke-Expression "& `"$engroot\Build-Native.cmd`" -architecture $architecture -configuration $configuration -verbosity $verbosity $remainingargs"
+# Install sdk for building, restore and build managed components.
+if (-not $skipmanaged) {
+ Invoke-Expression "& `"$engroot\common\build.ps1`" -configuration $configuration -verbosity $verbosity $bl /p:TargetOS=$os /p:TargetArch=$architecture /p:TestArchitectures=$architecture $remainingargs"
+
if ($lastExitCode -ne 0) {
exit $lastExitCode
}
@@ -79,25 +86,32 @@ if ($installruntimes -or $privatebuild) {
/t:InstallTestRuntimes `
/bl:$logdir\InstallRuntimes.binlog `
/p:PrivateBuildTesting=$privatebuildtesting `
- /p:BuildArch=$architecture `
- /p:TestArchitectures=$architecture
+ /p:TargetOS=$os `
+ /p:TargetArch=$architecture `
+ /p:TestArchitectures=$architecture `
+ /p:LiveRuntimeDir="$liveRuntimeDir"
}
# Run the xunit tests
if ($test) {
if (-not $crossbuild) {
+ if ($useCdac) {
+ $env:SOS_TEST_CDAC="true"
+ }
& "$engroot\common\build.ps1" `
-test `
-configuration $configuration `
-verbosity $verbosity `
-ci:$ci `
/bl:$logdir\Test.binlog `
- /p:BuildArch=$architecture `
+ /p:TargetOS=$os `
+ /p:TargetArch=$architecture `
/p:TestArchitectures=$architecture `
/p:DotnetRuntimeVersion="$dotnetruntimeversion" `
/p:DotnetRuntimeDownloadVersion="$dotnetruntimedownloadversion" `
/p:RuntimeSourceFeed="$runtimesourcefeed" `
- /p:RuntimeSourceFeedKey="$runtimesourcefeedkey"
+ /p:RuntimeSourceFeedKey="$runtimesourcefeedkey" `
+ /p:LiveRuntimeDir="$liveRuntimeDir"
if ($lastExitCode -ne 0) {
exit $lastExitCode
diff --git a/eng/build.sh b/eng/build.sh
index 101f8c33cc..e84a3eb197 100755
--- a/eng/build.sh
+++ b/eng/build.sh
@@ -30,13 +30,14 @@ __InstallRuntimes=0
__PrivateBuild=0
__Test=0
__UnprocessedBuildArgs=
+__UseCdac=0
+__LiveRuntimeDir=
usage_list+=("-skipmanaged: do not build managed components.")
usage_list+=("-skipnative: do not build native components.")
usage_list+=("-test: run xunit tests")
handle_arguments() {
-
lowerI="$(echo "${1/--/-}" | tr "[:upper:]" "[:lower:]")"
case "$lowerI" in
architecture|-architecture|-a)
@@ -79,6 +80,11 @@ handle_arguments() {
__ShiftArgs=1
;;
+ liveruntimedir|-liveruntimedir)
+ __LiveRuntimeDir="$2"
+ __ShiftArgs=1
+ ;;
+
skipmanaged|-skipmanaged)
__ManagedBuild=0
;;
@@ -99,6 +105,10 @@ handle_arguments() {
__Test=1
;;
+ usecdac|-usecdac)
+ __UseCdac=1
+ ;;
+
-warnaserror|-nodereuse)
__ManagedBuildArgs="$__ManagedBuildArgs $1 $2"
__ShiftArgs=1
@@ -127,49 +137,17 @@ mkdir -p "$__IntermediatesDir"
mkdir -p "$__LogsDir"
mkdir -p "$__CMakeBinDir"
-__ExtraCmakeArgs="$__CMakeArgs $__ExtraCmakeArgs -DCLR_MANAGED_BINARY_DIR=$__RootBinDir/bin -DCLR_BUILD_TYPE=$__BuildType"
+__ExtraCmakeArgs="$__CMakeArgs $__ExtraCmakeArgs -DCLR_BUILD_TYPE=$__BuildType"
# Specify path to be set for CMAKE_INSTALL_PREFIX.
# This is where all built native libraries will copied to.
export __CMakeBinDir="$__BinDir"
-
if [[ "$__TargetArch" == "armel" ]]; then
# Armel cross build is Tizen specific and does not support Portable RID build
__PortableBuild=0
fi
-#
-# Managed build
-#
-
-if [[ "$__ManagedBuild" == 1 ]]; then
-
- echo "Commencing managed build for $__BuildType in $__RootBinDir/bin"
- "$__RepoRootDir/eng/common/build.sh" --configuration "$__BuildType" $__CommonMSBuildArgs $__ManagedBuildArgs $__UnprocessedBuildArgs
-
- if [ "$?" != 0 ]; then
- exit 1
- fi
-
- echo "Generating Version Source File"
- __GenerateVersionLog="$__LogsDir/GenerateVersion.binlog"
-
- "$__RepoRootDir/eng/common/msbuild.sh" \
- $__RepoRootDir/eng/native-prereqs.proj \
- /bl:$__GenerateVersionLog \
- /t:BuildPrereqs \
- /restore \
- /p:Configuration="$__BuildType" \
- /p:Platform="$__TargetArch" \
- $__UnprocessedBuildArgs
-
- if [ $? != 0 ]; then
- echo "Generating Version Source File FAILED"
- exit 1
- fi
-fi
-
#
# Setup LLDB paths for native build
#
@@ -205,31 +183,53 @@ fi
# Build native components
#
if [[ "$__NativeBuild" == 1 ]]; then
+ echo "Generating Version Source File"
+ __GenerateVersionLog="$__LogsDir/GenerateVersion.binlog"
+
+ "$__RepoRootDir/eng/common/msbuild.sh" \
+ $__RepoRootDir/eng/native-prereqs.proj \
+ /bl:$__GenerateVersionLog \
+ /t:Build \
+ /restore \
+ /p:Configuration="$__BuildType" \
+ /p:TargetOS="$__TargetOS" \
+ /p:TargetArch="$__TargetArch" \
+ /p:TargetRid="$__TargetRid" \
+ /p:Platform="$__TargetArch" \
+ $__UnprocessedBuildArgs
+
+ if [ $? != 0 ]; then
+ echo "Generating Version Source File FAILED"
+ exit 1
+ fi
+
build_native "$__TargetOS" "$__TargetArch" "$__RepoRootDir" "$__IntermediatesDir" "install" "$__ExtraCmakeArgs" "diagnostic component" | tee "$__LogsDir"/make.log
- if [ "$?" != 0 ]; then
+ if [ "${PIPESTATUS[0]}" != 0 ]; then
echo "Native build FAILED"
exit 1
fi
fi
#
-# Copy the native SOS binaries to where these tools expect for testing
+# Managed build
#
-if [[ "$__NativeBuild" == 1 || "$__Test" == 1 ]]; then
- __targetRid=net8.0
- __dotnet_sos=$__RootBinDir/bin/dotnet-sos/$__BuildType/$__targetRid/publish/$__OutputRid
- __dotnet_dump=$__RootBinDir/bin/dotnet-dump/$__BuildType/$__targetRid/publish/$__OutputRid
-
- mkdir -p "$__dotnet_sos"
- mkdir -p "$__dotnet_dump"
+if [[ "$__ManagedBuild" == 1 ]]; then
- cp "$__BinDir"/* "$__dotnet_sos"
- echo "Copied SOS to $__dotnet_sos"
+ # __CommonMSBuildArgs contains TargetOS property
+ echo "Commencing managed build for $__BuildType in $__RootBinDir/bin"
+ "$__RepoRootDir/eng/common/build.sh" \
+ --configuration "$__BuildType" \
+ /p:TargetArch="$__TargetArch" \
+ /p:TargetRid="$__TargetRid" \
+ $__CommonMSBuildArgs \
+ $__ManagedBuildArgs \
+ $__UnprocessedBuildArgs
- cp "$__BinDir"/* "$__dotnet_dump"
- echo "Copied SOS to $__dotnet_dump"
+ if [ "$?" != 0 ]; then
+ exit 1
+ fi
fi
#
@@ -247,8 +247,11 @@ if [[ "$__InstallRuntimes" == 1 || "$__PrivateBuild" == 1 ]]; then
/t:InstallTestRuntimes \
/bl:"$__LogsDir/InstallRuntimes.binlog" \
/p:PrivateBuildTesting="$__privateBuildTesting" \
- /p:BuildArch="$__TargetArch" \
- /p:TestArchitectures="$__TargetArch"
+ /p:TargetOS="$__TargetOS" \
+ /p:TargetArch="$__TargetArch" \
+ /p:TargetRid="$__TargetRid" \
+ /p:TestArchitectures="$__TargetArch" \
+ /p:LiveRuntimeDir="$__LiveRuntimeDir"
fi
#
@@ -295,15 +298,22 @@ if [[ "$__Test" == 1 ]]; then
echo "lldb: '$LLDB_PATH' gdb: '$GDB_PATH'"
+ if [[ "$__UseCdac" == 1 ]]; then
+ export SOS_TEST_CDAC="true"
+ fi
+
+ # __CommonMSBuildArgs contains TargetOS property
"$__RepoRootDir/eng/common/build.sh" \
--test \
--configuration "$__BuildType" \
/bl:"$__LogsDir"/Test.binlog \
- /p:BuildArch="$__TargetArch" \
+ /p:TargetArch="$__TargetArch" \
+ /p:TargetRid="$__TargetRid" \
/p:DotnetRuntimeVersion="$__DotnetRuntimeVersion" \
/p:DotnetRuntimeDownloadVersion="$__DotnetRuntimeDownloadVersion" \
/p:RuntimeSourceFeed="$__RuntimeSourceFeed" \
/p:RuntimeSourceFeedKey="$__RuntimeSourceFeedKey" \
+ /p:LiveRuntimeDir="$__LiveRuntimeDir" \
$__CommonMSBuildArgs
if [ $? != 0 ]; then
diff --git a/eng/common/CIBuild.cmd b/eng/common/CIBuild.cmd
index 56c2f25ac2..ac1f72bf94 100644
--- a/eng/common/CIBuild.cmd
+++ b/eng/common/CIBuild.cmd
@@ -1,2 +1,2 @@
@echo off
-powershell -ExecutionPolicy ByPass -NoProfile -command "& """%~dp0Build.ps1""" -restore -build -test -sign -pack -publish -ci %*"
\ No newline at end of file
+powershell -ExecutionPolicy ByPass -NoProfile -command "& """%~dp0Build.ps1""" -restore -build -test -sign -pack -publish -ci %*"
diff --git a/eng/common/SetupNugetSources.sh b/eng/common/SetupNugetSources.sh
old mode 100644
new mode 100755
diff --git a/eng/common/build.ps1 b/eng/common/build.ps1
index 438f9920c4..8cfee107e7 100644
--- a/eng/common/build.ps1
+++ b/eng/common/build.ps1
@@ -7,6 +7,7 @@ Param(
[string] $msbuildEngine = $null,
[bool] $warnAsError = $true,
[bool] $nodeReuse = $true,
+ [switch] $buildCheck = $false,
[switch][Alias('r')]$restore,
[switch] $deployDeps,
[switch][Alias('b')]$build,
@@ -20,6 +21,7 @@ Param(
[switch] $publish,
[switch] $clean,
[switch][Alias('pb')]$productBuild,
+ [switch]$fromVMR,
[switch][Alias('bl')]$binaryLog,
[switch][Alias('nobl')]$excludeCIBinarylog,
[switch] $ci,
@@ -71,6 +73,9 @@ function Print-Usage() {
Write-Host " -msbuildEngine Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
Write-Host " -excludePrereleaseVS Set to exclude build engines in prerelease versions of Visual Studio"
Write-Host " -nativeToolsOnMachine Sets the native tools on machine environment variable (indicating that the script should use native tools on machine)"
+ Write-Host " -nodeReuse Sets nodereuse msbuild parameter ('true' or 'false')"
+ Write-Host " -buildCheck Sets /check msbuild parameter"
+ Write-Host " -fromVMR Set when building from within the VMR"
Write-Host ""
Write-Host "Command line arguments not listed above are passed thru to msbuild."
@@ -97,6 +102,7 @@ function Build {
$bl = if ($binaryLog) { '/bl:' + (Join-Path $LogDir 'Build.binlog') } else { '' }
$platformArg = if ($platform) { "/p:Platform=$platform" } else { '' }
+ $check = if ($buildCheck) { '/check' } else { '' }
if ($projects) {
# Re-assign properties to a new variable because PowerShell doesn't let us append properties directly for unclear reasons.
@@ -113,6 +119,7 @@ function Build {
MSBuild $toolsetBuildProj `
$bl `
$platformArg `
+ $check `
/p:Configuration=$configuration `
/p:RepoRoot=$RepoRoot `
/p:Restore=$restore `
@@ -122,11 +129,13 @@ function Build {
/p:Deploy=$deploy `
/p:Test=$test `
/p:Pack=$pack `
- /p:DotNetBuildRepo=$productBuild `
+ /p:DotNetBuild=$productBuild `
+ /p:DotNetBuildFromVMR=$fromVMR `
/p:IntegrationTest=$integrationTest `
/p:PerformanceTest=$performanceTest `
/p:Sign=$sign `
/p:Publish=$publish `
+ /p:RestoreStaticGraphEnableBinaryLogger=$binaryLog `
@properties
}
diff --git a/eng/common/build.sh b/eng/common/build.sh
index 483647daf1..9767bb411a 100755
--- a/eng/common/build.sh
+++ b/eng/common/build.sh
@@ -42,6 +42,8 @@ usage()
echo " --prepareMachine Prepare machine for CI run, clean up processes after build"
echo " --nodeReuse Sets nodereuse msbuild parameter ('true' or 'false')"
echo " --warnAsError Sets warnaserror msbuild parameter ('true' or 'false')"
+ echo " --buildCheck Sets /check msbuild parameter"
+ echo " --fromVMR Set when building from within the VMR"
echo ""
echo "Command line arguments not listed above are passed thru to msbuild."
echo "Arguments can also be passed in with a single hyphen."
@@ -63,6 +65,7 @@ restore=false
build=false
source_build=false
product_build=false
+from_vmr=false
rebuild=false
test=false
integration_test=false
@@ -76,6 +79,7 @@ clean=false
warn_as_error=true
node_reuse=true
+build_check=false
binary_log=false
exclude_ci_binary_log=false
pipelines_log=false
@@ -87,7 +91,7 @@ verbosity='minimal'
runtime_source_feed=''
runtime_source_feed_key=''
-properties=''
+properties=()
while [[ $# > 0 ]]; do
opt="$(echo "${1/#--/-}" | tr "[:upper:]" "[:lower:]")"
case "$opt" in
@@ -127,19 +131,22 @@ while [[ $# > 0 ]]; do
-pack)
pack=true
;;
- -sourcebuild|-sb)
+ -sourcebuild|-source-build|-sb)
build=true
source_build=true
product_build=true
restore=true
pack=true
;;
- -productBuild|-pb)
+ -productbuild|-product-build|-pb)
build=true
product_build=true
restore=true
pack=true
;;
+ -fromvmr|-from-vmr)
+ from_vmr=true
+ ;;
-test|-t)
test=true
;;
@@ -173,6 +180,9 @@ while [[ $# > 0 ]]; do
node_reuse=$2
shift
;;
+ -buildcheck)
+ build_check=true
+ ;;
-runtimesourcefeed)
runtime_source_feed=$2
shift
@@ -182,7 +192,7 @@ while [[ $# > 0 ]]; do
shift
;;
*)
- properties="$properties $1"
+ properties+=("$1")
;;
esac
@@ -216,7 +226,7 @@ function Build {
InitializeCustomToolset
if [[ ! -z "$projects" ]]; then
- properties="$properties /p:Projects=$projects"
+ properties+=("/p:Projects=$projects")
fi
local bl=""
@@ -224,14 +234,21 @@ function Build {
bl="/bl:\"$log_dir/Build.binlog\""
fi
+ local check=""
+ if [[ "$build_check" == true ]]; then
+ check="/check"
+ fi
+
MSBuild $_InitializeToolset \
$bl \
+ $check \
/p:Configuration=$configuration \
/p:RepoRoot="$repo_root" \
/p:Restore=$restore \
/p:Build=$build \
- /p:DotNetBuildRepo=$product_build \
+ /p:DotNetBuild=$product_build \
/p:DotNetBuildSourceOnly=$source_build \
+ /p:DotNetBuildFromVMR=$from_vmr \
/p:Rebuild=$rebuild \
/p:Test=$test \
/p:Pack=$pack \
@@ -239,7 +256,8 @@ function Build {
/p:PerformanceTest=$performance_test \
/p:Sign=$sign \
/p:Publish=$publish \
- $properties
+ /p:RestoreStaticGraphEnableBinaryLogger=$binary_log \
+ ${properties[@]+"${properties[@]}"}
ExitWithExitCode 0
}
diff --git a/eng/common/cibuild.sh b/eng/common/cibuild.sh
index 1a02c0dec8..66e3b0ac61 100755
--- a/eng/common/cibuild.sh
+++ b/eng/common/cibuild.sh
@@ -13,4 +13,4 @@ while [[ -h $source ]]; do
done
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
-. "$scriptroot/build.sh" --restore --build --test --pack --publish --ci $@
\ No newline at end of file
+. "$scriptroot/build.sh" --restore --build --test --pack --publish --ci $@
diff --git a/eng/common/core-templates/job/job.yml b/eng/common/core-templates/job/job.yml
index 4dfb3d8683..6badecba7b 100644
--- a/eng/common/core-templates/job/job.yml
+++ b/eng/common/core-templates/job/job.yml
@@ -73,9 +73,6 @@ jobs:
- ${{ if ne(parameters.enableTelemetry, 'false') }}:
- name: DOTNET_CLI_TELEMETRY_PROFILE
value: '$(Build.Repository.Uri)'
- - ${{ if eq(parameters.enableRichCodeNavigation, 'true') }}:
- - name: EnableRichCodeNavigation
- value: 'true'
# Retry signature validation up to three times, waiting 2 seconds between attempts.
# See https://learn.microsoft.com/en-us/nuget/reference/errors-and-warnings/nu3028#retry-untrusted-root-failures
- name: NUGET_EXPERIMENTAL_CHAIN_BUILD_RETRY_POLICY
@@ -147,16 +144,6 @@ jobs:
- ${{ each step in parameters.steps }}:
- ${{ step }}
- - ${{ if eq(parameters.enableRichCodeNavigation, true) }}:
- - task: RichCodeNavIndexer@0
- displayName: RichCodeNav Upload
- inputs:
- languages: ${{ coalesce(parameters.richCodeNavigationLanguage, 'csharp') }}
- environment: ${{ coalesce(parameters.richCodeNavigationEnvironment, 'internal') }}
- richNavLogOutputDirectory: $(Build.SourcesDirectory)/artifacts/bin
- uploadRichNavArtifacts: ${{ coalesce(parameters.richCodeNavigationUploadArtifacts, false) }}
- continueOnError: true
-
- ${{ each step in parameters.componentGovernanceSteps }}:
- ${{ step }}
diff --git a/eng/common/core-templates/job/onelocbuild.yml b/eng/common/core-templates/job/onelocbuild.yml
index 00feec8ebb..8034815f42 100644
--- a/eng/common/core-templates/job/onelocbuild.yml
+++ b/eng/common/core-templates/job/onelocbuild.yml
@@ -86,8 +86,7 @@ jobs:
isAutoCompletePrSelected: ${{ parameters.AutoCompletePr }}
${{ if eq(parameters.CreatePr, true) }}:
isUseLfLineEndingsSelected: ${{ parameters.UseLfLineEndings }}
- ${{ if eq(parameters.RepoType, 'gitHub') }}:
- isShouldReusePrSelected: ${{ parameters.ReusePr }}
+ isShouldReusePrSelected: ${{ parameters.ReusePr }}
packageSourceAuth: patAuth
patVariable: ${{ parameters.CeapexPat }}
${{ if eq(parameters.RepoType, 'gitHub') }}:
@@ -118,4 +117,4 @@ jobs:
pathToPublish: '$(Build.SourcesDirectory)/eng/Localize/'
publishLocation: Container
artifactName: Loc
- condition: ${{ parameters.condition }}
\ No newline at end of file
+ condition: ${{ parameters.condition }}
diff --git a/eng/common/core-templates/job/publish-build-assets.yml b/eng/common/core-templates/job/publish-build-assets.yml
index 5da3c2a2a2..d5303229c9 100644
--- a/eng/common/core-templates/job/publish-build-assets.yml
+++ b/eng/common/core-templates/job/publish-build-assets.yml
@@ -29,6 +29,15 @@ parameters:
is1ESPipeline: ''
+ # Optional: 🌤️ or not the build has assets it wants to publish to BAR
+ isAssetlessBuild: false
+
+ # Optional, publishing version
+ publishingVersion: 3
+
+ # Optional: A minimatch pattern for the asset manifests to publish to BAR
+ assetManifestsPattern: '*/manifests/**/*.xml'
+
jobs:
- job: Asset_Registry_Publish
@@ -72,14 +81,33 @@ jobs:
- checkout: self
fetchDepth: 3
clean: true
-
- - task: DownloadPipelineArtifact@2
- displayName: Download Asset Manifests
- inputs:
- artifactName: AssetManifests
- targetPath: '$(Build.StagingDirectory)/AssetManifests'
- condition: ${{ parameters.condition }}
- continueOnError: ${{ parameters.continueOnError }}
+
+ - ${{ if eq(parameters.isAssetlessBuild, 'false') }}:
+ - ${{ if eq(parameters.publishingVersion, 3) }}:
+ - task: DownloadPipelineArtifact@2
+ displayName: Download Asset Manifests
+ inputs:
+ artifactName: AssetManifests
+ targetPath: '$(Build.StagingDirectory)/AssetManifests'
+ condition: ${{ parameters.condition }}
+ continueOnError: ${{ parameters.continueOnError }}
+ - ${{ if eq(parameters.publishingVersion, 4) }}:
+ - task: DownloadPipelineArtifact@2
+ displayName: Download V4 asset manifests
+ inputs:
+ itemPattern: '*/manifests/**/*.xml'
+ targetPath: '$(Build.StagingDirectory)/AllAssetManifests'
+ condition: ${{ parameters.condition }}
+ continueOnError: ${{ parameters.continueOnError }}
+ - task: CopyFiles@2
+ displayName: Copy V4 asset manifests to AssetManifests
+ inputs:
+ SourceFolder: '$(Build.StagingDirectory)/AllAssetManifests'
+ Contents: ${{ parameters.assetManifestsPattern }}
+ TargetFolder: '$(Build.StagingDirectory)/AssetManifests'
+ flattenFolders: true
+ condition: ${{ parameters.condition }}
+ continueOnError: ${{ parameters.continueOnError }}
- task: NuGetAuthenticate@1
@@ -92,6 +120,7 @@ jobs:
scriptPath: $(Build.SourcesDirectory)/eng/common/sdk-task.ps1
arguments: -task PublishBuildAssets -restore -msbuildEngine dotnet
/p:ManifestsPath='$(Build.StagingDirectory)/AssetManifests'
+ /p:IsAssetlessBuild=${{ parameters.isAssetlessBuild }}
/p:MaestroApiEndpoint=https://maestro.dot.net
/p:OfficialBuildId=$(Build.BuildNumber)
condition: ${{ parameters.condition }}
@@ -115,6 +144,17 @@ jobs:
Copy-Item -Path $symbolExclusionfile -Destination "$(Build.StagingDirectory)/ReleaseConfigs"
}
+ - ${{ if eq(parameters.publishingVersion, 4) }}:
+ - template: /eng/common/core-templates/steps/publish-pipeline-artifacts.yml
+ parameters:
+ is1ESPipeline: ${{ parameters.is1ESPipeline }}
+ args:
+ targetPath: '$(Build.ArtifactStagingDirectory)/MergedManifest.xml'
+ artifactName: AssetManifests
+ displayName: 'Publish Merged Manifest'
+ retryCountOnTaskFailure: 10 # for any logs being locked
+ sbomEnabled: false # we don't need SBOM for logs
+
- template: /eng/common/core-templates/steps/publish-build-artifacts.yml
parameters:
is1ESPipeline: ${{ parameters.is1ESPipeline }}
@@ -124,7 +164,7 @@ jobs:
publishLocation: Container
artifactName: ReleaseConfigs
- - ${{ if eq(parameters.publishAssetsImmediately, 'true') }}:
+ - ${{ if or(eq(parameters.publishAssetsImmediately, 'true'), eq(parameters.isAssetlessBuild, 'true')) }}:
- template: /eng/common/core-templates/post-build/setup-maestro-vars.yml
parameters:
BARBuildId: ${{ parameters.BARBuildId }}
@@ -145,6 +185,7 @@ jobs:
-WaitPublishingFinish true
-ArtifactsPublishingAdditionalParameters '${{ parameters.artifactsPublishingAdditionalParameters }}'
-SymbolPublishingAdditionalParameters '${{ parameters.symbolPublishingAdditionalParameters }}'
+ -SkipAssetsPublishing '${{ parameters.isAssetlessBuild }}'
- ${{ if eq(parameters.enablePublishBuildArtifacts, 'true') }}:
- template: /eng/common/core-templates/steps/publish-logs.yml
diff --git a/eng/common/core-templates/job/source-build.yml b/eng/common/core-templates/job/source-build.yml
index 05f7ad6ef0..d805d5faeb 100644
--- a/eng/common/core-templates/job/source-build.yml
+++ b/eng/common/core-templates/job/source-build.yml
@@ -27,6 +27,8 @@ parameters:
# Specifies the build script to invoke to perform the build in the repo. The default
# './build.sh' should work for typical Arcade repositories, but this is customizable for
# difficult situations.
+ # buildArguments: ''
+ # Specifies additional build arguments to pass to the build script.
# jobProperties: {}
# A list of job properties to inject at the top level, for potential extensibility beyond
# container and pool.
diff --git a/eng/common/core-templates/jobs/jobs.yml b/eng/common/core-templates/jobs/jobs.yml
index 3d2deaa4a0..bf35b78faa 100644
--- a/eng/common/core-templates/jobs/jobs.yml
+++ b/eng/common/core-templates/jobs/jobs.yml
@@ -27,6 +27,9 @@ parameters:
# Optional: Publish the assets as soon as the publish to BAR stage is complete, rather doing so in a separate stage.
publishAssetsImmediately: false
+ # Optional: 🌤️ or not the build has assets it wants to publish to BAR
+ isAssetlessBuild: false
+
# Optional: If using publishAssetsImmediately and additional parameters are needed, can be used to send along additional parameters (normally sent to post-build.yml)
artifactsPublishingAdditionalParameters: ''
signingValidationAdditionalParameters: ''
@@ -93,7 +96,7 @@ jobs:
${{ parameter.key }}: ${{ parameter.value }}
- ${{ if and(eq(parameters.runAsPublic, 'false'), ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
- - ${{ if or(eq(parameters.enablePublishBuildAssets, true), eq(parameters.artifacts.publish.manifests, 'true'), ne(parameters.artifacts.publish.manifests, '')) }}:
+ - ${{ if or(eq(parameters.enablePublishBuildAssets, true), eq(parameters.artifacts.publish.manifests, 'true'), ne(parameters.artifacts.publish.manifests, ''), eq(parameters.isAssetlessBuild, true)) }}:
- template: ../job/publish-build-assets.yml
parameters:
is1ESPipeline: ${{ parameters.is1ESPipeline }}
@@ -109,7 +112,8 @@ jobs:
- Source_Build_Complete
runAsPublic: ${{ parameters.runAsPublic }}
- publishAssetsImmediately: ${{ parameters.publishAssetsImmediately }}
+ publishAssetsImmediately: ${{ or(parameters.publishAssetsImmediately, parameters.isAssetlessBuild) }}
+ isAssetlessBuild: ${{ parameters.isAssetlessBuild }}
enablePublishBuildArtifacts: ${{ parameters.enablePublishBuildArtifacts }}
artifactsPublishingAdditionalParameters: ${{ parameters.artifactsPublishingAdditionalParameters }}
signingValidationAdditionalParameters: ${{ parameters.signingValidationAdditionalParameters }}
diff --git a/eng/common/core-templates/jobs/source-build.yml b/eng/common/core-templates/jobs/source-build.yml
index a10ccfbee6..df24c948ba 100644
--- a/eng/common/core-templates/jobs/source-build.yml
+++ b/eng/common/core-templates/jobs/source-build.yml
@@ -14,7 +14,7 @@ parameters:
# This is the default platform provided by Arcade, intended for use by a managed-only repo.
defaultManagedPlatform:
name: 'Managed'
- container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:centos-stream9'
+ container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:centos-stream-10-amd64'
# Defines the platforms on which to run build jobs. One job is created for each platform, and the
# object in this array is sent to the job template as 'platform'. If no platforms are specified,
diff --git a/eng/common/core-templates/post-build/post-build.yml b/eng/common/core-templates/post-build/post-build.yml
index a8c0bd3b92..a151fd811e 100644
--- a/eng/common/core-templates/post-build/post-build.yml
+++ b/eng/common/core-templates/post-build/post-build.yml
@@ -60,6 +60,11 @@ parameters:
artifactNames: ''
downloadArtifacts: true
+ - name: isAssetlessBuild
+ type: boolean
+ displayName: Is Assetless Build
+ default: false
+
# These parameters let the user customize the call to sdk-task.ps1 for publishing
# symbols & general artifacts as well as for signing validation
- name: symbolPublishingAdditionalParameters
@@ -188,9 +193,6 @@ stages:
buildId: $(AzDOBuildId)
artifactName: PackageArtifacts
checkDownloadedFiles: true
- itemPattern: |
- **
- !**/Microsoft.SourceBuild.Intermediate.*.nupkg
# This is necessary whenever we want to publish/restore to an AzDO private feed
# Since sdk-task.ps1 tries to restore packages we need to do this authentication here
@@ -320,3 +322,4 @@ stages:
-RequireDefaultChannels ${{ parameters.requireDefaultChannels }}
-ArtifactsPublishingAdditionalParameters '${{ parameters.artifactsPublishingAdditionalParameters }}'
-SymbolPublishingAdditionalParameters '${{ parameters.symbolPublishingAdditionalParameters }}'
+ -SkipAssetsPublishing '${{ parameters.isAssetlessBuild }}'
diff --git a/eng/common/core-templates/steps/install-microbuild.yml b/eng/common/core-templates/steps/install-microbuild.yml
index 2a6a529482..f3064a7834 100644
--- a/eng/common/core-templates/steps/install-microbuild.yml
+++ b/eng/common/core-templates/steps/install-microbuild.yml
@@ -5,43 +5,19 @@ parameters:
# Will be ignored if 'enableMicrobuild' is false or 'Agent.Os' is 'Windows_NT'
enableMicrobuildForMacAndLinux: false
# Location of the MicroBuild output folder
- microBuildOutputFolder: '$(Agent.TempDirectory)'
+ microBuildOutputFolder: '$(Build.SourcesDirectory)'
continueOnError: false
steps:
- ${{ if eq(parameters.enableMicrobuild, 'true') }}:
- ${{ if eq(parameters.enableMicrobuildForMacAndLinux, 'true') }}:
- # Install Python 3.12.x on when Python > 3.12.x is installed - https://github.com/dotnet/source-build/issues/4802
- - script: |
- version=$(python3 --version | awk '{print $2}')
- major=$(echo $version | cut -d. -f1)
- minor=$(echo $version | cut -d. -f2)
-
- installPython=false
- if [ "$major" -gt 3 ] || { [ "$major" -eq 3 ] && [ "$minor" -gt 12 ]; }; then
- installPython=true
- fi
-
- echo "Python version: $version."
- echo "Install Python 3.12.x: $installPython."
- echo "##vso[task.setvariable variable=installPython;isOutput=true]$installPython"
- name: InstallPython
- displayName: 'Determine Python installation'
- condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))
-
- - task: UsePythonVersion@0
- inputs:
- versionSpec: '3.12.x'
- displayName: 'Use Python 3.12.x'
- condition: and(succeeded(), eq(variables['InstallPython.installPython'], 'true'), ne(variables['Agent.Os'], 'Windows_NT'))
-
# Needed to download the MicroBuild plugin nupkgs on Mac and Linux when nuget.exe is unavailable
- task: UseDotNet@2
displayName: Install .NET 8.0 SDK for MicroBuild Plugin
inputs:
packageType: sdk
version: 8.0.x
- installationPath: ${{ parameters.microBuildOutputFolder }}/dotnet
+ installationPath: ${{ parameters.microBuildOutputFolder }}/.dotnet
workingDirectory: ${{ parameters.microBuildOutputFolder }}
condition: and(succeeded(), ne(variables['Agent.Os'], 'Windows_NT'))
@@ -53,6 +29,11 @@ steps:
feedSource: https://dnceng.pkgs.visualstudio.com/_packaging/MicroBuildToolset/nuget/v3/index.json
${{ if and(eq(parameters.enableMicrobuildForMacAndLinux, 'true'), ne(variables['Agent.Os'], 'Windows_NT')) }}:
azureSubscription: 'MicroBuild Signing Task (DevDiv)'
+ useEsrpCli: true
+ ${{ elseif eq(variables['System.TeamProject'], 'DevDiv') }}:
+ ConnectedPMEServiceName: 6cc74545-d7b9-4050-9dfa-ebefcc8961ea
+ ${{ else }}:
+ ConnectedPMEServiceName: 248d384a-b39b-46e3-8ad5-c2c210d5e7ca
env:
TeamName: $(_TeamName)
MicroBuildOutputFolderOverride: ${{ parameters.microBuildOutputFolder }}
diff --git a/eng/common/core-templates/steps/source-build.yml b/eng/common/core-templates/steps/source-build.yml
index 2341706b06..acf16ed349 100644
--- a/eng/common/core-templates/steps/source-build.yml
+++ b/eng/common/core-templates/steps/source-build.yml
@@ -19,19 +19,6 @@ steps:
set -x
df -h
- # If file changes are detected, set CopyWipIntoInnerSourceBuildRepo to copy the WIP changes into the inner source build repo.
- internalRestoreArgs=
- if ! git diff --quiet; then
- internalRestoreArgs='/p:CopyWipIntoInnerSourceBuildRepo=true'
- # The 'Copy WIP' feature of source build uses git stash to apply changes from the original repo.
- # This only works if there is a username/email configured, which won't be the case in most CI runs.
- git config --get user.email
- if [ $? -ne 0 ]; then
- git config user.email dn-bot@microsoft.com
- git config user.name dn-bot
- fi
- fi
-
# If building on the internal project, the internal storage variable may be available (usually only if needed)
# In that case, add variables to allow the download of internal runtimes if the specified versions are not found
# in the default public locations.
@@ -46,36 +33,11 @@ steps:
buildConfig='$(_BuildConfig)'
fi
- officialBuildArgs=
- if [ '${{ and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}' = 'True' ]; then
- officialBuildArgs='/p:DotNetPublishUsingPipelines=true /p:OfficialBuildId=$(BUILD.BUILDNUMBER)'
- fi
-
targetRidArgs=
if [ '${{ parameters.platform.targetRID }}' != '' ]; then
targetRidArgs='/p:TargetRid=${{ parameters.platform.targetRID }}'
fi
- runtimeOsArgs=
- if [ '${{ parameters.platform.runtimeOS }}' != '' ]; then
- runtimeOsArgs='/p:RuntimeOS=${{ parameters.platform.runtimeOS }}'
- fi
-
- baseOsArgs=
- if [ '${{ parameters.platform.baseOS }}' != '' ]; then
- baseOsArgs='/p:BaseOS=${{ parameters.platform.baseOS }}'
- fi
-
- publishArgs=
- if [ '${{ parameters.platform.skipPublishValidation }}' != 'true' ]; then
- publishArgs='--publish'
- fi
-
- assetManifestFileName=SourceBuild_RidSpecific.xml
- if [ '${{ parameters.platform.name }}' != '' ]; then
- assetManifestFileName=SourceBuild_${{ parameters.platform.name }}.xml
- fi
-
portableBuildArgs=
if [ '${{ parameters.platform.portableBuild }}' != '' ]; then
portableBuildArgs='/p:PortableBuild=${{ parameters.platform.portableBuild }}'
@@ -83,51 +45,21 @@ steps:
${{ coalesce(parameters.platform.buildScript, './build.sh') }} --ci \
--configuration $buildConfig \
- --restore --build --pack $publishArgs -bl \
- $officialBuildArgs \
+ --restore --build --pack -bl \
+ --source-build \
+ ${{ parameters.platform.buildArguments }} \
$internalRuntimeDownloadArgs \
- $internalRestoreArgs \
$targetRidArgs \
- $runtimeOsArgs \
- $baseOsArgs \
$portableBuildArgs \
- /p:DotNetBuildSourceOnly=true \
- /p:DotNetBuildRepo=true \
- /p:AssetManifestFileName=$assetManifestFileName
displayName: Build
-# Upload build logs for diagnosis.
-- task: CopyFiles@2
- displayName: Prepare BuildLogs staging directory
- inputs:
- SourceFolder: '$(Build.SourcesDirectory)'
- Contents: |
- **/*.log
- **/*.binlog
- artifacts/sb/prebuilt-report/**
- TargetFolder: '$(Build.StagingDirectory)/BuildLogs'
- CleanTargetFolder: true
- continueOnError: true
- condition: succeededOrFailed()
-
- template: /eng/common/core-templates/steps/publish-pipeline-artifacts.yml
parameters:
is1ESPipeline: ${{ parameters.is1ESPipeline }}
args:
displayName: Publish BuildLogs
- targetPath: '$(Build.StagingDirectory)/BuildLogs'
+ targetPath: artifacts/log/${{ coalesce(variables._BuildConfig, 'Release') }}
artifactName: BuildLogs_SourceBuild_${{ parameters.platform.name }}_Attempt$(System.JobAttempt)
continueOnError: true
condition: succeededOrFailed()
sbomEnabled: false # we don't need SBOM for logs
-
-# Manually inject component detection so that we can ignore the source build upstream cache, which contains
-# a nupkg cache of input packages (a local feed).
-# This path must match the upstream cache path in property 'CurrentRepoSourceBuiltNupkgCacheDir'
-# in src\Microsoft.DotNet.Arcade.Sdk\tools\SourceBuild\SourceBuildArcade.targets
-- template: /eng/common/core-templates/steps/component-governance.yml
- parameters:
- displayName: Component Detection (Exclude upstream cache)
- is1ESPipeline: ${{ parameters.is1ESPipeline }}
- componentGovernanceIgnoreDirectories: '$(Build.SourcesDirectory)/artifacts/sb/src/artifacts/obj/source-built-upstream-cache'
- disableComponentGovernance: ${{ eq(variables['System.TeamProject'], 'public') }}
diff --git a/eng/common/core-templates/steps/source-index-stage1-publish.yml b/eng/common/core-templates/steps/source-index-stage1-publish.yml
index 473a22c471..c2917c1efc 100644
--- a/eng/common/core-templates/steps/source-index-stage1-publish.yml
+++ b/eng/common/core-templates/steps/source-index-stage1-publish.yml
@@ -1,15 +1,15 @@
parameters:
- sourceIndexUploadPackageVersion: 2.0.0-20240522.1
- sourceIndexProcessBinlogPackageVersion: 1.0.1-20240522.1
+ sourceIndexUploadPackageVersion: 2.0.0-20250425.2
+ sourceIndexProcessBinlogPackageVersion: 1.0.1-20250515.1
sourceIndexPackageSource: https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json
binlogPath: artifacts/log/Debug/Build.binlog
steps:
- task: UseDotNet@2
- displayName: "Source Index: Use .NET 8 SDK"
+ displayName: "Source Index: Use .NET 9 SDK"
inputs:
packageType: sdk
- version: 8.0.x
+ version: 9.0.x
installationPath: $(Agent.TempDirectory)/dotnet
workingDirectory: $(Agent.TempDirectory)
diff --git a/eng/common/cross/arm64/tizen/tizen.patch b/eng/common/cross/arm64/tizen/tizen.patch
index af7c8be059..2cebc54738 100644
--- a/eng/common/cross/arm64/tizen/tizen.patch
+++ b/eng/common/cross/arm64/tizen/tizen.patch
@@ -5,5 +5,5 @@ diff -u -r a/usr/lib/libc.so b/usr/lib/libc.so
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf64-littleaarch64)
--GROUP ( /lib64/libc.so.6 /usr/lib64/libc_nonshared.a AS_NEEDED ( /lib/ld-linux-aarch64.so.1 ) )
+-GROUP ( /lib64/libc.so.6 /usr/lib64/libc_nonshared.a AS_NEEDED ( /lib64/ld-linux-aarch64.so.1 ) )
+GROUP ( libc.so.6 libc_nonshared.a AS_NEEDED ( ld-linux-aarch64.so.1 ) )
diff --git a/eng/common/cross/armel/armel.jessie.patch b/eng/common/cross/armel/armel.jessie.patch
deleted file mode 100644
index 2d26156193..0000000000
--- a/eng/common/cross/armel/armel.jessie.patch
+++ /dev/null
@@ -1,43 +0,0 @@
-diff -u -r a/usr/include/urcu/uatomic/generic.h b/usr/include/urcu/uatomic/generic.h
---- a/usr/include/urcu/uatomic/generic.h 2014-10-22 15:00:58.000000000 -0700
-+++ b/usr/include/urcu/uatomic/generic.h 2020-10-30 21:38:28.550000000 -0700
-@@ -69,10 +69,10 @@
- #endif
- #ifdef UATOMIC_HAS_ATOMIC_SHORT
- case 2:
-- return __sync_val_compare_and_swap_2(addr, old, _new);
-+ return __sync_val_compare_and_swap_2((uint16_t*) addr, old, _new);
- #endif
- case 4:
-- return __sync_val_compare_and_swap_4(addr, old, _new);
-+ return __sync_val_compare_and_swap_4((uint32_t*) addr, old, _new);
- #if (CAA_BITS_PER_LONG == 64)
- case 8:
- return __sync_val_compare_and_swap_8(addr, old, _new);
-@@ -109,7 +109,7 @@
- return;
- #endif
- case 4:
-- __sync_and_and_fetch_4(addr, val);
-+ __sync_and_and_fetch_4((uint32_t*) addr, val);
- return;
- #if (CAA_BITS_PER_LONG == 64)
- case 8:
-@@ -148,7 +148,7 @@
- return;
- #endif
- case 4:
-- __sync_or_and_fetch_4(addr, val);
-+ __sync_or_and_fetch_4((uint32_t*) addr, val);
- return;
- #if (CAA_BITS_PER_LONG == 64)
- case 8:
-@@ -187,7 +187,7 @@
- return __sync_add_and_fetch_2(addr, val);
- #endif
- case 4:
-- return __sync_add_and_fetch_4(addr, val);
-+ return __sync_add_and_fetch_4((uint32_t*) addr, val);
- #if (CAA_BITS_PER_LONG == 64)
- case 8:
- return __sync_add_and_fetch_8(addr, val);
diff --git a/eng/common/cross/build-android-rootfs.sh b/eng/common/cross/build-android-rootfs.sh
old mode 100644
new mode 100755
diff --git a/eng/common/cross/build-rootfs.sh b/eng/common/cross/build-rootfs.sh
old mode 100644
new mode 100755
index 74f399716b..8abfb71f72
--- a/eng/common/cross/build-rootfs.sh
+++ b/eng/common/cross/build-rootfs.sh
@@ -164,9 +164,13 @@ while :; do
armel)
__BuildArch=armel
__UbuntuArch=armel
- __UbuntuRepo="http://ftp.debian.org/debian/"
- __CodeName=jessie
+ __UbuntuRepo="http://archive.debian.org/debian/"
+ __CodeName=buster
__KeyringFile="/usr/share/keyrings/debian-archive-keyring.gpg"
+ __LLDB_Package="liblldb-6.0-dev"
+ __UbuntuPackages="${__UbuntuPackages// libomp-dev/}"
+ __UbuntuPackages="${__UbuntuPackages// libomp5/}"
+ __UbuntuSuites=
;;
armv6)
__BuildArch=armv6
@@ -278,44 +282,21 @@ while :; do
;;
xenial) # Ubuntu 16.04
- if [[ "$__CodeName" != "jessie" ]]; then
- __CodeName=xenial
- fi
- ;;
- zesty) # Ubuntu 17.04
- if [[ "$__CodeName" != "jessie" ]]; then
- __CodeName=zesty
- fi
+ __CodeName=xenial
;;
bionic) # Ubuntu 18.04
- if [[ "$__CodeName" != "jessie" ]]; then
- __CodeName=bionic
- fi
+ __CodeName=bionic
;;
focal) # Ubuntu 20.04
- if [[ "$__CodeName" != "jessie" ]]; then
- __CodeName=focal
- fi
+ __CodeName=focal
;;
jammy) # Ubuntu 22.04
- if [[ "$__CodeName" != "jessie" ]]; then
- __CodeName=jammy
- fi
+ __CodeName=jammy
;;
noble) # Ubuntu 24.04
- if [[ "$__CodeName" != "jessie" ]]; then
- __CodeName=noble
- fi
- if [[ -n "$__LLDB_Package" ]]; then
- __LLDB_Package="liblldb-18-dev"
- fi
- ;;
- jessie) # Debian 8
- __CodeName=jessie
- __KeyringFile="/usr/share/keyrings/debian-archive-keyring.gpg"
-
- if [[ -z "$__UbuntuRepo" ]]; then
- __UbuntuRepo="http://ftp.debian.org/debian/"
+ __CodeName=noble
+ if [[ -z "$__LLDB_Package" ]]; then
+ __LLDB_Package="liblldb-19-dev"
fi
;;
stretch) # Debian 9
@@ -333,7 +314,7 @@ while :; do
__KeyringFile="/usr/share/keyrings/debian-archive-keyring.gpg"
if [[ -z "$__UbuntuRepo" ]]; then
- __UbuntuRepo="http://ftp.debian.org/debian/"
+ __UbuntuRepo="http://archive.debian.org/debian/"
fi
;;
bullseye) # Debian 11
@@ -473,10 +454,6 @@ if [[ "$__AlpineVersion" =~ 3\.1[345] ]]; then
__AlpinePackages="${__AlpinePackages/compiler-rt/compiler-rt-static}"
fi
-if [[ "$__BuildArch" == "armel" ]]; then
- __LLDB_Package="lldb-3.5-dev"
-fi
-
__UbuntuPackages+=" ${__LLDB_Package:-}"
if [[ -z "$__UbuntuRepo" ]]; then
@@ -850,12 +827,6 @@ EOF
if [[ "$__SkipUnmount" == "0" ]]; then
umount "$__RootfsDir"/* || true
fi
-
- if [[ "$__BuildArch" == "armel" && "$__CodeName" == "jessie" ]]; then
- pushd "$__RootfsDir"
- patch -p1 < "$__CrossDir/$__BuildArch/armel.jessie.patch"
- popd
- fi
elif [[ "$__Tizen" == "tizen" ]]; then
ROOTFS_DIR="$__RootfsDir" "$__CrossDir/tizen-build-rootfs.sh" "$__BuildArch"
else
diff --git a/eng/common/cross/install-debs.py b/eng/common/cross/install-debs.py
old mode 100644
new mode 100755
diff --git a/eng/common/cross/tizen-build-rootfs.sh b/eng/common/cross/tizen-build-rootfs.sh
old mode 100644
new mode 100755
diff --git a/eng/common/cross/tizen-fetch.sh b/eng/common/cross/tizen-fetch.sh
old mode 100644
new mode 100755
index 28936ceef3..37c3a61f1d
--- a/eng/common/cross/tizen-fetch.sh
+++ b/eng/common/cross/tizen-fetch.sh
@@ -156,13 +156,8 @@ fetch_tizen_pkgs()
done
}
-if [ "$TIZEN_ARCH" == "riscv64" ]; then
- BASE="Tizen-Base-RISCV"
- UNIFIED="Tizen-Unified-RISCV"
-else
- BASE="Tizen-Base"
- UNIFIED="Tizen-Unified"
-fi
+BASE="Tizen-Base"
+UNIFIED="Tizen-Unified"
Inform "Initialize ${TIZEN_ARCH} base"
fetch_tizen_pkgs_init standard $BASE
diff --git a/eng/common/darc-init.sh b/eng/common/darc-init.sh
old mode 100644
new mode 100755
index 36dbd45e1c..e889f439b8
--- a/eng/common/darc-init.sh
+++ b/eng/common/darc-init.sh
@@ -68,7 +68,7 @@ function InstallDarcCli {
fi
fi
- local arcadeServicesSource="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json"
+ local arcadeServicesSource="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json"
echo "Installing Darc CLI version $darcVersion..."
echo "You may need to restart your command shell if this is the first dotnet tool you have installed."
diff --git a/eng/common/dotnet.cmd b/eng/common/dotnet.cmd
new file mode 100644
index 0000000000..527fa4bb38
--- /dev/null
+++ b/eng/common/dotnet.cmd
@@ -0,0 +1,7 @@
+@echo off
+
+:: This script is used to install the .NET SDK.
+:: It will also invoke the SDK with any provided arguments.
+
+powershell -ExecutionPolicy ByPass -NoProfile -command "& """%~dp0dotnet.ps1""" %*"
+exit /b %ErrorLevel%
diff --git a/eng/common/dotnet.ps1 b/eng/common/dotnet.ps1
new file mode 100644
index 0000000000..45e5676c9e
--- /dev/null
+++ b/eng/common/dotnet.ps1
@@ -0,0 +1,11 @@
+# This script is used to install the .NET SDK.
+# It will also invoke the SDK with any provided arguments.
+
+. $PSScriptRoot\tools.ps1
+$dotnetRoot = InitializeDotNetCli -install:$true
+
+# Invoke acquired SDK with args if they are provided
+if ($args.count -gt 0) {
+ $env:DOTNET_NOLOGO=1
+ & "$dotnetRoot\dotnet.exe" $args
+}
diff --git a/eng/common/dotnet.sh b/eng/common/dotnet.sh
new file mode 100755
index 0000000000..2ef6823567
--- /dev/null
+++ b/eng/common/dotnet.sh
@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+
+# This script is used to install the .NET SDK.
+# It will also invoke the SDK with any provided arguments.
+
+source="${BASH_SOURCE[0]}"
+# resolve $SOURCE until the file is no longer a symlink
+while [[ -h $source ]]; do
+ scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
+ source="$(readlink "$source")"
+
+ # if $source was a relative symlink, we need to resolve it relative to the path where the
+ # symlink file was located
+ [[ $source != /* ]] && source="$scriptroot/$source"
+done
+scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
+
+source $scriptroot/tools.sh
+InitializeDotNetCli true # install
+
+# Invoke acquired SDK with args if they are provided
+if [[ $# > 0 ]]; then
+ __dotnetDir=${_InitializeDotNetCli}
+ dotnetPath=${__dotnetDir}/dotnet
+ ${dotnetPath} "$@"
+fi
diff --git a/eng/common/generate-sbom-prep.sh b/eng/common/generate-sbom-prep.sh
old mode 100644
new mode 100755
diff --git a/eng/common/init-tools-native.sh b/eng/common/init-tools-native.sh
old mode 100644
new mode 100755
diff --git a/eng/common/internal-feed-operations.sh b/eng/common/internal-feed-operations.sh
old mode 100644
new mode 100755
diff --git a/eng/common/internal/NuGet.config b/eng/common/internal/NuGet.config
index 19d3d311b1..f70261ed68 100644
--- a/eng/common/internal/NuGet.config
+++ b/eng/common/internal/NuGet.config
@@ -4,4 +4,7 @@
+
+
+
diff --git a/eng/common/native/common-library.sh b/eng/common/native/common-library.sh
old mode 100644
new mode 100755
diff --git a/eng/common/native/install-cmake-test.sh b/eng/common/native/install-cmake-test.sh
old mode 100644
new mode 100755
diff --git a/eng/common/native/install-cmake.sh b/eng/common/native/install-cmake.sh
old mode 100644
new mode 100755
diff --git a/eng/common/native/install-dependencies.sh b/eng/common/native/install-dependencies.sh
old mode 100644
new mode 100755
index 13d849ae93..477a44f335
--- a/eng/common/native/install-dependencies.sh
+++ b/eng/common/native/install-dependencies.sh
@@ -45,7 +45,7 @@ case "$os" in
export HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK=1
# Skip brew update for now, see https://github.com/actions/setup-python/issues/577
# brew update --preinstall
- brew bundle --no-upgrade --no-lock --file=- < Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
+ Write-Host " -excludeCIBinaryLog When running on CI, allow no binary log (short: -nobl)"
Write-Host ""
Write-Host "Command line arguments not listed above are passed thru to msbuild."
}
@@ -34,10 +36,11 @@ function Print-Usage() {
function Build([string]$target) {
$logSuffix = if ($target -eq 'Execute') { '' } else { ".$target" }
$log = Join-Path $LogDir "$task$logSuffix.binlog"
+ $binaryLogArg = if ($binaryLog) { "/bl:$log" } else { "" }
$outputPath = Join-Path $ToolsetDir "$task\"
MSBuild $taskProject `
- /bl:$log `
+ $binaryLogArg `
/t:$target `
/p:Configuration=$configuration `
/p:RepoRoot=$RepoRoot `
diff --git a/eng/common/sdk-task.sh b/eng/common/sdk-task.sh
old mode 100644
new mode 100755
index b9b9e58db9..2f83adc026
--- a/eng/common/sdk-task.sh
+++ b/eng/common/sdk-task.sh
@@ -7,6 +7,10 @@ show_usage() {
echo " --verbosity Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic]"
echo " --help Print help and exit"
echo ""
+
+ echo "Advanced settings:"
+ echo " --excludeCIBinarylog Don't output binary log (short: -nobl)"
+ echo ""
echo "Command line arguments not listed above are passed thru to msbuild."
}
@@ -27,10 +31,12 @@ Build() {
local log_suffix=""
[[ "$target" != "Execute" ]] && log_suffix=".$target"
local log="$log_dir/$task$log_suffix.binlog"
+ local binaryLogArg=""
+ [[ $binary_log == true ]] && binaryLogArg="/bl:$log"
local output_path="$toolset_dir/$task/"
MSBuild "$taskProject" \
- /bl:"$log" \
+ $binaryLogArg \
/t:"$target" \
/p:Configuration="$configuration" \
/p:RepoRoot="$repo_root" \
@@ -39,8 +45,10 @@ Build() {
$properties
}
+binary_log=true
configuration="Debug"
verbosity="minimal"
+exclude_ci_binary_log=false
restore=false
help=false
properties=''
@@ -60,6 +68,11 @@ while (($# > 0)); do
verbosity=$2
shift 2
;;
+ --excludecibinarylog|--nobl)
+ binary_log=false
+ exclude_ci_binary_log=true
+ shift 1
+ ;;
--help)
help=true
shift 1
@@ -72,7 +85,6 @@ while (($# > 0)); do
done
ci=true
-binaryLog=true
warnAsError=true
if $help; then
diff --git a/eng/common/sdl/packages.config b/eng/common/sdl/packages.config
index 4585cfd6bb..e5f543ea68 100644
--- a/eng/common/sdl/packages.config
+++ b/eng/common/sdl/packages.config
@@ -1,4 +1,4 @@
-
+
diff --git a/eng/common/templates-official/job/job.yml b/eng/common/templates-official/job/job.yml
index 817555505a..a8a9432874 100644
--- a/eng/common/templates-official/job/job.yml
+++ b/eng/common/templates-official/job/job.yml
@@ -31,6 +31,7 @@ jobs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/artifacts'
ArtifactName: ${{ coalesce(parameters.artifacts.publish.artifacts.name , 'Artifacts_$(Agent.Os)_$(_BuildConfig)') }}
condition: always()
+ retryCountOnTaskFailure: 10 # for any logs being locked
continueOnError: true
- ${{ if and(ne(parameters.artifacts.publish.logs, 'false'), ne(parameters.artifacts.publish.logs, '')) }}:
- output: pipelineArtifact
@@ -39,6 +40,7 @@ jobs:
displayName: 'Publish logs'
continueOnError: true
condition: always()
+ retryCountOnTaskFailure: 10 # for any logs being locked
sbomEnabled: false # we don't need SBOM for logs
- ${{ if eq(parameters.enablePublishBuildArtifacts, true) }}:
@@ -46,7 +48,7 @@ jobs:
displayName: Publish Logs
PathtoPublish: '$(Build.ArtifactStagingDirectory)/artifacts/log/$(_BuildConfig)'
publishLocation: Container
- ArtifactName: ${{ coalesce(parameters.enablePublishBuildArtifacts.artifactName, '$(Agent.Os)_$(Agent.JobName)' ) }}
+ ArtifactName: ${{ coalesce(parameters.enablePublishBuildArtifacts.artifactName, '$(Agent.Os)_$(Agent.JobName)_Attempt$(System.JobAttempt)' ) }}
continueOnError: true
condition: always()
sbomEnabled: false # we don't need SBOM for logs
diff --git a/eng/common/templates-official/steps/publish-build-artifacts.yml b/eng/common/templates-official/steps/publish-build-artifacts.yml
index 100a3fc984..fcf6637b2e 100644
--- a/eng/common/templates-official/steps/publish-build-artifacts.yml
+++ b/eng/common/templates-official/steps/publish-build-artifacts.yml
@@ -24,6 +24,10 @@ parameters:
- name: is1ESPipeline
type: boolean
default: true
+
+- name: retryCountOnTaskFailure
+ type: string
+ default: 10
steps:
- ${{ if ne(parameters.is1ESPipeline, true) }}:
@@ -38,4 +42,5 @@ steps:
PathtoPublish: ${{ parameters.pathToPublish }}
${{ if parameters.artifactName }}:
ArtifactName: ${{ parameters.artifactName }}
-
+ ${{ if parameters.retryCountOnTaskFailure }}:
+ retryCountOnTaskFailure: ${{ parameters.retryCountOnTaskFailure }}
diff --git a/eng/common/templates/job/job.yml b/eng/common/templates/job/job.yml
index d1aeb92fce..7cbf668c22 100644
--- a/eng/common/templates/job/job.yml
+++ b/eng/common/templates/job/job.yml
@@ -46,6 +46,7 @@ jobs:
artifactName: ${{ coalesce(parameters.artifacts.publish.artifacts.name , 'Artifacts_$(Agent.Os)_$(_BuildConfig)') }}
continueOnError: true
condition: always()
+ retryCountOnTaskFailure: 10 # for any logs being locked
- ${{ if and(ne(parameters.artifacts.publish.logs, 'false'), ne(parameters.artifacts.publish.logs, '')) }}:
- template: /eng/common/core-templates/steps/publish-pipeline-artifacts.yml
parameters:
@@ -56,6 +57,7 @@ jobs:
displayName: 'Publish logs'
continueOnError: true
condition: always()
+ retryCountOnTaskFailure: 10 # for any logs being locked
sbomEnabled: false # we don't need SBOM for logs
- ${{ if ne(parameters.enablePublishBuildArtifacts, 'false') }}:
@@ -66,7 +68,7 @@ jobs:
displayName: Publish Logs
pathToPublish: '$(Build.ArtifactStagingDirectory)/artifacts/log/$(_BuildConfig)'
publishLocation: Container
- artifactName: ${{ coalesce(parameters.enablePublishBuildArtifacts.artifactName, '$(Agent.Os)_$(Agent.JobName)' ) }}
+ artifactName: ${{ coalesce(parameters.enablePublishBuildArtifacts.artifactName, '$(Agent.Os)_$(Agent.JobName)_Attempt$(System.JobAttempt)' ) }}
continueOnError: true
condition: always()
diff --git a/eng/common/templates/steps/publish-build-artifacts.yml b/eng/common/templates/steps/publish-build-artifacts.yml
index 6428a98dfe..605e602e94 100644
--- a/eng/common/templates/steps/publish-build-artifacts.yml
+++ b/eng/common/templates/steps/publish-build-artifacts.yml
@@ -25,6 +25,10 @@ parameters:
type: string
default: 'Container'
+- name: retryCountOnTaskFailure
+ type: string
+ default: 10
+
steps:
- ${{ if eq(parameters.is1ESPipeline, true) }}:
- 'eng/common/templates cannot be referenced from a 1ES managed template': error
@@ -37,4 +41,6 @@ steps:
PublishLocation: ${{ parameters.publishLocation }}
PathtoPublish: ${{ parameters.pathToPublish }}
${{ if parameters.artifactName }}:
- ArtifactName: ${{ parameters.artifactName }}
\ No newline at end of file
+ ArtifactName: ${{ parameters.artifactName }}
+ ${{ if parameters.retryCountOnTaskFailure }}:
+ retryCountOnTaskFailure: ${{ parameters.retryCountOnTaskFailure }}
diff --git a/eng/common/templates/steps/vmr-sync.yml b/eng/common/templates/steps/vmr-sync.yml
new file mode 100644
index 0000000000..599afb6186
--- /dev/null
+++ b/eng/common/templates/steps/vmr-sync.yml
@@ -0,0 +1,207 @@
+### These steps synchronize new code from product repositories into the VMR (https://github.com/dotnet/dotnet).
+### They initialize the darc CLI and pull the new updates.
+### Changes are applied locally onto the already cloned VMR (located in $vmrPath).
+
+parameters:
+- name: targetRef
+ displayName: Target revision in dotnet/ to synchronize
+ type: string
+ default: $(Build.SourceVersion)
+
+- name: vmrPath
+ displayName: Path where the dotnet/dotnet is checked out to
+ type: string
+ default: $(Agent.BuildDirectory)/vmr
+
+- name: additionalSyncs
+ displayName: Optional list of package names whose repo's source will also be synchronized in the local VMR, e.g. NuGet.Protocol
+ type: object
+ default: []
+
+steps:
+- checkout: vmr
+ displayName: Clone dotnet/dotnet
+ path: vmr
+ clean: true
+
+- checkout: self
+ displayName: Clone $(Build.Repository.Name)
+ path: repo
+ fetchDepth: 0
+
+# This step is needed so that when we get a detached HEAD / shallow clone,
+# we still pull the commit into the temporary repo clone to use it during the sync.
+# Also unshallow the clone so that forwardflow command would work.
+- script: |
+ git branch repo-head
+ git rev-parse HEAD
+ displayName: Label PR commit
+ workingDirectory: $(Agent.BuildDirectory)/repo
+
+- script: |
+ vmr_sha=$(grep -oP '(?<=Sha=")[^"]*' $(Agent.BuildDirectory)/repo/eng/Version.Details.xml)
+ echo "##vso[task.setvariable variable=vmr_sha]$vmr_sha"
+ displayName: Obtain the vmr sha from Version.Details.xml (Unix)
+ condition: ne(variables['Agent.OS'], 'Windows_NT')
+ workingDirectory: $(Agent.BuildDirectory)/repo
+
+- powershell: |
+ [xml]$xml = Get-Content -Path $(Agent.BuildDirectory)/repo/eng/Version.Details.xml
+ $vmr_sha = $xml.SelectSingleNode("//Source").Sha
+ Write-Output "##vso[task.setvariable variable=vmr_sha]$vmr_sha"
+ displayName: Obtain the vmr sha from Version.Details.xml (Windows)
+ condition: eq(variables['Agent.OS'], 'Windows_NT')
+ workingDirectory: $(Agent.BuildDirectory)/repo
+
+- script: |
+ git fetch --all
+ git checkout $(vmr_sha)
+ displayName: Checkout VMR at correct sha for repo flow
+ workingDirectory: ${{ parameters.vmrPath }}
+
+- script: |
+ git config --global user.name "dotnet-maestro[bot]"
+ git config --global user.email "dotnet-maestro[bot]@users.noreply.github.com"
+ displayName: Set git author to dotnet-maestro[bot]
+ workingDirectory: ${{ parameters.vmrPath }}
+
+- script: |
+ ./eng/common/vmr-sync.sh \
+ --vmr ${{ parameters.vmrPath }} \
+ --tmp $(Agent.TempDirectory) \
+ --azdev-pat '$(dn-bot-all-orgs-code-r)' \
+ --ci \
+ --debug
+
+ if [ "$?" -ne 0 ]; then
+ echo "##vso[task.logissue type=error]Failed to synchronize the VMR"
+ exit 1
+ fi
+ displayName: Sync repo into VMR (Unix)
+ condition: ne(variables['Agent.OS'], 'Windows_NT')
+ workingDirectory: $(Agent.BuildDirectory)/repo
+
+- script: |
+ git config --global diff.astextplain.textconv echo
+ git config --system core.longpaths true
+ displayName: Configure Windows git (longpaths, astextplain)
+ condition: eq(variables['Agent.OS'], 'Windows_NT')
+
+- powershell: |
+ ./eng/common/vmr-sync.ps1 `
+ -vmr ${{ parameters.vmrPath }} `
+ -tmp $(Agent.TempDirectory) `
+ -azdevPat '$(dn-bot-all-orgs-code-r)' `
+ -ci `
+ -debugOutput
+
+ if ($LASTEXITCODE -ne 0) {
+ echo "##vso[task.logissue type=error]Failed to synchronize the VMR"
+ exit 1
+ }
+ displayName: Sync repo into VMR (Windows)
+ condition: eq(variables['Agent.OS'], 'Windows_NT')
+ workingDirectory: $(Agent.BuildDirectory)/repo
+
+- ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
+ - task: CopyFiles@2
+ displayName: Collect failed patches
+ condition: failed()
+ inputs:
+ SourceFolder: '$(Agent.TempDirectory)'
+ Contents: '*.patch'
+ TargetFolder: '$(Build.ArtifactStagingDirectory)/FailedPatches'
+
+ - publish: '$(Build.ArtifactStagingDirectory)/FailedPatches'
+ artifact: $(System.JobDisplayName)_FailedPatches
+ displayName: Upload failed patches
+ condition: failed()
+
+- ${{ each assetName in parameters.additionalSyncs }}:
+ # The vmr-sync script ends up staging files in the local VMR so we have to commit those
+ - script:
+ git commit --allow-empty -am "Forward-flow $(Build.Repository.Name)"
+ displayName: Commit local VMR changes
+ workingDirectory: ${{ parameters.vmrPath }}
+
+ - script: |
+ set -ex
+
+ echo "Searching for details of asset ${{ assetName }}..."
+
+ # Use darc to get dependencies information
+ dependencies=$(./.dotnet/dotnet darc get-dependencies --name '${{ assetName }}' --ci)
+
+ # Extract repository URL and commit hash
+ repository=$(echo "$dependencies" | grep 'Repo:' | sed 's/Repo:[[:space:]]*//' | head -1)
+
+ if [ -z "$repository" ]; then
+ echo "##vso[task.logissue type=error]Asset ${{ assetName }} not found in the dependency list"
+ exit 1
+ fi
+
+ commit=$(echo "$dependencies" | grep 'Commit:' | sed 's/Commit:[[:space:]]*//' | head -1)
+
+ echo "Updating the VMR from $repository / $commit..."
+ cd ..
+ git clone $repository ${{ assetName }}
+ cd ${{ assetName }}
+ git checkout $commit
+ git branch "sync/$commit"
+
+ ./eng/common/vmr-sync.sh \
+ --vmr ${{ parameters.vmrPath }} \
+ --tmp $(Agent.TempDirectory) \
+ --azdev-pat '$(dn-bot-all-orgs-code-r)' \
+ --ci \
+ --debug
+
+ if [ "$?" -ne 0 ]; then
+ echo "##vso[task.logissue type=error]Failed to synchronize the VMR"
+ exit 1
+ fi
+ displayName: Sync ${{ assetName }} into (Unix)
+ condition: ne(variables['Agent.OS'], 'Windows_NT')
+ workingDirectory: $(Agent.BuildDirectory)/repo
+
+ - powershell: |
+ $ErrorActionPreference = 'Stop'
+
+ Write-Host "Searching for details of asset ${{ assetName }}..."
+
+ $dependencies = .\.dotnet\dotnet darc get-dependencies --name '${{ assetName }}' --ci
+
+ $repository = $dependencies | Select-String -Pattern 'Repo:\s+([^\s]+)' | Select-Object -First 1
+ $repository -match 'Repo:\s+([^\s]+)' | Out-Null
+ $repository = $matches[1]
+
+ if ($repository -eq $null) {
+ Write-Error "Asset ${{ assetName }} not found in the dependency list"
+ exit 1
+ }
+
+ $commit = $dependencies | Select-String -Pattern 'Commit:\s+([^\s]+)' | Select-Object -First 1
+ $commit -match 'Commit:\s+([^\s]+)' | Out-Null
+ $commit = $matches[1]
+
+ Write-Host "Updating the VMR from $repository / $commit..."
+ cd ..
+ git clone $repository ${{ assetName }}
+ cd ${{ assetName }}
+ git checkout $commit
+ git branch "sync/$commit"
+
+ .\eng\common\vmr-sync.ps1 `
+ -vmr ${{ parameters.vmrPath }} `
+ -tmp $(Agent.TempDirectory) `
+ -azdevPat '$(dn-bot-all-orgs-code-r)' `
+ -ci `
+ -debugOutput
+
+ if ($LASTEXITCODE -ne 0) {
+ echo "##vso[task.logissue type=error]Failed to synchronize the VMR"
+ exit 1
+ }
+ displayName: Sync ${{ assetName }} into (Windows)
+ condition: ne(variables['Agent.OS'], 'Windows_NT')
+ workingDirectory: $(Agent.BuildDirectory)/repo
diff --git a/eng/common/templates/vmr-build-pr.yml b/eng/common/templates/vmr-build-pr.yml
new file mode 100644
index 0000000000..ce3c29a62f
--- /dev/null
+++ b/eng/common/templates/vmr-build-pr.yml
@@ -0,0 +1,42 @@
+# This pipeline is used for running the VMR verification of the PR changes in repo-level PRs.
+#
+# It will run a full set of verification jobs defined in:
+# https://github.com/dotnet/dotnet/blob/10060d128e3f470e77265f8490f5e4f72dae738e/eng/pipelines/templates/stages/vmr-build.yml#L27-L38
+#
+# For repos that do not need to run the full set, you would do the following:
+#
+# 1. Copy this YML file to a repo-specific location, i.e. outside of eng/common.
+#
+# 2. Add `verifications` parameter to VMR template reference
+#
+# Examples:
+# - For source-build stage 1 verification, add the following:
+# verifications: [ "source-build-stage1" ]
+#
+# - For Windows only verifications, add the following:
+# verifications: [ "unified-build-windows-x64", "unified-build-windows-x86" ]
+
+trigger: none
+pr: none
+
+variables:
+- template: /eng/common/templates/variables/pool-providers.yml@self
+
+- name: skipComponentGovernanceDetection # we run CG on internal builds only
+ value: true
+
+- name: Codeql.Enabled # we run CodeQL on internal builds only
+ value: false
+
+resources:
+ repositories:
+ - repository: vmr
+ type: github
+ name: dotnet/dotnet
+ endpoint: dotnet
+
+stages:
+- template: /eng/pipelines/templates/stages/vmr-build.yml@vmr
+ parameters:
+ isBuiltFromVmr: false
+ scope: lite
diff --git a/eng/common/tools.ps1 b/eng/common/tools.ps1
index 95ccdf82e4..40f0aa8612 100644
--- a/eng/common/tools.ps1
+++ b/eng/common/tools.ps1
@@ -65,10 +65,8 @@ $ErrorActionPreference = 'Stop'
# Base-64 encoded SAS token that has permission to storage container described by $runtimeSourceFeed
[string]$runtimeSourceFeedKey = if (Test-Path variable:runtimeSourceFeedKey) { $runtimeSourceFeedKey } else { $null }
-# True if the build is a product build
-[bool]$productBuild = if (Test-Path variable:productBuild) { $productBuild } else { $false }
-
-[String[]]$properties = if (Test-Path variable:properties) { $properties } else { @() }
+# True when the build is running within the VMR.
+[bool]$fromVMR = if (Test-Path variable:fromVMR) { $fromVMR } else { $false }
function Create-Directory ([string[]] $path) {
New-Item -Path $path -Force -ItemType 'Directory' | Out-Null
@@ -262,7 +260,7 @@ function GetDotNetInstallScript([string] $dotnetRoot) {
if (!(Test-Path $installScript)) {
Create-Directory $dotnetRoot
$ProgressPreference = 'SilentlyContinue' # Don't display the console progress UI - it's a huge perf hit
- $uri = "https://builds.dotnet.microsoft.com/dotnet/scripts/v1/dotnet-install.ps1"
+ $uri = "https://builds.dotnet.microsoft.com/dotnet/scripts/$dotnetInstallScriptVersion/dotnet-install.ps1"
Retry({
Write-Host "GET $uri"
@@ -646,7 +644,6 @@ function GetNuGetPackageCachePath() {
$env:NUGET_PACKAGES = Join-Path $env:UserProfile '.nuget\packages\'
} else {
$env:NUGET_PACKAGES = Join-Path $RepoRoot '.packages\'
- $env:RESTORENOHTTPCACHE = $true
}
}
@@ -768,28 +765,13 @@ function MSBuild() {
$toolsetBuildProject = InitializeToolset
$basePath = Split-Path -parent $toolsetBuildProject
- $possiblePaths = @(
- # new scripts need to work with old packages, so we need to look for the old names/versions
- (Join-Path $basePath (Join-Path $buildTool.Framework 'Microsoft.DotNet.ArcadeLogging.dll')),
- (Join-Path $basePath (Join-Path $buildTool.Framework 'Microsoft.DotNet.Arcade.Sdk.dll')),
-
- # This list doesn't need to be updated anymore and can eventually be removed.
- (Join-Path $basePath (Join-Path net9.0 'Microsoft.DotNet.ArcadeLogging.dll')),
- (Join-Path $basePath (Join-Path net9.0 'Microsoft.DotNet.Arcade.Sdk.dll')),
- (Join-Path $basePath (Join-Path net8.0 'Microsoft.DotNet.ArcadeLogging.dll')),
- (Join-Path $basePath (Join-Path net8.0 'Microsoft.DotNet.Arcade.Sdk.dll'))
- )
- $selectedPath = $null
- foreach ($path in $possiblePaths) {
- if (Test-Path $path -PathType Leaf) {
- $selectedPath = $path
- break
- }
- }
+ $selectedPath = Join-Path $basePath (Join-Path $buildTool.Framework 'Microsoft.DotNet.ArcadeLogging.dll')
+
if (-not $selectedPath) {
- Write-PipelineTelemetryError -Category 'Build' -Message 'Unable to find arcade sdk logger assembly.'
+ Write-PipelineTelemetryError -Category 'Build' -Message "Unable to find arcade sdk logger assembly: $selectedPath"
ExitWithExitCode 1
}
+
$args += "/logger:$selectedPath"
}
@@ -852,8 +834,8 @@ function MSBuild-Core() {
}
# When running on Azure Pipelines, override the returned exit code to avoid double logging.
- # Skip this when the build is a child of the VMR orchestrator build.
- if ($ci -and $env:SYSTEM_TEAMPROJECT -ne $null -and !$productBuild -and -not($properties -like "*DotNetBuildRepo=true*")) {
+ # Skip this when the build is a child of the VMR build.
+ if ($ci -and $env:SYSTEM_TEAMPROJECT -ne $null -and !$fromVMR) {
Write-PipelineSetResult -Result "Failed" -Message "msbuild execution failed."
# Exiting with an exit code causes the azure pipelines task to log yet another "noise" error
# The above Write-PipelineSetResult will cause the task to be marked as failure without adding yet another error
diff --git a/eng/common/tools.sh b/eng/common/tools.sh
old mode 100644
new mode 100755
index 4a5fa99478..3def02a638
--- a/eng/common/tools.sh
+++ b/eng/common/tools.sh
@@ -5,6 +5,9 @@
# CI mode - set to true on CI server for PR validation build or official build.
ci=${ci:-false}
+# Build mode
+source_build=${source_build:-false}
+
# Set to true to use the pipelines logger which will enable Azure logging output.
# https://github.com/Microsoft/azure-pipelines-tasks/blob/master/docs/authoring/commands.md
# This flag is meant as a temporary opt-opt for the feature while validate it across
@@ -58,7 +61,8 @@ use_installed_dotnet_cli=${use_installed_dotnet_cli:-true}
dotnetInstallScriptVersion=${dotnetInstallScriptVersion:-'v1'}
# True to use global NuGet cache instead of restoring packages to repository-local directory.
-if [[ "$ci" == true ]]; then
+# Keep in sync with NuGetPackageroot in Arcade SDK's RepositoryLayout.props.
+if [[ "$ci" == true || "$source_build" == true ]]; then
use_global_nuget_cache=${use_global_nuget_cache:-false}
else
use_global_nuget_cache=${use_global_nuget_cache:-true}
@@ -68,8 +72,8 @@ fi
runtime_source_feed=${runtime_source_feed:-''}
runtime_source_feed_key=${runtime_source_feed_key:-''}
-# True if the build is a product build
-product_build=${product_build:-false}
+# True when the build is running within the VMR.
+from_vmr=${from_vmr:-false}
# Resolve any symlinks in the given path.
function ResolvePath {
@@ -295,7 +299,7 @@ function with_retries {
function GetDotNetInstallScript {
local root=$1
local install_script="$root/dotnet-install.sh"
- local install_script_url="https://builds.dotnet.microsoft.com/dotnet/scripts/v1/dotnet-install.sh"
+ local install_script_url="https://builds.dotnet.microsoft.com/dotnet/scripts/$dotnetInstallScriptVersion/dotnet-install.sh"
if [[ ! -a "$install_script" ]]; then
mkdir -p "$root"
@@ -341,14 +345,12 @@ function InitializeBuildTool {
_InitializeBuildToolCommand="msbuild"
}
-# Set RestoreNoHttpCache as a workaround for https://github.com/NuGet/Home/issues/3116
function GetNuGetPackageCachePath {
if [[ -z ${NUGET_PACKAGES:-} ]]; then
if [[ "$use_global_nuget_cache" == true ]]; then
export NUGET_PACKAGES="$HOME/.nuget/packages/"
else
export NUGET_PACKAGES="$repo_root/.packages/"
- export RESTORENOHTTPCACHE=true
fi
fi
@@ -445,27 +447,13 @@ function MSBuild {
fi
local toolset_dir="${_InitializeToolset%/*}"
- # new scripts need to work with old packages, so we need to look for the old names/versions
- local selectedPath=
- local possiblePaths=()
- possiblePaths+=( "$toolset_dir/net/Microsoft.DotNet.ArcadeLogging.dll" )
- possiblePaths+=( "$toolset_dir/net/Microsoft.DotNet.Arcade.Sdk.dll" )
-
- # This list doesn't need to be updated anymore and can eventually be removed.
- possiblePaths+=( "$toolset_dir/net9.0/Microsoft.DotNet.ArcadeLogging.dll" )
- possiblePaths+=( "$toolset_dir/net9.0/Microsoft.DotNet.Arcade.Sdk.dll" )
- possiblePaths+=( "$toolset_dir/net8.0/Microsoft.DotNet.ArcadeLogging.dll" )
- possiblePaths+=( "$toolset_dir/net8.0/Microsoft.DotNet.Arcade.Sdk.dll" )
- for path in "${possiblePaths[@]}"; do
- if [[ -f $path ]]; then
- selectedPath=$path
- break
- fi
- done
+ local selectedPath="$toolset_dir/net/Microsoft.DotNet.ArcadeLogging.dll"
+
if [[ -z "$selectedPath" ]]; then
- Write-PipelineTelemetryError -category 'Build' "Unable to find arcade sdk logger assembly."
+ Write-PipelineTelemetryError -category 'Build' "Unable to find arcade sdk logger assembly: $selectedPath"
ExitWithExitCode 1
fi
+
args+=( "-logger:$selectedPath" )
fi
@@ -502,8 +490,8 @@ function MSBuild-Core {
echo "Build failed with exit code $exit_code. Check errors above."
# When running on Azure Pipelines, override the returned exit code to avoid double logging.
- # Skip this when the build is a child of the VMR orchestrator build.
- if [[ "$ci" == true && -n ${SYSTEM_TEAMPROJECT:-} && "$product_build" != true && "$properties" != *"DotNetBuildRepo=true"* ]]; then
+ # Skip this when the build is a child of the VMR build.
+ if [[ "$ci" == true && -n ${SYSTEM_TEAMPROJECT:-} && "$from_vmr" != true ]]; then
Write-PipelineSetResult -result "Failed" -message "msbuild execution failed."
# Exiting with an exit code causes the azure pipelines task to log yet another "noise" error
# The above Write-PipelineSetResult will cause the task to be marked as failure without adding yet another error
@@ -526,6 +514,7 @@ function GetDarc {
fi
"$eng_root/common/darc-init.sh" --toolpath "$darc_path" $version
+ darc_tool="$darc_path/darc"
}
# Returns a full path to an Arcade SDK task project file.
diff --git a/eng/common/vmr-sync.ps1 b/eng/common/vmr-sync.ps1
new file mode 100755
index 0000000000..97302f3205
--- /dev/null
+++ b/eng/common/vmr-sync.ps1
@@ -0,0 +1,138 @@
+<#
+.SYNOPSIS
+
+This script is used for synchronizing the current repository into a local VMR.
+It pulls the current repository's code into the specified VMR directory for local testing or
+Source-Build validation.
+
+.DESCRIPTION
+
+The tooling used for synchronization will clone the VMR repository into a temporary folder if
+it does not already exist. These clones can be reused in future synchronizations, so it is
+recommended to dedicate a folder for this to speed up re-runs.
+
+.EXAMPLE
+ Synchronize current repository into a local VMR:
+ ./vmr-sync.ps1 -vmrDir "$HOME/repos/dotnet" -tmpDir "$HOME/repos/tmp"
+
+.PARAMETER tmpDir
+Required. Path to the temporary folder where repositories will be cloned
+
+.PARAMETER vmrBranch
+Optional. Branch of the 'dotnet/dotnet' repo to synchronize. The VMR will be checked out to this branch
+
+.PARAMETER azdevPat
+Optional. Azure DevOps PAT to use for cloning private repositories.
+
+.PARAMETER vmrDir
+Optional. Path to the dotnet/dotnet repository. When null, gets cloned to the temporary folder
+
+.PARAMETER debugOutput
+Optional. Enables debug logging in the darc vmr command.
+
+.PARAMETER ci
+Optional. Denotes that the script is running in a CI environment.
+#>
+param (
+ [Parameter(Mandatory=$true, HelpMessage="Path to the temporary folder where repositories will be cloned")]
+ [string][Alias('t', 'tmp')]$tmpDir,
+ [string][Alias('b', 'branch')]$vmrBranch,
+ [string]$remote,
+ [string]$azdevPat,
+ [string][Alias('v', 'vmr')]$vmrDir,
+ [switch]$ci,
+ [switch]$debugOutput
+)
+
+function Fail {
+ Write-Host "> $($args[0])" -ForegroundColor 'Red'
+}
+
+function Highlight {
+ Write-Host "> $($args[0])" -ForegroundColor 'Cyan'
+}
+
+$verbosity = 'verbose'
+if ($debugOutput) {
+ $verbosity = 'debug'
+}
+# Validation
+
+if (-not $tmpDir) {
+ Fail "Missing -tmpDir argument. Please specify the path to the temporary folder where the repositories will be cloned"
+ exit 1
+}
+
+# Sanitize the input
+
+if (-not $vmrDir) {
+ $vmrDir = Join-Path $tmpDir 'dotnet'
+}
+
+if (-not (Test-Path -Path $tmpDir -PathType Container)) {
+ New-Item -ItemType Directory -Path $tmpDir | Out-Null
+}
+
+# Prepare the VMR
+
+if (-not (Test-Path -Path $vmrDir -PathType Container)) {
+ Highlight "Cloning 'dotnet/dotnet' into $vmrDir.."
+ git clone https://github.com/dotnet/dotnet $vmrDir
+
+ if ($vmrBranch) {
+ git -C $vmrDir switch -c $vmrBranch
+ }
+}
+else {
+ if ((git -C $vmrDir diff --quiet) -eq $false) {
+ Fail "There are changes in the working tree of $vmrDir. Please commit or stash your changes"
+ exit 1
+ }
+
+ if ($vmrBranch) {
+ Highlight "Preparing $vmrDir"
+ git -C $vmrDir checkout $vmrBranch
+ git -C $vmrDir pull
+ }
+}
+
+Set-StrictMode -Version Latest
+
+# Prepare darc
+
+Highlight 'Installing .NET, preparing the tooling..'
+. .\eng\common\tools.ps1
+$dotnetRoot = InitializeDotNetCli -install:$true
+$darc = Get-Darc
+$dotnet = "$dotnetRoot\dotnet.exe"
+
+Highlight "Starting the synchronization of VMR.."
+
+# Synchronize the VMR
+$darcArgs = (
+ "vmr", "forwardflow",
+ "--tmp", $tmpDir,
+ "--$verbosity",
+ $vmrDir
+)
+
+if ($ci) {
+ $darcArgs += ("--ci")
+}
+
+if ($azdevPat) {
+ $darcArgs += ("--azdev-pat", $azdevPat)
+}
+
+& "$darc" $darcArgs
+
+if ($LASTEXITCODE -eq 0) {
+ Highlight "Synchronization succeeded"
+}
+else {
+ Fail "Synchronization of repo to VMR failed!"
+ Fail "'$vmrDir' is left in its last state (re-run of this script will reset it)."
+ Fail "Please inspect the logs which contain path to the failing patch file (use -debugOutput to get all the details)."
+ Fail "Once you make changes to the conflicting VMR patch, commit it locally and re-run this script."
+ exit 1
+}
diff --git a/eng/common/vmr-sync.sh b/eng/common/vmr-sync.sh
new file mode 100755
index 0000000000..44239e331c
--- /dev/null
+++ b/eng/common/vmr-sync.sh
@@ -0,0 +1,207 @@
+#!/bin/bash
+
+### This script is used for synchronizing the current repository into a local VMR.
+### It pulls the current repository's code into the specified VMR directory for local testing or
+### Source-Build validation.
+###
+### The tooling used for synchronization will clone the VMR repository into a temporary folder if
+### it does not already exist. These clones can be reused in future synchronizations, so it is
+### recommended to dedicate a folder for this to speed up re-runs.
+###
+### USAGE:
+### Synchronize current repository into a local VMR:
+### ./vmr-sync.sh --tmp "$HOME/repos/tmp" "$HOME/repos/dotnet"
+###
+### Options:
+### -t, --tmp, --tmp-dir PATH
+### Required. Path to the temporary folder where repositories will be cloned
+###
+### -b, --branch, --vmr-branch BRANCH_NAME
+### Optional. Branch of the 'dotnet/dotnet' repo to synchronize. The VMR will be checked out to this branch
+###
+### --debug
+### Optional. Turns on the most verbose logging for the VMR tooling
+###
+### --remote name:URI
+### Optional. Additional remote to use during the synchronization
+### This can be used to synchronize to a commit from a fork of the repository
+### Example: 'runtime:https://github.com/yourfork/runtime'
+###
+### --azdev-pat
+### Optional. Azure DevOps PAT to use for cloning private repositories.
+###
+### -v, --vmr, --vmr-dir PATH
+### Optional. Path to the dotnet/dotnet repository. When null, gets cloned to the temporary folder
+
+source="${BASH_SOURCE[0]}"
+
+# resolve $source until the file is no longer a symlink
+while [[ -h "$source" ]]; do
+ scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
+ source="$(readlink "$source")"
+ # if $source was a relative symlink, we need to resolve it relative to the path where the
+ # symlink file was located
+ [[ $source != /* ]] && source="$scriptroot/$source"
+done
+scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
+
+function print_help () {
+ sed -n '/^### /,/^$/p' "$source" | cut -b 5-
+}
+
+COLOR_RED=$(tput setaf 1 2>/dev/null || true)
+COLOR_CYAN=$(tput setaf 6 2>/dev/null || true)
+COLOR_CLEAR=$(tput sgr0 2>/dev/null || true)
+COLOR_RESET=uniquesearchablestring
+FAILURE_PREFIX='> '
+
+function fail () {
+ echo "${COLOR_RED}$FAILURE_PREFIX${1//${COLOR_RESET}/${COLOR_RED}}${COLOR_CLEAR}" >&2
+}
+
+function highlight () {
+ echo "${COLOR_CYAN}$FAILURE_PREFIX${1//${COLOR_RESET}/${COLOR_CYAN}}${COLOR_CLEAR}"
+}
+
+tmp_dir=''
+vmr_dir=''
+vmr_branch=''
+additional_remotes=''
+verbosity=verbose
+azdev_pat=''
+ci=false
+
+while [[ $# -gt 0 ]]; do
+ opt="$(echo "$1" | tr "[:upper:]" "[:lower:]")"
+ case "$opt" in
+ -t|--tmp|--tmp-dir)
+ tmp_dir=$2
+ shift
+ ;;
+ -v|--vmr|--vmr-dir)
+ vmr_dir=$2
+ shift
+ ;;
+ -b|--branch|--vmr-branch)
+ vmr_branch=$2
+ shift
+ ;;
+ --remote)
+ additional_remotes="$additional_remotes $2"
+ shift
+ ;;
+ --azdev-pat)
+ azdev_pat=$2
+ shift
+ ;;
+ --ci)
+ ci=true
+ ;;
+ -d|--debug)
+ verbosity=debug
+ ;;
+ -h|--help)
+ print_help
+ exit 0
+ ;;
+ *)
+ fail "Invalid argument: $1"
+ print_help
+ exit 1
+ ;;
+ esac
+
+ shift
+done
+
+# Validation
+
+if [[ -z "$tmp_dir" ]]; then
+ fail "Missing --tmp-dir argument. Please specify the path to the temporary folder where the repositories will be cloned"
+ exit 1
+fi
+
+# Sanitize the input
+
+if [[ -z "$vmr_dir" ]]; then
+ vmr_dir="$tmp_dir/dotnet"
+fi
+
+if [[ ! -d "$tmp_dir" ]]; then
+ mkdir -p "$tmp_dir"
+fi
+
+if [[ "$verbosity" == "debug" ]]; then
+ set -x
+fi
+
+# Prepare the VMR
+
+if [[ ! -d "$vmr_dir" ]]; then
+ highlight "Cloning 'dotnet/dotnet' into $vmr_dir.."
+ git clone https://github.com/dotnet/dotnet "$vmr_dir"
+
+ if [[ -n "$vmr_branch" ]]; then
+ git -C "$vmr_dir" switch -c "$vmr_branch"
+ fi
+else
+ if ! git -C "$vmr_dir" diff --quiet; then
+ fail "There are changes in the working tree of $vmr_dir. Please commit or stash your changes"
+ exit 1
+ fi
+
+ if [[ -n "$vmr_branch" ]]; then
+ highlight "Preparing $vmr_dir"
+ git -C "$vmr_dir" checkout "$vmr_branch"
+ git -C "$vmr_dir" pull
+ fi
+fi
+
+set -e
+
+# Prepare darc
+
+highlight 'Installing .NET, preparing the tooling..'
+source "./eng/common/tools.sh"
+InitializeDotNetCli true
+GetDarc
+dotnetDir=$( cd ./.dotnet/; pwd -P )
+dotnet=$dotnetDir/dotnet
+
+highlight "Starting the synchronization of VMR.."
+set +e
+
+if [[ -n "$additional_remotes" ]]; then
+ additional_remotes="--additional-remotes $additional_remotes"
+fi
+
+if [[ -n "$azdev_pat" ]]; then
+ azdev_pat="--azdev-pat $azdev_pat"
+fi
+
+ci_arg=''
+if [[ "$ci" == "true" ]]; then
+ ci_arg="--ci"
+fi
+
+# Synchronize the VMR
+
+export DOTNET_ROOT="$dotnetDir"
+
+"$darc_tool" vmr forwardflow \
+ --tmp "$tmp_dir" \
+ $azdev_pat \
+ --$verbosity \
+ $ci_arg \
+ $additional_remotes \
+ "$vmr_dir"
+
+if [[ $? == 0 ]]; then
+ highlight "Synchronization succeeded"
+else
+ fail "Synchronization of repo to VMR failed!"
+ fail "'$vmr_dir' is left in its last state (re-run of this script will reset it)."
+ fail "Please inspect the logs which contain path to the failing patch file (use --debug to get all the details)."
+ fail "Once you make changes to the conflicting VMR patch, commit it locally and re-run this script."
+ exit 1
+fi
diff --git a/eng/native-prereqs.proj b/eng/native-prereqs.proj
index bea0af21d1..f017e9d0ac 100644
--- a/eng/native-prereqs.proj
+++ b/eng/native-prereqs.proj
@@ -67,6 +67,13 @@
-
+
+
+
+
+
diff --git a/eng/native/build-commons.sh b/eng/native/build-commons.sh
index 2cf33442a9..e0346c9d33 100755
--- a/eng/native/build-commons.sh
+++ b/eng/native/build-commons.sh
@@ -266,7 +266,7 @@ usage()
echo "-gccx.y: optional argument to build using gcc version x.y."
echo "-ninja: target ninja instead of GNU make"
echo "-numproc: set the number of build processes."
- echo "-outputrid: optional argument that overrides the target rid name."
+ echo "-targetrid: optional argument that overrides the target rid name."
echo "-portablebuild: pass -portablebuild=false to force a non-portable build."
echo "-skipconfigure: skip build configuration."
echo "-keepnativesymbols: keep native/unmanaged debug symbols."
@@ -286,7 +286,7 @@ source "$__RepoRootDir/eng/common/native/init-os-and-arch.sh"
__TargetArch=$arch
__TargetOS=$os
-__OutputRid=''
+__TargetRid=''
# Get the number of processors available to the scheduler
platform="$(uname -s | tr '[:upper:]' '[:lower:]')"
@@ -458,12 +458,12 @@ while :; do
__TargetArch=wasm
;;
- outputrid|-outputrid)
+ targetrid|-targetrid|outputrid|-outputrid)
if [[ -n "$2" ]]; then
- __OutputRid="$2"
+ __TargetRid="$2"
shift
else
- echo "ERROR: 'outputrid' requires a non-empty option argument"
+ echo "ERROR: 'targetrid' requires a non-empty option argument"
exit 1
fi
;;
@@ -565,15 +565,15 @@ fi
# init the target distro name (__DistroRid) and target portable os (__PortableTargetOS).
initTargetDistroRid
-if [ -z "$__OutputRid" ]; then
+if [ -z "$__TargetRid" ]; then
if [[ "$__PortableBuild" == 0 ]]; then
- __OutputRid="$__DistroRid"
+ __TargetRid="$__DistroRid"
else
- __OutputRid="$__PortableTargetOS-$__TargetArch"
+ __TargetRid="$__PortableTargetOS-$__TargetArch"
fi
fi
-export __OutputRid
-echo "__OutputRid: ${__OutputRid}"
+export __TargetRid
+echo "__TargetRid: ${__TargetRid}"
# When the host runs on an unknown rid, it falls back to the output rid
-__HostFallbackOS="${__OutputRid%-*}" # Strip architecture
+__HostFallbackOS="${__TargetRid%-*}" # Strip architecture
diff --git a/eng/native/configurecompiler.cmake b/eng/native/configurecompiler.cmake
index 35367f9605..312312daa3 100644
--- a/eng/native/configurecompiler.cmake
+++ b/eng/native/configurecompiler.cmake
@@ -19,9 +19,7 @@ set(CMAKE_TRY_COMPILE_CONFIGURATION Release)
include(CheckCCompilerFlag)
include(CheckCXXCompilerFlag)
-if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.18")
- include(CheckLinkerFlag)
-endif()
+include(CheckLinkerFlag)
# "configureoptimization.cmake" must be included after CLR_CMAKE_HOST_UNIX has been set.
include(${CMAKE_CURRENT_LIST_DIR}/configureoptimization.cmake)
@@ -672,22 +670,22 @@ if (CLR_CMAKE_HOST_UNIX)
set(DISABLE_OVERRIDING_MIN_VERSION_ERROR -Wno-overriding-t-option)
add_link_options(-Wno-overriding-t-option)
if(CLR_CMAKE_HOST_ARCH_ARM64)
- set(CLR_CMAKE_MACCATALYST_COMPILER_TARGET "arm64-apple-ios15.0-macabi")
- add_link_options(-target ${CLR_CMAKE_MACCATALYST_COMPILER_TARGET})
+ set(MACOS_VERSION_MIN_FLAGS "-target arm64-apple-ios15.0-macabi")
+ add_link_options(-target arm64-apple-ios15.0-macabi)
elseif(CLR_CMAKE_HOST_ARCH_AMD64)
- set(CLR_CMAKE_MACCATALYST_COMPILER_TARGET "x86_64-apple-ios15.0-macabi")
- add_link_options(-target ${CLR_CMAKE_MACCATALYST_COMPILER_TARGET})
+ set(MACOS_VERSION_MIN_FLAGS "-target x86_64-apple-ios15.0-macabi")
+ add_link_options(-target x86_64-apple-ios15.0-macabi)
else()
clr_unknown_arch()
endif()
# These options are intentionally set using the CMAKE_XXX_FLAGS instead of
# add_compile_options so that they take effect on the configuration functions
# in various configure.cmake files.
- set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -target ${CLR_CMAKE_MACCATALYST_COMPILER_TARGET} ${DISABLE_OVERRIDING_MIN_VERSION_ERROR}")
- set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -target ${CLR_CMAKE_MACCATALYST_COMPILER_TARGET} ${DISABLE_OVERRIDING_MIN_VERSION_ERROR}")
- set(CMAKE_ASM_FLAGS "${CMAKE_ASM_FLAGS} -target ${CLR_CMAKE_MACCATALYST_COMPILER_TARGET} ${DISABLE_OVERRIDING_MIN_VERSION_ERROR}")
- set(CMAKE_OBJC_FLAGS "${CMAKE_OBJC_FLAGS}-target ${CLR_CMAKE_MACCATALYST_COMPILER_TARGET} ${DISABLE_OVERRIDING_MIN_VERSION_ERROR}")
- set(CMAKE_OBJCXX_FLAGS "${CMAKE_OBJCXX_FLAGS} -target ${CLR_CMAKE_MACCATALYST_COMPILER_TARGET} ${DISABLE_OVERRIDING_MIN_VERSION_ERROR}")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${MACOS_VERSION_MIN_FLAGS} ${DISABLE_OVERRIDING_MIN_VERSION_ERROR}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${MACOS_VERSION_MIN_FLAGS} ${DISABLE_OVERRIDING_MIN_VERSION_ERROR}")
+ set(CMAKE_ASM_FLAGS "${CMAKE_ASM_FLAGS} ${MACOS_VERSION_MIN_FLAGS} ${DISABLE_OVERRIDING_MIN_VERSION_ERROR}")
+ set(CMAKE_OBJC_FLAGS "${CMAKE_OBJC_FLAGS} ${MACOS_VERSION_MIN_FLAGS} ${DISABLE_OVERRIDING_MIN_VERSION_ERROR}")
+ set(CMAKE_OBJCXX_FLAGS "${CMAKE_OBJCXX_FLAGS} ${MACOS_VERSION_MIN_FLAGS} ${DISABLE_OVERRIDING_MIN_VERSION_ERROR}")
elseif(CLR_CMAKE_HOST_OSX)
set(CMAKE_OSX_DEPLOYMENT_TARGET "12.0")
if(CLR_CMAKE_HOST_ARCH_ARM64)
@@ -799,6 +797,8 @@ if (MSVC)
add_compile_options($<$:/fp:precise>) # Enable precise floating point
# Disable C++ RTTI
+ # /GR is added by default by CMake, so remove it manually.
+ string(REPLACE "/GR " " " CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /GR-")
add_compile_options($<$:/FC>) # use full pathnames in diagnostics
diff --git a/eng/native/gen-buildsys.cmd b/eng/native/gen-buildsys.cmd
index 83682d69cf..3a6e1b5426 100644
--- a/eng/native/gen-buildsys.cmd
+++ b/eng/native/gen-buildsys.cmd
@@ -26,7 +26,7 @@ if /i "%__Ninja%" == "1" (
set __CmakeGenerator=Ninja
) else (
if /i NOT "%__Arch%" == "wasm" (
- if /i "%__VSVersion%" == "vs2022" (set __CmakeGenerator=%__CmakeGenerator% 17 2022)
+ if /i "%__VSVersion%" == "17.0" (set __CmakeGenerator=%__CmakeGenerator% 17 2022)
if /i "%__Arch%" == "x64" (set __ExtraCmakeParams=%__ExtraCmakeParams% -A x64)
if /i "%__Arch%" == "arm" (set __ExtraCmakeParams=%__ExtraCmakeParams% -A ARM)
@@ -80,6 +80,33 @@ if /i "%__Arch%" == "wasm" (
set __ExtraCmakeParams=%__ExtraCmakeParams% "-DCMAKE_SYSTEM_VERSION=10.0"
)
+if /i "%__Os%" == "android" (
+ :: Keep in sync with $(AndroidApiLevelMin) in Directory.Build.props in the repository rooot
+ set __ANDROID_API_LEVEL=21
+ if "%ANDROID_NDK_ROOT%" == "" (
+ echo Error: You need to set the ANDROID_NDK_ROOT environment variable pointing to the Android NDK root.
+ exit /B 1
+ )
+
+ set __ExtraCmakeParams=!__ExtraCmakeParams! "-DANDROID_BUILD=1" "-DANDROID_CPP_FEATURES='no-rtti exceptions'"
+ set __ExtraCmakeParams=!__ExtraCmakeParams! "-DANDROID_PLATFORM=android-!__ANDROID_API_LEVEL!" "-DANDROID_NATIVE_API_LEVEL=!__ANDROID_API_LEVEL!"
+
+ if "%__Arch%" == "x64" (
+ set __ExtraCmakeParams=!__ExtraCmakeParams! "-DANDROID_ABI=x86_64"
+ )
+ if "%__Arch%" == "x86" (
+ set __ExtraCmakeParams=!__ExtraCmakeParams! "-DANDROID_ABI=x86"
+ )
+ if "%__Arch%" == "arm64" (
+ set __ExtraCmakeParams=!__ExtraCmakeParams! "-DANDROID_ABI=arm64-v8a"
+ )
+ if "%__Arch%" == "arm" (
+ set __ExtraCmakeParams=!__ExtraCmakeParams! "-DANDROID_ABI=armeabi-v7a"
+ )
+
+ set __ExtraCmakeParams=!__ExtraCmakeParams! "-DCMAKE_TOOLCHAIN_FILE='%ANDROID_NDK_ROOT:\=/%/build/cmake/android.toolchain.cmake'" "-C %__repoRoot%/eng/native/tryrun.cmake"
+)
+
:loop
if [%6] == [] goto end_loop
set __ExtraCmakeParams=%__ExtraCmakeParams% %6
@@ -111,6 +138,7 @@ if not "%__ConfigureOnly%" == "1" (
if /i "%__UseEmcmake%" == "1" (
call "!EMSDK_PATH!/emsdk_env.cmd" > nul 2>&1 && emcmake "%CMakePath%" %__ExtraCmakeParams% --no-warn-unused-cli -G "%__CmakeGenerator%" -B %__IntermediatesDir% -S %__SourceDir%
) else (
+ echo "%CMakePath% %__ExtraCmakeParams% --no-warn-unused-cli -G %__CmakeGenerator% -B %__IntermediatesDir% -S %__SourceDir%"
"%CMakePath%" %__ExtraCmakeParams% --no-warn-unused-cli -G "%__CmakeGenerator%" -B %__IntermediatesDir% -S %__SourceDir%
)
diff --git a/eng/native/init-vs-env.cmd b/eng/native/init-vs-env.cmd
index 0d28cac989..6f417fc23f 100644
--- a/eng/native/init-vs-env.cmd
+++ b/eng/native/init-vs-env.cmd
@@ -53,11 +53,7 @@ set "__VSCOMNTOOLS="
set "VSCMD_START_DIR="
:VSDetected
-if "%VisualStudioVersion%"=="17.0" (
- set __VSVersion=vs2022
- set __PlatformToolset=v143
- goto :SetVCEnvironment
-)
+goto :SetVCEnvironment
:VSMissing
echo %__MsgPrefix%Error: Visual Studio 2022 with C++ tools required. ^
diff --git a/eng/pipelines/build.yml b/eng/pipelines/build.yml
index ffc2b959ee..ba061cc2f2 100644
--- a/eng/pipelines/build.yml
+++ b/eng/pipelines/build.yml
@@ -182,6 +182,7 @@ jobs:
- script: $(_buildScript)
-ci
+ -binaryLog
-configuration ${{ config.configuration }}
-architecture ${{ config.architecture }}
$(_TestArgs)
diff --git a/eng/pipelines/global-variables.yml b/eng/pipelines/global-variables.yml
index fd258a9883..83c51fa583 100644
--- a/eng/pipelines/global-variables.yml
+++ b/eng/pipelines/global-variables.yml
@@ -69,3 +69,4 @@ variables:
- ${{ if eq(parameters.runtimeFeedToken, 'dotnetclimsrc-sas-token-base64') }}:
- name: RuntimeFeedBase64SasToken
value: $(dotnetclimsrc-read-sas-token-base64)
+
diff --git a/eng/pipelines/pipeline-resources.yml b/eng/pipelines/pipeline-resources.yml
index 6c418d6693..6733c0dadb 100644
--- a/eng/pipelines/pipeline-resources.yml
+++ b/eng/pipelines/pipeline-resources.yml
@@ -53,17 +53,17 @@ extends:
ROOTFS_DIR: /crossrootfs/arm64
linux_s390x:
- image: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-s390x
+ image: mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-s390x
env:
ROOTFS_DIR: /crossrootfs/s390x
linux_ppc64le:
- image: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-ppc64le
+ image: mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-ppc64le
env:
ROOTFS_DIR: /crossrootfs/ppc64le
linux_riscv64:
- image: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-22.04-cross-riscv64
+ image: mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-riscv64
env:
ROOTFS_DIR: /crossrootfs/riscv64
@@ -85,10 +85,6 @@ extends:
test_opensuse_15_2:
image: mcr.microsoft.com/dotnet-buildtools/prereqs:opensuse-15.2-helix-amd64
- test_ubuntu_20_04:
- image: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-20.04
- options: '--env PYTHONPATH=/usr/bin/python3.8'
-
test_ubuntu_22_04:
image: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-22.04
options: '--env PYTHONPATH=/usr/bin/python3.10'
diff --git a/eng/testsoscdac.cmd b/eng/testsoscdac.cmd
new file mode 100644
index 0000000000..17f025b0c8
--- /dev/null
+++ b/eng/testsoscdac.cmd
@@ -0,0 +1,2 @@
+set SOS_TEST_CDAC=true
+%~dp0..\.dotnet\dotnet.exe test --no-build --logger "console;verbosity=detailed" %~dp0..\src\SOS\SOS.UnitTests\SOS.UnitTests.csproj --filter "Category=CDACCompatible"
diff --git a/eng/testsoscdac.sh b/eng/testsoscdac.sh
new file mode 100644
index 0000000000..4bad4dd7b4
--- /dev/null
+++ b/eng/testsoscdac.sh
@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+
+source="${BASH_SOURCE[0]}"
+
+# resolve $SOURCE until the file is no longer a symlink
+while [[ -h $source ]]; do
+ scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
+ source="$(readlink "$source")"
+
+ # if $source was a relative symlink, we need to resolve it relative to the path where the
+ # symlink file was located
+ [[ $source != /* ]] && source="$scriptroot/$source"
+done
+
+scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
+export LLDB_PATH=/usr/bin/lldb
+export SOS_TEST_CDAC=true
+$scriptroot/../.dotnet/dotnet test --no-build --logger "console;verbosity=detailed" $scriptroot/../src/SOS/SOS.UnitTests/SOS.UnitTests.csproj --filter "Category=CDACCompatible"
diff --git a/global.json b/global.json
index bdce7808f6..9c3adfeb03 100644
--- a/global.json
+++ b/global.json
@@ -1,6 +1,11 @@
{
+ "sdk": {
+ "version": "10.0.100-preview.6.25315.102",
+ "allowPrerelease": true,
+ "rollForward": "major"
+ },
"tools": {
- "dotnet": "10.0.100-preview.3.25125.5",
+ "dotnet": "10.0.100-preview.6.25315.102",
"runtimes": {
"dotnet": [
"$(MicrosoftNETCoreApp80Version)"
@@ -12,6 +17,6 @@
},
"msbuild-sdks": {
"Microsoft.Build.NoTargets": "3.5.0",
- "Microsoft.DotNet.Arcade.Sdk": "10.0.0-beta.25157.1"
+ "Microsoft.DotNet.Arcade.Sdk": "10.0.0-beta.25351.106"
}
}
diff --git a/src/Directory.Build.props b/src/Directory.Build.props
index ed7c3a0764..c2d80f41ed 100644
--- a/src/Directory.Build.props
+++ b/src/Directory.Build.props
@@ -1,6 +1,5 @@
- false
@@ -12,6 +11,5 @@
-
+
diff --git a/src/Microsoft.Diagnostics.DebugServices.Implementation/CommandService.cs b/src/Microsoft.Diagnostics.DebugServices.Implementation/CommandService.cs
index abf259de40..89b9a609bd 100644
--- a/src/Microsoft.Diagnostics.DebugServices.Implementation/CommandService.cs
+++ b/src/Microsoft.Diagnostics.DebugServices.Implementation/CommandService.cs
@@ -276,6 +276,7 @@ private sealed class CommandGroup
{
private Command _rootCommand;
private readonly Dictionary _commandHandlers = new();
+ private readonly ParseResult _emptyParseResult;
///
/// Create an instance of the command processor;
@@ -284,6 +285,10 @@ private sealed class CommandGroup
public CommandGroup(string commandPrompt = null)
{
_rootCommand = new Command(commandPrompt);
+
+ // The actual ParseResult.Empty() has a bug in it where it tries to get the executable name
+ // and nothing is returned under lldb on Linux causing an index out of range exception.
+ _emptyParseResult = _rootCommand.Parse(Array.Empty());
}
///
@@ -434,7 +439,7 @@ internal string GetDetailedHelp(Command command, IServiceProvider services, int
// Get the command help
HelpBuilder helpBuilder = new(maxWidth: windowWidth);
- HelpContext helpContext = new(helpBuilder, command, console);
+ HelpContext helpContext = new(helpBuilder, command, console, _emptyParseResult);
helpBuilder.Write(helpContext);
// Get the detailed help if any
@@ -608,7 +613,16 @@ internal string GetDetailedHelp(Command parser, IServiceProvider services)
// requesting help (either the help command or some other command using
// --help) won't work for the command instance that implements it's own
// help (SOS command).
- return (string)Invoke(_methodInfoHelp, context: null, parser, services);
+ string help = (string)Invoke(_methodInfoHelp, context: null, parser, services);
+
+ // Replace "{prompt}" with the host debugger's prompt
+ string prompt = services.GetService().HostType switch
+ {
+ HostType.Lldb => "(lldb) ",
+ HostType.DbgEng => "0:000> !",
+ _ => "> "
+ };
+ return help.Replace("{prompt}", prompt);
}
private object Invoke(MethodInfo methodInfo, ParseResult context, Command parser, IServiceProvider services)
diff --git a/src/Microsoft.Diagnostics.DebugServices.Implementation/Host.cs b/src/Microsoft.Diagnostics.DebugServices.Implementation/Host.cs
index f8935fb0d5..212714d6b5 100644
--- a/src/Microsoft.Diagnostics.DebugServices.Implementation/Host.cs
+++ b/src/Microsoft.Diagnostics.DebugServices.Implementation/Host.cs
@@ -6,11 +6,13 @@
using System.Collections.Immutable;
using System.Diagnostics;
using System.IO;
+using System.Runtime.InteropServices;
+using System.Text;
using Microsoft.Diagnostics.DebugServices;
namespace Microsoft.Diagnostics.DebugServices.Implementation
{
- public class Host : IHost
+ public class Host : IHost, ISettingsService
{
private readonly ServiceManager _serviceManager;
private ServiceContainer _serviceContainer;
@@ -49,6 +51,7 @@ public ServiceContainer CreateServiceContainer()
_serviceContainer = _serviceManager.CreateServiceContainer(ServiceScope.Global, parent: null);
_serviceContainer.AddService(_serviceManager);
_serviceContainer.AddService(this);
+ _serviceContainer.AddService(this);
return _serviceContainer;
}
@@ -91,11 +94,19 @@ public string GetTempDirectory()
{
if (_tempDirectory == null)
{
- // Use the SOS process's id if can't get the target's
+ string tempPath;
+ if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
+ {
+ tempPath = Path.GetTempPath();
+ }
+ else
+ {
+ tempPath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.UserProfile), ".dotnet");
+ }
+ // Make the temp directory per SOS session
uint processId = (uint)Process.GetCurrentProcess().Id;
-
// SOS depends on that the temp directory ends with "/".
- _tempDirectory = Path.Combine(Path.GetTempPath(), "sos" + processId.ToString()) + Path.DirectorySeparatorChar;
+ _tempDirectory = Path.Combine(tempPath, "sos" + processId.ToString()) + Path.DirectorySeparatorChar;
Directory.CreateDirectory(_tempDirectory);
}
return _tempDirectory;
@@ -103,6 +114,16 @@ public string GetTempDirectory()
#endregion
+ #region ISettingsService
+
+ public virtual bool DacSignatureVerificationEnabled { get; set; }
+
+ public bool UseContractReader { get; set; }
+
+ public bool ForceUseContractReader { get; set; }
+
+ #endregion
+
private void CleanupTempDirectory()
{
if (_tempDirectory != null)
diff --git a/src/Microsoft.Diagnostics.DebugServices.Implementation/ImageMappingMemoryService.cs b/src/Microsoft.Diagnostics.DebugServices.Implementation/ImageMappingMemoryService.cs
index 35fb9c9afc..36287a5160 100644
--- a/src/Microsoft.Diagnostics.DebugServices.Implementation/ImageMappingMemoryService.cs
+++ b/src/Microsoft.Diagnostics.DebugServices.Implementation/ImageMappingMemoryService.cs
@@ -5,8 +5,10 @@
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
+using System.Linq;
using System.Reflection.Metadata;
using System.Reflection.PortableExecutable;
+using Microsoft.Diagnostics.Runtime;
using Microsoft.FileFormats;
namespace Microsoft.Diagnostics.DebugServices.Implementation
@@ -37,7 +39,7 @@ public ImageMappingMemoryService(ServiceContainer container, IMemoryService memo
container.AddService(memoryService);
_memoryService = memoryService;
- _moduleService = managed ? new ManagedImageMappingModuleService(container) : container.GetService();
+ _moduleService = managed ? new ManagedModuleService(container) : container.GetService();
_memoryCache = new MemoryCache(ReadMemoryFromModule);
_recursionProtection = new HashSet();
@@ -325,5 +327,58 @@ private static void ApplyRelocations(IModule module, PEReader reader, int dataVA
}
}
}
+
+ ///
+ /// Module service implementation for managed image mapping. Enumerates all managed modules in all runtimes.
+ ///
+ private sealed class ManagedModuleService : ModuleService
+ {
+ private readonly IRuntimeService _runtimeService;
+
+ public ManagedModuleService(IServiceProvider services)
+ : base(services)
+ {
+ _runtimeService = services.GetService();
+ }
+
+ ///
+ /// Get/create the modules dictionary.
+ ///
+ protected override Dictionary GetModulesInner()
+ {
+ Dictionary modules = new();
+ int moduleIndex = 0;
+
+ IEnumerable runtimes = _runtimeService.EnumerateRuntimes();
+ if (runtimes.Any())
+ {
+ foreach (IRuntime runtime in runtimes)
+ {
+ ClrRuntime clrRuntime = runtime.Services.GetService();
+ if (clrRuntime is not null)
+ {
+ foreach (ClrModule clrModule in clrRuntime.EnumerateModules())
+ {
+ if (clrModule.ImageBase != 0)
+ {
+ IModule module = this.CreateModule(moduleIndex, clrModule);
+ try
+ {
+ modules.Add(module.ImageBase, module);
+ moduleIndex++;
+ }
+ catch (ArgumentException)
+ {
+ Trace.TraceError($"GetModulesInner(): duplicate module base '{module}' dup '{modules[module.ImageBase]}'");
+ }
+ }
+ }
+ }
+ }
+ }
+
+ return modules;
+ }
+ }
}
}
diff --git a/src/Microsoft.Diagnostics.DebugServices.Implementation/ManagedImageMappingModuleService.cs b/src/Microsoft.Diagnostics.DebugServices.Implementation/ManagedImageMappingModuleService.cs
deleted file mode 100644
index 0d72ae0a6b..0000000000
--- a/src/Microsoft.Diagnostics.DebugServices.Implementation/ManagedImageMappingModuleService.cs
+++ /dev/null
@@ -1,61 +0,0 @@
-// Licensed to the .NET Foundation under one or more agreements.
-// The .NET Foundation licenses this file to you under the MIT license.
-
-using System;
-using System.Collections.Generic;
-using System.Diagnostics;
-using System.Linq;
-using Microsoft.Diagnostics.Runtime;
-
-namespace Microsoft.Diagnostics.DebugServices.Implementation
-{
- ///
- /// Module service implementation for managed image mapping
- ///
- public class ManagedImageMappingModuleService : ModuleService
- {
- private readonly IRuntimeService _runtimeService;
-
- public ManagedImageMappingModuleService(IServiceProvider services)
- : base(services)
- {
- _runtimeService = services.GetService();
- }
-
- ///
- /// Get/create the modules dictionary.
- ///
- protected override Dictionary GetModulesInner()
- {
- Dictionary modules = new();
- int moduleIndex = 0;
-
- IEnumerable runtimes = _runtimeService.EnumerateRuntimes();
- if (runtimes.Any())
- {
- foreach (IRuntime runtime in runtimes)
- {
- ClrRuntime clrRuntime = runtime.Services.GetService();
- if (clrRuntime is not null)
- {
- foreach (ClrModule clrModule in clrRuntime.EnumerateModules())
- {
- ModuleFromAddress module = new(this, moduleIndex, clrModule.ImageBase, clrModule.Size, clrModule.Name);
- try
- {
- modules.Add(module.ImageBase, module);
- moduleIndex++;
- }
- catch (ArgumentException)
- {
- Trace.TraceError($"GetModulesInner(): duplicate module base '{module}' dup '{modules[module.ImageBase]}'");
- }
- }
- }
- }
- }
-
- return modules;
- }
- }
-}
diff --git a/src/Microsoft.Diagnostics.DebugServices.Implementation/MetadataMappingMemoryService.cs b/src/Microsoft.Diagnostics.DebugServices.Implementation/MetadataMappingMemoryService.cs
index 8db6375452..c0ba821dee 100644
--- a/src/Microsoft.Diagnostics.DebugServices.Implementation/MetadataMappingMemoryService.cs
+++ b/src/Microsoft.Diagnostics.DebugServices.Implementation/MetadataMappingMemoryService.cs
@@ -2,18 +2,22 @@
// The .NET Foundation licenses this file to you under the MIT license.
using System;
+using System.Collections.Generic;
using System.Collections.Immutable;
using System.Diagnostics;
using System.IO;
using System.Linq;
+using System.Reflection.PortableExecutable;
using Microsoft.Diagnostics.Runtime;
using Microsoft.FileFormats;
using Microsoft.FileFormats.PE;
+using Microsoft.SymbolStore;
+using Microsoft.SymbolStore.KeyGenerators;
namespace Microsoft.Diagnostics.DebugServices.Implementation
{
///
- /// Memory service wrapper that maps always module's metadata into the address
+ /// Memory service wrapper that always maps clrmodule's metadata into the address
/// space even is some or all of the memory exists in the coredump. lldb returns
/// zero's (instead of failing the memory read) for missing pages in core dumps
/// that older (less than 5.0) createdumps generate so it needs this special
@@ -22,6 +26,7 @@ namespace Microsoft.Diagnostics.DebugServices.Implementation
public class MetadataMappingMemoryService : IMemoryService, IDisposable
{
private readonly ServiceContainer _serviceContainer;
+ private readonly IModuleService _moduleService;
private readonly IMemoryService _memoryService;
private readonly IRuntimeService _runtimeService;
private readonly ISymbolService _symbolService;
@@ -40,6 +45,7 @@ public MetadataMappingMemoryService(ServiceContainer container, IMemoryService m
container.AddService(memoryService);
_memoryService = memoryService;
+ _moduleService = container.GetService();
_runtimeService = container.GetService();
_symbolService = container.GetService();
@@ -124,7 +130,7 @@ private MetadataRegion FindRegion(ulong address)
// Need to set this before enumerating the runtimes to prevent reentrancy
_regionInitialized = true;
- System.Collections.Generic.IEnumerable runtimes = _runtimeService.EnumerateRuntimes();
+ IEnumerable runtimes = _runtimeService.EnumerateRuntimes();
if (runtimes.Any())
{
foreach (IRuntime runtime in runtimes)
@@ -176,32 +182,6 @@ private MetadataRegion FindRegion(ulong address)
return null;
}
- private ImmutableArray GetMetaDataFromAssembly(ClrModule module)
- {
- Debug.Assert(module.ImageBase != 0);
-
- ImmutableArray metadata = ImmutableArray.Empty;
- bool isVirtual = module.Layout != ModuleLayout.Flat;
- try
- {
- Stream stream = _memoryService.CreateMemoryStream(module.ImageBase, module.Size > 0 ? module.Size : 4096);
- PEFile peFile = new(new StreamAddressSpace(stream), isVirtual);
- if (peFile.IsValid())
- {
- metadata = _symbolService.GetMetadata(module.Name, peFile.Timestamp, peFile.SizeOfImage);
- }
- else
- {
- Trace.TraceError($"GetMetaData: {module.ImageBase:X16} not valid PE");
- }
- }
- catch (Exception ex) when (ex is InvalidVirtualAddressException or BadInputFormatException)
- {
- Trace.TraceError($"GetMetaData: loaded {module.ImageBase:X16} exception {ex.Message}");
- }
- return metadata;
- }
-
private sealed class MetadataRegion : IComparable
{
private readonly MetadataMappingMemoryService _memoryService;
@@ -242,7 +222,7 @@ private ImmutableArray GetMetaData()
{
if (_metadata.IsDefault)
{
- _metadata = _memoryService.GetMetaDataFromAssembly(_module);
+ _metadata = _memoryService._moduleService.CreateModule(-1, _module).GetMetadata();
}
return _metadata;
}
diff --git a/src/Microsoft.Diagnostics.DebugServices.Implementation/Module.cs b/src/Microsoft.Diagnostics.DebugServices.Implementation/Module.cs
index 0a1d2309e7..7eecd08205 100644
--- a/src/Microsoft.Diagnostics.DebugServices.Implementation/Module.cs
+++ b/src/Microsoft.Diagnostics.DebugServices.Implementation/Module.cs
@@ -110,7 +110,7 @@ public bool? IsFileLayout
{
return false;
}
- // Native Windows dlls default to file layout
+ // Native Windows dlls default to loaded layout
if ((_flags & Flags.IsManaged) == 0 && Target.OperatingSystem == OSPlatform.Windows)
{
return false;
@@ -193,7 +193,7 @@ public ImmutableArray GetMetadata()
{
try
{
- PEReader reader = Services.GetService();
+ PEReader reader = Services.GetService()?.GetPEReader();
if (reader is not null && reader.HasMetadata)
{
PEMemoryBlock metadataInfo = reader.GetMetadata();
@@ -201,13 +201,13 @@ public ImmutableArray GetMetadata()
}
}
catch (Exception ex) when
- (ex is InvalidOperationException ||
- ex is BadImageFormatException ||
- ex is IOException)
+ (ex is InvalidOperationException or
+ BadImageFormatException or
+ IOException)
{
Trace.TraceError($"GetMetaData: {ex.Message}");
}
- return ImmutableArray.Empty;
+ return [];
}
#endregion
diff --git a/src/Microsoft.Diagnostics.DebugServices.Implementation/ModuleService.cs b/src/Microsoft.Diagnostics.DebugServices.Implementation/ModuleService.cs
index 413e582c88..75f9a087c3 100644
--- a/src/Microsoft.Diagnostics.DebugServices.Implementation/ModuleService.cs
+++ b/src/Microsoft.Diagnostics.DebugServices.Implementation/ModuleService.cs
@@ -175,13 +175,22 @@ IEnumerable IModuleService.GetModuleFromModuleName(string moduleName)
///
/// Create a module instance
///
- /// artificial index
+ /// artificial index or -1 for none
/// module base address
/// module size
/// module name
/// IModule
- IModule IModuleService.CreateModule(int moduleIndex, ulong imageBase, ulong imageSize, string imageName)
+ /// thrown if imageBase or imageSize is 0
+ public IModule CreateModule(int moduleIndex, ulong imageBase, ulong imageSize, string imageName)
{
+ if (imageBase == 0)
+ {
+ throw new ArgumentNullException(nameof(imageBase));
+ }
+ if (imageSize == 0)
+ {
+ throw new ArgumentNullException(nameof(imageSize));
+ }
return new ModuleFromAddress(this, moduleIndex, imageBase, imageSize, imageName);
}
diff --git a/src/Microsoft.Diagnostics.DebugServices.Implementation/Runtime.cs b/src/Microsoft.Diagnostics.DebugServices.Implementation/Runtime.cs
index 26513a095c..f3eeb4aa95 100644
--- a/src/Microsoft.Diagnostics.DebugServices.Implementation/Runtime.cs
+++ b/src/Microsoft.Diagnostics.DebugServices.Implementation/Runtime.cs
@@ -20,10 +20,13 @@ namespace Microsoft.Diagnostics.DebugServices.Implementation
public class Runtime : IRuntime, IDisposable
{
private readonly ClrInfo _clrInfo;
+ private readonly ISettingsService _settingsService;
private readonly ISymbolService _symbolService;
private Version _runtimeVersion;
private ClrRuntime _clrRuntime;
private string _dacFilePath;
+ private bool _verifySignature; // This only applies to the regular DAC, not the CDAC
+ private string _cdacFilePath;
private string _dbiFilePath;
protected readonly ServiceContainer _serviceContainer;
@@ -33,7 +36,8 @@ public Runtime(IServiceProvider services, int id, ClrInfo clrInfo)
Target = services.GetService() ?? throw new DiagnosticsException("Dump or live session target required");
Id = id;
_clrInfo = clrInfo ?? throw new ArgumentNullException(nameof(clrInfo));
- _symbolService = services.GetService();
+ _settingsService = services.GetService() ?? throw new ArgumentException("ISettingsService required");
+ _symbolService = services.GetService() ?? throw new ArgumentException("ISymbolService required");
RuntimeType = RuntimeType.Unknown;
if (clrInfo.Flavor == ClrFlavor.Core)
@@ -96,9 +100,36 @@ public Version RuntimeVersion
}
}
- public string GetDacFilePath()
+ public string GetCDacFilePath()
{
- _dacFilePath ??= GetLibraryPath(DebugLibraryKind.Dac);
+ if (_cdacFilePath is null)
+ {
+ if (_settingsService.UseContractReader || _settingsService.ForceUseContractReader)
+ {
+ _cdacFilePath = GetLibraryPath(DebugLibraryKind.CDac);
+ }
+ }
+ return _cdacFilePath;
+ }
+
+ public string GetDacFilePath(out bool verifySignature)
+ {
+ if (_settingsService.ForceUseContractReader)
+ {
+ // Don't verify signature when using the CDAC and don't change the cached value
+ // because it only applies to the regular DAC in _dacFilePath.
+ verifySignature = false;
+ return GetCDacFilePath();
+ }
+ if (_dacFilePath is null)
+ {
+ _dacFilePath = GetLibraryPath(DebugLibraryKind.Dac);
+ if (_dacFilePath is not null)
+ {
+ _verifySignature = _settingsService.DacSignatureVerificationEnabled;
+ }
+ }
+ verifySignature = _verifySignature;
return _dacFilePath;
}
@@ -115,7 +146,7 @@ public string GetDbiFilePath()
///
private ClrRuntime CreateRuntime()
{
- string dacFilePath = GetDacFilePath();
+ string dacFilePath = GetDacFilePath(out bool verifySignature);
if (dacFilePath is not null)
{
Trace.TraceInformation($"Creating ClrRuntime #{Id} {dacFilePath}");
@@ -123,7 +154,7 @@ private ClrRuntime CreateRuntime()
{
// Ignore the DAC version mismatch that can happen because the clrmd ELF dump reader
// returns 0.0.0.0 for the runtime module that the DAC is matched against.
- return _clrRuntime = _clrInfo.CreateRuntime(dacFilePath, ignoreMismatch: true);
+ return _clrRuntime = _clrInfo.CreateRuntime(dacFilePath, ignoreMismatch: true, verifySignature);
}
catch (Exception ex) when
(ex is DllNotFoundException or
@@ -151,15 +182,18 @@ private string GetLibraryPath(DebugLibraryKind kind)
{
if (libraryInfo.Kind == kind && RuntimeInformation.IsOSPlatform(libraryInfo.Platform) && libraryInfo.TargetArchitecture == currentArch)
{
- libraryPath = GetLocalPath(libraryInfo.FileName);
+ libraryPath = GetLocalPath(libraryInfo);
if (libraryPath is not null)
{
break;
}
- libraryPath = DownloadFile(libraryInfo);
- if (libraryPath is not null)
+ if (libraryInfo.ArchivedUnder != SymbolProperties.None)
{
- break;
+ libraryPath = DownloadFile(libraryInfo);
+ if (libraryPath is not null)
+ {
+ break;
+ }
}
}
}
@@ -167,16 +201,23 @@ private string GetLibraryPath(DebugLibraryKind kind)
return libraryPath;
}
- private string GetLocalPath(string fileName)
+ private string GetLocalPath(DebugLibraryInfo libraryInfo)
{
string localFilePath;
- if (!string.IsNullOrEmpty(RuntimeModuleDirectory))
+ if (libraryInfo.Kind == DebugLibraryKind.CDac)
{
- localFilePath = Path.Combine(RuntimeModuleDirectory, Path.GetFileName(fileName));
+ localFilePath = libraryInfo.FileName;
}
else
{
- localFilePath = Path.Combine(Path.GetDirectoryName(RuntimeModule.FileName), Path.GetFileName(fileName));
+ if (!string.IsNullOrEmpty(RuntimeModuleDirectory))
+ {
+ localFilePath = Path.Combine(RuntimeModuleDirectory, Path.GetFileName(libraryInfo.FileName));
+ }
+ else
+ {
+ localFilePath = Path.Combine(Path.GetDirectoryName(RuntimeModule.FileName), Path.GetFileName(libraryInfo.FileName));
+ }
}
if (!File.Exists(localFilePath))
{
@@ -302,7 +343,13 @@ public override string ToString()
if (_dacFilePath is not null)
{
sb.AppendLine();
- sb.Append($" DAC: {_dacFilePath}");
+ string verify = _verifySignature ? "(verify)" : "(don't verify)";
+ sb.Append($" DAC: {_dacFilePath} {verify}");
+ }
+ if (_cdacFilePath is not null)
+ {
+ sb.AppendLine();
+ sb.Append($" CDAC: {_cdacFilePath}");
}
if (_dbiFilePath is not null)
{
diff --git a/src/Microsoft.Diagnostics.DebugServices.Implementation/RuntimeProvider.cs b/src/Microsoft.Diagnostics.DebugServices.Implementation/RuntimeProvider.cs
index 44ddf04715..4639a0ebcf 100644
--- a/src/Microsoft.Diagnostics.DebugServices.Implementation/RuntimeProvider.cs
+++ b/src/Microsoft.Diagnostics.DebugServices.Implementation/RuntimeProvider.cs
@@ -29,14 +29,12 @@ public RuntimeProvider(IServiceProvider services)
/// Enumeration control flags
public IEnumerable EnumerateRuntimes(int startingRuntimeId, RuntimeEnumerationFlags flags)
{
- ISettingsService settingsService = _services.GetService();
// The ClrInfo and DataTarget instances are disposed when Runtime instance is disposed. Runtime instances are
// not flushed when the Target/RuntimeService is flushed; they are all disposed and the list cleared. They are
// all re-created the next time the IRuntime or ClrRuntime instance is queried.
DataTarget dataTarget = new(new CustomDataTarget(_services.GetService())
{
ForceCompleteRuntimeEnumeration = (flags & RuntimeEnumerationFlags.All) != 0,
- DacSignatureVerificationEnabled = settingsService is null || settingsService.DacSignatureVerificationEnabled,
FileLocator = null
});
for (int i = 0; i < dataTarget.ClrVersions.Length; i++)
diff --git a/src/Microsoft.Diagnostics.DebugServices.Implementation/SymbolService.cs b/src/Microsoft.Diagnostics.DebugServices.Implementation/SymbolService.cs
index 3958b47176..404e81e01f 100644
--- a/src/Microsoft.Diagnostics.DebugServices.Implementation/SymbolService.cs
+++ b/src/Microsoft.Diagnostics.DebugServices.Implementation/SymbolService.cs
@@ -460,48 +460,6 @@ public string DownloadSymbolFile(IModule module)
/// the full path name of the file
public string DownloadFile(string index, string file) => DownloadFile(new SymbolStoreKey(index, file));
- ///
- /// Returns the metadata for the assembly
- ///
- /// file name and path to module
- /// module timestamp
- /// size of PE image
- /// metadata
- public ImmutableArray GetMetadata(string imagePath, uint imageTimestamp, uint imageSize)
- {
- try
- {
- Stream peStream = null;
- if (imagePath != null && File.Exists(imagePath))
- {
- peStream = Utilities.TryOpenFile(imagePath);
- }
- else if (IsSymbolStoreEnabled)
- {
- SymbolStoreKey key = PEFileKeyGenerator.GetKey(imagePath, imageTimestamp, imageSize);
- peStream = GetSymbolStoreFile(key)?.Stream;
- }
- if (peStream != null)
- {
- using PEReader peReader = new(peStream, PEStreamOptions.Default);
- if (peReader.HasMetadata)
- {
- PEMemoryBlock metadataInfo = peReader.GetMetadata();
- return metadataInfo.GetContent();
- }
- }
- }
- catch (Exception ex) when
- (ex is UnauthorizedAccessException or
- BadImageFormatException or
- InvalidVirtualAddressException or
- IOException)
- {
- Trace.TraceError($"GetMetaData: {ex.Message}");
- }
- return ImmutableArray.Empty;
- }
-
///
/// Returns the portable PDB reader for the assembly path
///
@@ -887,7 +845,7 @@ private string DownloadFile(SymbolStoreKey key)
// If the downloaded doesn't already exists on disk in the cache, then write it to a temporary location.
if (!File.Exists(downloadFilePath))
{
- downloadFilePath = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName() + "-" + Path.GetFileName(key.FullPathName));
+ downloadFilePath = Path.Combine(_host.GetTempDirectory(), Path.GetRandomFileName() + "-" + Path.GetFileName(key.FullPathName));
using (Stream destinationStream = File.OpenWrite(downloadFilePath))
{
file.Stream.CopyTo(destinationStream);
diff --git a/src/Microsoft.Diagnostics.DebugServices.Implementation/Utilities.cs b/src/Microsoft.Diagnostics.DebugServices.Implementation/Utilities.cs
index a966eeecc9..36634a60f0 100644
--- a/src/Microsoft.Diagnostics.DebugServices.Implementation/Utilities.cs
+++ b/src/Microsoft.Diagnostics.DebugServices.Implementation/Utilities.cs
@@ -9,6 +9,7 @@
using System.Reflection;
using System.Reflection.PortableExecutable;
using System.Runtime.InteropServices;
+using Microsoft.Diagnostics.Runtime;
using Microsoft.FileFormats;
using Microsoft.FileFormats.ELF;
using Microsoft.FileFormats.MachO;
@@ -51,6 +52,19 @@ public static int GetPointerSizeFromArchitecture(Architecture architecture)
}
}
+ ///
+ /// Create an IModule from a managed ClrModule
+ ///
+ /// module service
+ /// module index or -1 if none
+ /// ClrModule instance
+ ///
+ public static IModule CreateModule(this IModuleService moduleService, int moduleIndex, ClrModule module)
+ {
+ ulong size = module.Size > 0 ? module.Size : 4096;
+ return moduleService.CreateModule(moduleIndex, module.ImageBase, size, module.Name);
+ }
+
///
/// Combines two hash codes into a single hash code, in an order-dependent manner.
///
diff --git a/src/Microsoft.Diagnostics.DebugServices/IModuleService.cs b/src/Microsoft.Diagnostics.DebugServices/IModuleService.cs
index 8eb1981248..7d65d8e796 100644
--- a/src/Microsoft.Diagnostics.DebugServices/IModuleService.cs
+++ b/src/Microsoft.Diagnostics.DebugServices/IModuleService.cs
@@ -1,6 +1,7 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
+using System;
using System.Collections.Generic;
namespace Microsoft.Diagnostics.DebugServices
@@ -47,13 +48,14 @@ public interface IModuleService
IEnumerable GetModuleFromModuleName(string moduleName);
///
- /// Create a module instance from a stream (memory or file).
+ /// Create a module instance
///
- /// artifical index
+ /// artificial index or -1 for none
/// module base address
/// module size
/// module name
/// IModule
+ /// thrown if imageBase or imageSize is 0
IModule CreateModule(int moduleIndex, ulong imageBase, ulong imageSize, string imageName);
}
}
diff --git a/src/Microsoft.Diagnostics.DebugServices/IRuntime.cs b/src/Microsoft.Diagnostics.DebugServices/IRuntime.cs
index 5dd9e4ce93..6bf7f05ef6 100644
--- a/src/Microsoft.Diagnostics.DebugServices/IRuntime.cs
+++ b/src/Microsoft.Diagnostics.DebugServices/IRuntime.cs
@@ -62,7 +62,13 @@ public interface IRuntime
///
/// Returns the DAC file path
///
- string GetDacFilePath();
+ /// returns if the DAC signature should be verified
+ string GetDacFilePath(out bool verifySignature);
+
+ ///
+ /// Returns the CDac file path if enabled by global settings
+ ///
+ string GetCDacFilePath();
///
/// Returns the DBI file path
diff --git a/src/Microsoft.Diagnostics.DebugServices/ISettingsService.cs b/src/Microsoft.Diagnostics.DebugServices/ISettingsService.cs
index ae3ce83160..60ad98c338 100644
--- a/src/Microsoft.Diagnostics.DebugServices/ISettingsService.cs
+++ b/src/Microsoft.Diagnostics.DebugServices/ISettingsService.cs
@@ -15,5 +15,15 @@ public interface ISettingsService
/// If true, enforces the proper DAC certificate signing when loaded
///
bool DacSignatureVerificationEnabled { get; set; }
+
+ ///
+ /// If true, uses the CDAC contract reader if available.
+ ///
+ bool UseContractReader { get; set; }
+
+ ///
+ /// If true, always use the CDAC contract reader even when not requested
+ ///
+ bool ForceUseContractReader { get; set; }
}
}
diff --git a/src/Microsoft.Diagnostics.DebugServices/ISymbolService.cs b/src/Microsoft.Diagnostics.DebugServices/ISymbolService.cs
index fb3c23bdc4..5292a5a35e 100644
--- a/src/Microsoft.Diagnostics.DebugServices/ISymbolService.cs
+++ b/src/Microsoft.Diagnostics.DebugServices/ISymbolService.cs
@@ -129,15 +129,6 @@ public bool AddSymbolServer(
/// the full path name of the file
string DownloadFile(string index, string file);
- ///
- /// Returns the metadata for the assembly
- ///
- /// file name and path to module
- /// module timestamp
- /// size of PE image
- /// metadata
- ImmutableArray GetMetadata(string imagePath, uint imageTimestamp, uint imageSize);
-
///
/// Returns the portable PDB reader for the assembly path
///
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/AnalyzeOOMCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/AnalyzeOOMCommand.cs
index ad970e05bb..85257bedda 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/AnalyzeOOMCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/AnalyzeOOMCommand.cs
@@ -56,5 +56,61 @@ public override void Invoke()
Console.WriteLine("There was no managed OOM due to allocations on the GC heap");
}
}
+
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"AnalyzeOOM displays the info of the last OOM occurred on an allocation request to
+the GC heap (in Server GC it displays OOM, if any, on each GC heap).
+
+To see the managed exception(s) use the 'clrthreads' command which will show you
+managed exception(s), if any, on each managed thread. If you do see an
+OutOfMemoryException exception you can use the 'printexception' command on it.
+To get the full call stack use the ""kb"" command in the debugger for that thread.
+For example, to display thread 3's stack use ~3kb.
+
+OOM exceptions could be because of the following reasons:
+
+1) allocation request to GC heap
+ in which case you will see JIT_New* on the call stack because managed code called new.
+2) other runtime allocation failure
+ for example, failure to expand the finalize queue when GC.ReRegisterForFinalize is
+ called.
+3) some other code you use throws a managed OOM exception
+ for example, some .NET framework code converts a native OOM exception to managed
+ and throws it.
+
+The 'analyzeoom' command aims to help you with investigating 1) which is the most
+difficult because it requires some internal info from GC. The only exception is
+we don't support allocating objects larger than 2GB on CLR v2.0 or prior. And this
+command will not display any managed OOM because we will throw OOM right away
+instead of even trying to allocate it on the GC heap.
+
+There are 2 legitimate scenarios where GC would return OOM to allocation requests -
+one is if the process is running out of VM space to reserve a segment; the other
+is if the system is running out physical memory (+ page file if you have one) so
+GC can not commit memory it needs. You can look at these scenarios by using performance
+counters or debugger commands. For example for the former scenario the ""!address
+-summary"" debugger command will show you the largest free region in the VM. For
+the latter scenario you can look at the ""Memory% Committed Bytes In Use"" see
+if you are running low on commit space. One important thing to keep in mind is
+when you do this kind of memory analysis it could an aftereffect and doesn't
+completely agree with what this command tells you, in which case the command should
+be respected because it truly reflects what happened during GC.
+
+The other cases should be fairly obvious from the call stack.
+
+Sample output:
+
+ {prompt}analyzeoom
+ ---------Heap 2 ---------
+ Managed OOM occurred after GC #28 (Requested to allocate 1234 bytes)
+ Reason: Didn't have enough memory to commit
+ Detail: SOH: Didn't have enough memory to grow the internal GC data structures (800000 bytes) -
+ on GC entry available commit space was 500 MB
+ ---------Heap 4 ---------
+ Managed OOM occurred after GC #12 (Requested to allocate 100000 bytes)
+ Reason: Didn't have enough memory to allocate an LOH segment
+ Detail: LOH: Failed to reserve memory (16777216 bytes)
+";
}
}
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/ClrMDHelper.cs b/src/Microsoft.Diagnostics.ExtensionCommands/ClrMDHelper.cs
index c676f5d2ae..2fa6511d2c 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/ClrMDHelper.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/ClrMDHelper.cs
@@ -150,7 +150,7 @@ public IEnumerable EnumerateTimers()
continue;
}
- if (string.CompareOrdinal(objType.Name, "System.Threading.TimerQueue") != 0)
+ if (objType.Name != "System.Threading.TimerQueue")
{
continue;
}
@@ -386,7 +386,7 @@ private IEnumerable EnumerateGlobalThreadPoolItemsInNetCore()
continue;
}
- if (string.CompareOrdinal(objType.Name, "System.Threading.ThreadPoolWorkQueue") == 0)
+ if (objType.Name == "System.Threading.ThreadPoolWorkQueue")
{
// work items are stored in a ConcurrentQueue stored in the "workItems" field
ClrInstanceField workItemsField = objType.GetFieldByName("workItems");
@@ -413,11 +413,9 @@ private ThreadPoolItem GetThreadPoolItem(ClrObject item)
return GetTask(item);
}
- if (
- (string.CompareOrdinal(itemType.Name, "System.Threading.QueueUserWorkItemCallback") == 0) ||
- // new to .NET Core
- (string.CompareOrdinal(itemType.Name, "System.Threading.QueueUserWorkItemCallbackDefaultContext") == 0)
- )
+ if (itemType.Name is
+ "System.Threading.QueueUserWorkItemCallback" or
+ "System.Threading.QueueUserWorkItemCallbackDefaultContext") //new to .net core
{
return GetQueueUserWorkItemCallback(item);
}
@@ -463,7 +461,7 @@ private ThreadPoolItem GetTask(ClrObject task)
if (taskScheduler.IsValid)
{
string schedulerType = taskScheduler.Type.ToString();
- if (string.CompareOrdinal("System.Threading.Tasks.ThreadPoolTaskScheduler", schedulerType) != 0)
+ if ("System.Threading.Tasks.ThreadPoolTaskScheduler" != schedulerType)
{
tpi.MethodName = $"{tpi.MethodName} [{schedulerType}]";
}
@@ -542,10 +540,8 @@ internal string BuildDelegateMethodName(ClrType targetType, ClrObject action)
// method is implemented by an class inherited from targetType
// ... or a simple delegate indirection to a static/instance method
{
- if (
- (string.CompareOrdinal(targetType.Name, "System.Threading.WaitCallback") == 0) ||
- targetType.Name.StartsWith("System.Action<", StringComparison.Ordinal)
- )
+ if (targetType.Name == "System.Threading.WaitCallback"
+ || targetType.Name.StartsWith("System.Action<", StringComparison.Ordinal))
{
return $"{method.Type.Name}.{method.Name}";
}
@@ -611,7 +607,7 @@ private IEnumerable EnumerateLocalThreadPoolItemsInNetCore()
continue;
}
- if (string.CompareOrdinal(type.Name, "System.Threading.ThreadPoolWorkQueue+WorkStealingQueue") == 0)
+ if (type.Name == "System.Threading.ThreadPoolWorkQueue+WorkStealingQueue")
{
ClrObject stealingQueue = obj;
ClrArray workItems = stealingQueue.ReadObjectField("m_array").AsArray();
@@ -667,7 +663,7 @@ private IEnumerable EnumerateGlobalThreadPoolItemsInNetFramework
continue;
}
- if (string.CompareOrdinal(workQueueType.Name, "System.Threading.ThreadPoolWorkQueue") != 0)
+ if (workQueueType.Name != "System.Threading.ThreadPoolWorkQueue")
{
continue;
}
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/DumpConcurrentDictionaryCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/DumpConcurrentDictionaryCommand.cs
index 06bee8bc6e..60a6569ca7 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/DumpConcurrentDictionaryCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/DumpConcurrentDictionaryCommand.cs
@@ -69,7 +69,7 @@ public static string GetDetailedHelp() =>
DumpConcurrentDictionary
Lists all items (key/value pairs) in the given concurrent dictionary.
-> dcd 00000184aa23e2e0
+{prompt}dcd 00000184aa23e2e0
System.Collections.Concurrent.ConcurrentDictionary
-----
Key: 31
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/DumpConcurrentQueueCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/DumpConcurrentQueueCommand.cs
index 9252a6379e..96cc0322a9 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/DumpConcurrentQueueCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/DumpConcurrentQueueCommand.cs
@@ -67,14 +67,14 @@ public static string GetDetailedHelp() =>
Lists all items in the given concurrent queue.
For simple types such as numbers, boolean and string, values are shown.
-> dcq 00000202a79320e8
+{prompt}dcq 00000202a79320e8
System.Collections.Concurrent.ConcurrentQueue
1 - 0
2 - 1
3 - 2
In case of reference types, the command to dump each object is shown.
-> dcq 00000202a79337f8
+{prompt}dcq 00000202a79337f8
System.Collections.Concurrent.ConcurrentQueue
1 - dumpobj 0x202a7934e38
2 - dumpobj 0x202a7934fd0
@@ -82,7 +82,7 @@ Lists all items in the given concurrent queue.
For value types, the command to dump each array segment is shown.
The next step is to manually dump each element with dumpvc <[item] address>.
-> dcq 00000202a7933370
+{prompt}dcq 00000202a7933370
System.Collections.Concurrent.ConcurrentQueue
1 - dumparray 202a79334e0
2 - dumparray 202a7938a88
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/DumpGenCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/DumpGenCommand.cs
index a3ec51e17b..2e02a53a62 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/DumpGenCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/DumpGenCommand.cs
@@ -132,7 +132,7 @@ Generation number can take the following values (case insensitive):
- poh
- foh
-> dumpgen gen0
+{prompt}dumpgen gen0
Statistics:
MT Count TotalSize Class Name
00007ff9ea6601c8 1 24 System.Collections.Generic.GenericEqualityComparer
@@ -143,7 +143,7 @@ 00007ff9ea651e18 204 41154 System.String
Total 651 objects
As the original dumpheap command, we can pass an additional '-type' parameter to filter out on type name
-> dumpgen gen2 -type Object
+{prompt}dumpgen gen2 -type Object
Statistics:
MT Count TotalSize Class Name
00007ff9ea590af0 26 624 System.Object
@@ -151,7 +151,7 @@ 00007ff9ea590af0 26 624 System.Object
00007ff9ea596618 17 2080 System.Object[]
Total 46 objects
-> dumpgen gen0 -mt 00007ff9ea6e75b8
+{prompt}dumpgen gen0 -mt 00007ff9ea6e75b8
Address MT Size
00000184aa23e8a0 00007ff9ea6e75b8 40
00000184aa23e8f0 00007ff9ea6e75b8 40
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/DumpHeapCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/DumpHeapCommand.cs
index d23e68e5b0..88f8b99481 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/DumpHeapCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/DumpHeapCommand.cs
@@ -166,6 +166,136 @@ public override void Invoke()
}
}
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"Dumpheap is a powerful command that traverses the garbage collected heap,
+collection statistics about objects. With it's various options, it can look for
+particular types, restrict to a range, or look for ThinLocks (see !SyncBlk
+documentation). Finally, it will provide a warning if it detects excessive
+fragmentation in the GC heap.
+
+When called without options, the output is first a list of objects in the heap,
+followed by a report listing all the types found, their size and number:
+
+ {prompt}dumpheap
+ Address MT Size
+ 00a71000 0015cde8 12 Free
+ 00a7100c 0015cde8 12 Free
+ 00a71018 0015cde8 12 Free
+ 00a71024 5ba58328 68
+ 00a71068 5ba58380 68
+ 00a710ac 5ba58430 68
+ 00a710f0 5ba5dba4 68
+ ...
+ total 619 objects
+ Statistics:
+ MT Count TotalSize Class Name
+ 5ba7607c 1 12 System.Security.Permissions.HostProtectionResource
+ 5ba75d54 1 12 System.Security.Permissions.SecurityPermissionFlag
+ 5ba61f18 1 12 System.Collections.CaseInsensitiveComparer
+ ...
+ 0015cde8 6 10260 Free
+ 5ba57bf8 318 18136 System.String
+ ...
+
+""Free"" objects are simply regions of space the garbage collector can use later.
+If 30% or more of the heap contains ""Free"" objects, the process may suffer from
+heap fragmentation. This is usually caused by pinning objects for a long time
+combined with a high rate of allocation. Here is example output where 'dumpheap'
+provides a warning about fragmentation:
+
+
+ Fragmented blocks larger than 1MB:
+ Addr Size Followed by
+ 00a780c0 1.5MB 00bec800 System.Byte[]
+ 00da4e38 1.2MB 00ed2c00 System.Byte[]
+ 00f16df0 1.2MB 01044338 System.Byte[]
+
+The arguments in detail:
+
+-stat Restrict the output to the statistical type summary
+-strings Restrict the output to a statistical string value summary
+-short Limits output to just the address of each object. This allows you
+ to easily pipe output from the command to another debugger
+ command for automation.
+-min Ignore objects less than the size given in bytes (hex)
+-max Ignore objects larger than the size given in bytes (hex)
+-live Only print live objects
+-dead Only print dead objects (objects which will be collected in the
+ next full GC)
+-thinlock Report on any ThinLocks (an efficient locking scheme, see 'syncblk'
+ documentation for more info)
+-startAtLowerBound
+ Force heap walk to begin at lower bound of a supplied address range.
+ (During plan phase, the heap is often not walkable because objects
+ are being moved. In this case, 'dumpheap' may report spurious errors,
+ in particular bad objects. It may be possible to traverse more of
+ the heap after the reported bad object. Even if you specify an
+ address range, 'dumpheap' will start its walk from the beginning of
+ the heap by default. If it finds a bad object before the specified
+ range, it will stop before displaying the part of the heap in which
+ you are interested. This switch will force 'dumpheap' to begin its
+ walk at the specified lower bound. You must supply the address of a
+ good object as the lower bound for this to work. Display memory at
+ the address of the bad object to manually find the next method
+ table (use 'dumpmt' to verify). If the GC is currently in a call to
+ memcopy, You may also be able to find the next object's address by
+ adding the size to the start address given as parameters.)
+-mt List only those objects with the MethodTable given
+-type List only those objects whose type name is a substring match of the
+ string provided.
+start Begin listing from this address
+end Stop listing at this address
+
+A special note about -type: Often, you'd like to find not only Strings, but
+System.Object arrays that are constrained to contain Strings. (""new
+String[100]"" actually creates a System.Object array, but it can only hold
+System.String object pointers). You can use -type in a special way to find
+these arrays. Just pass ""-type System.String[]"" and those Object arrays will
+be returned. More generally, ""-type []"".
+
+The start/end parameters can be obtained from the output of 'eeheap -gc'. For
+example, if you only want to list objects in the large heap segment:
+
+ {prompt}eeheap -gc
+ Number of GC Heaps: 1
+ generation 0 starts at 0x00c32754
+ generation 1 starts at 0x00c32748
+ generation 2 starts at 0x00a71000
+ segment begin allocated size
+ 00a70000 00a71000 010443a8 005d33a8(6108072)
+ Large object heap starts at 0x01a71000
+ segment begin allocated size
+ 01a70000 01a71000 01a75000 0x00004000(16384)
+ Total Size 0x5d73a8(6124456)
+ ------------------------------
+ GC Heap Size 0x5d73a8(6124456)
+
+ {prompt}dumpheap 1a71000 1a75000
+ Address MT Size
+ 01a71000 5ba88bd8 2064
+ 01a71810 0019fe48 2032 Free
+ 01a72000 5ba88bd8 4096
+ 01a73000 0019fe48 4096 Free
+ 01a74000 5ba88bd8 4096
+ total 5 objects
+ Statistics:
+ MT Count TotalSize Class Name
+ 0019fe48 2 6128 Free
+ 5ba88bd8 3 10256 System.Object[]
+ Total 5 objects
+
+Finally, if GC heap corruption is present, you may see an error like this:
+
+ {prompt}dumpheap -stat
+ object 00a73d24: does not have valid MT
+ curr_object : 00a73d24
+ Last good object: 00a73d14
+ ----------------
+
+That indicates a serious problem. See the help for 'verifyheap' for more
+information on diagnosing the cause.
+";
private void ParseArguments()
{
if (!Runtime.Heap.CanWalkHeap && !IgnoreGCState)
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/DumpRuntimeTypeCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/DumpRuntimeTypeCommand.cs
index 8c2de445c6..81b64f94e5 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/DumpRuntimeTypeCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/DumpRuntimeTypeCommand.cs
@@ -8,7 +8,7 @@
namespace Microsoft.Diagnostics.ExtensionCommands
{
- [Command(Name = "dumpruntimetypes", Aliases = new[] { "DumpRuntimeTypes" }, Help = "Finds all System.RuntimeType objects in the GC heap and prints the type name and MethodTable they refer too.")]
+ [Command(Name = "dumpruntimetypes", Aliases = new[] { "DumpRuntimeTypes" }, Help = "Finds all System.RuntimeType objects in the GC heap and prints the type name and MethodTable they refer to.")]
public sealed class DumpRuntimeTypeCommand : ClrRuntimeCommandBase
{
public override void Invoke()
@@ -63,5 +63,40 @@ public override void Invoke()
Console.WriteLine("No System.RuntimeType objects found.");
}
}
+
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"DumpRuntimeTypes finds all System.RuntimeType objects in the gc heap and
+prints the type name and MethodTable they refer too. Sample output:
+
+ Address Domain MT Type Name
+ ------------------------------------------------------------------------------
+ a515f4 14a740 5baf8d28 System.TypedReference
+ a51608 14a740 5bb05764 System.Globalization.BaseInfoTable
+ a51958 14a740 5bb05b24 System.Globalization.CultureInfo
+ a51a44 14a740 5bb06298 System.Globalization.GlobalizationAssembly
+ a51de0 14a740 5bb069c8 System.Globalization.TextInfo
+ a56b98 14a740 5bb12d28 System.Security.Permissions.HostProtectionResource
+ a56bbc 14a740 5baf7248 System.Int32
+ a56bd0 14a740 5baf3fdc System.String
+ a56cfc 14a740 5baf36a4 System.ValueType
+ ...
+
+This command will print a ""?"" in the domain column if the type is loaded into multiple
+AppDomains. For example:
+
+ {prompt}dumpruntimetypes
+ Address Domain MT Type Name
+ ------------------------------------------------------------------------------
+ 28435a0 ? 3f6a8c System.TypedReference
+ 28435b4 ? 214d6c System.ValueType
+ 28435c8 ? 216314 System.Enum
+ 28435dc ? 2147cc System.Object
+ 284365c ? 3cd57c System.IntPtr
+ 2843670 ? 3feaac System.Byte
+ 2843684 ? 23a544c System.IEquatable`1[[System.IntPtr, mscorlib]]
+ 2843784 ? 3c999c System.Int32
+ 2843798 ? 3caa04 System.IEquatable`1[[System.Int32, mscorlib]]
+";
}
}
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/DumpStackObjectsCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/DumpStackObjectsCommand.cs
index a6f94d055b..1a0919a057 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/DumpStackObjectsCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/DumpStackObjectsCommand.cs
@@ -69,6 +69,21 @@ public override void Invoke()
PrintStackObjects(range);
}
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"This command will display any managed objects it finds within the bounds of
+the current stack. Combined with the stack tracing commands like K and
+!ClrStack, it is a good aid to determining the values of locals and
+parameters.
+
+If you use the -verify option, each non-static CLASS field of an object
+candidate is validated. This helps to eliminate false positives. It is not
+on by default because very often in a debugging scenario, you are
+interested in objects with invalid fields.
+
+The abbreviation 'dso' can be used for brevity.
+";
+
private void PrintStackObjects(MemoryRange stack)
{
Console.WriteLine($"OS Thread Id: 0x{CurrentThread.ThreadId:x} ({CurrentThread.ThreadIndex})");
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/EEHeapCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/EEHeapCommand.cs
index 30013d5daa..636e207d5e 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/EEHeapCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/EEHeapCommand.cs
@@ -71,6 +71,91 @@ public override void Invoke()
PrintOneRuntime(Runtime);
}
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"The eeheap command enumerates process memory consumed by internal CLR data
+structures. You can limit the output by passing ""-gc"" or ""-loader"". All
+information will be displayed otherwise.
+
+The information for the Garbage Collector lists the ranges of each Segment in
+the managed heap. This can be useful if you believe you have an object pointer.
+If the pointer falls within a segment range given by 'eeheap -gc', then you do
+have an object pointer, and can attempt to run 'dumpobj' on it.
+
+Here is output for a simple program:
+
+ {prompt}eeheap -gc
+ Number of GC Heaps: 1
+ generation 0 starts at 0x00a71018
+ generation 1 starts at 0x00a7100c
+ generation 2 starts at 0x00a71000
+ segment begin allocated size
+ 00a70000 00a71000 00a7e01c 0000d01c(53276)
+ Large object heap starts at 0x01a71000
+ segment begin allocated size
+ 01a70000 01a71000 01a76000 0x00005000(20480)
+ Total Size 0x1201c(73756)
+ ------------------------------
+ GC Heap Size 0x1201c(73756)
+
+So the total size of the GC Heap is only 72K. On a large web server, with
+multiple processors, you can expect to see a GC Heap of 400MB or more. The
+Garbage Collector attempts to collect and reclaim memory only when required to
+by memory pressure for better performance. You can also see the notion of
+""generations,"" wherein the youngest objects live in generation 0, and
+long-lived objects eventually get ""promoted"" to generation 2.
+
+The loader output lists various private heaps associated with AppDomains. It
+also lists heaps associated with the JIT compiler, and heaps associated with
+Modules. For example:
+
+ {prompt}eeheap -loader
+ Loader Heap:
+ --------------------------------------
+ System Domain: 5e0662a0
+ LowFrequencyHeap:008f0000(00002000:00001000) Size: 0x00001000 bytes.
+ HighFrequencyHeap:008f2000(00008000:00001000) Size: 0x00001000 bytes.
+ StubHeap:008fa000(00002000:00001000) Size: 0x00001000 bytes.
+ Total size: 0x3000(12288)bytes
+ --------------------------------------
+ Shared Domain: 5e066970
+ LowFrequencyHeap:00920000(00002000:00001000) 03e30000(00010000:00003000) Size: 0x00004000 bytes.
+ Wasted: 0x00001000 bytes.
+ HighFrequencyHeap:00922000(00008000:00001000) Size: 0x00001000 bytes.
+ StubHeap:0092a000(00002000:00001000) Size: 0x00001000 bytes.
+ Total size: 0x6000(24576)bytes
+ --------------------------------------
+ Domain 1: 14f000
+ LowFrequencyHeap:00900000(00002000:00001000) 03ee0000(00010000:00003000) Size: 0x00004000 bytes.
+ Wasted: 0x00001000 bytes.
+ HighFrequencyHeap:00902000(00008000:00003000) Size: 0x00003000 bytes.
+ StubHeap:0090a000(00002000:00001000) Size: 0x00001000 bytes.
+ Total size: 0x8000(32768)bytes
+ --------------------------------------
+ Jit code heap:
+ Normal JIT:03ef0000(00010000:00002000) Size: 0x00002000 bytes.
+ Total size: 0x2000(8192)bytes
+ --------------------------------------
+ Module Thunk heaps:
+ Module 5ba22410: Size: 0x00000000 bytes.
+ Module 001c1320: Size: 0x00000000 bytes.
+ Module 001c03f0: Size: 0x00000000 bytes.
+ Module 001caa38: Size: 0x00000000 bytes.
+ Total size: 0x0(0)bytes
+ --------------------------------------
+ Module Lookup Table heaps:
+ Module 5ba22410:Size: 0x00000000 bytes.
+ Module 001c1320:Size: 0x00000000 bytes.
+ Module 001c03f0:Size: 0x00000000 bytes.
+ Module 001caa38:03ec0000(00010000:00002000) Size: 0x00002000 bytes.
+ Total size: 0x2000(8192)bytes
+ --------------------------------------
+ Total LoaderHeap size: 0x15000(86016)bytes
+ =======================================
+
+By using 'eeheap' to keep track of the growth of these private heaps, we are
+able to rule out or include them as a source of a memory leak.
+";
private ulong PrintOneRuntime(ClrRuntime clrRuntime)
{
StringBuilder stringBuilder = null;
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/FinalizeQueueCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/FinalizeQueueCommand.cs
index f74aee474a..4be92a9a1c 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/FinalizeQueueCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/FinalizeQueueCommand.cs
@@ -85,6 +85,66 @@ public override void Invoke()
}
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"This command lists the objects registered for finalization. Here is output from
+a simple program:
+
+ {prompt}finalizequeue
+ SyncBlocks to be cleaned up: 0
+ MTA Interfaces to be released: 0
+ STA Interfaces to be released: 1
+ generation 0 has 4 finalizable objects (0015bc90->0015bca0)
+ generation 1 has 0 finalizable objects (0015bc90->0015bc90)
+ generation 2 has 0 finalizable objects (0015bc90->0015bc90)
+ Ready for finalization 0 objects (0015bca0->0015bca0)
+ Statistics:
+ MT Count TotalSize Class Name
+ 5ba6cf78 1 24 Microsoft.Win32.SafeHandles.SafeFileHandle
+ 5ba5db04 1 68 System.Threading.Thread
+ 5ba73e28 2 112 System.IO.StreamWriter
+ Total 4 objects
+
+The GC heap is divided into generations, and objects are listed accordingly. We
+see that only generation 0 (the youngest generation) has any objects registered
+for finalization. The notation ""(0015bc90->0015bca0)"" means that if you look at
+memory in that range, you'll see the object pointers that are registered:
+
+ {prompt}dd 15bc90 15bca0-4
+ 0015bc90 00a743f4 00a79f00 00a7b3d8 00a7b47c
+
+You could run 'dumpobj' on any of those pointers to learn more. In this example,
+there are no objects ready for finalization, presumably because they still have
+roots (You can use 'gcroot' to find out). The statistics section provides a
+higher-level summary of the objects registered for finalization. Note that
+objects ready for finalization are also included in the statistics (if any).
+
+Specifying -short will inhibit any display related to SyncBlocks or RCWs.
+
+The arguments in detail:
+
+-allReady Specifying this argument will allow for the display of all objects
+ that are ready for finalization, whether they are already marked by
+ the GC as such or not. The former means GC already put them in the
+ ""Ready for finalization"" list and their finalizers are ready to run
+ but haven't run yet. The latter means there is nothing holding onto
+ these objects but GC hasn't noticed it yet because a GC that collects
+ the generation this object lives in has not happened yet. When that
+ GC happens, this object will be moved to the ""Ready for finalization""
+ list. For example, if a finalizable object lives in gen2 and a gen2 GC
+ has not happened, even if it's displayed by -allReady it's not actually
+ ready for finalization. This option can be very expensive, as it
+ verifies whether all the objects in the finalizable queues are still
+ rooted or not.
+-short Limits the output to just the address of each object. If used in
+ conjunction with -allReady it enumerates all objects that have a
+ finalizer that are no longer rooted. If used independently it lists
+ all objects in the finalizable and ""ready for finalization"" queues.
+-detail Will display extra information on any SyncBlocks that need to be
+ cleaned up, and on any RuntimeCallableWrappers (RCWs) that await
+ cleanup. Both of these data structures are cached and cleaned up by
+ the finalizer thread when it gets a chance to run.
+";
private IEnumerable EnumerateFinalizableObjects(bool allReady, ulong mt)
{
IEnumerable result = EnumerateValidFinalizableObjectsWithTypeFilter(mt);
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/GCRootCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/GCRootCommand.cs
index 28b5ab35b6..115a4d1132 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/GCRootCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/GCRootCommand.cs
@@ -111,6 +111,28 @@ public override void Invoke()
Console.WriteLine($"Found {count:n0} unique roots.");
}
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"GCRoot looks for references (or roots) to an object. These can exist in four
+places:
+
+ 1. On the stack
+ 2. Within a GC Handle
+ 3. In an object ready for finalization
+ 4. As a member of an object found in 1, 2 or 3 above.
+
+First, all stacks will be searched for roots, then handle tables, and finally
+the reachable queue of the finalizer. Some caution about the stack roots:
+GCRoot doesn't attempt to determine if a stack root it encountered is valid
+or is old (discarded) data. You would have to use !ClrStack and !U to
+disassemble the frame that the local or argument value belongs to in order to
+determine if it is still in use.
+
+Because people often want to restrict the search to gc handles and reachable
+objects, there is a -nostacks option.
+
+The -all option forces all roots to be displayed instead of just the unique roots.
+";
private int PrintOlderGenerationRoots(GCRoot gcroot, int gen, int limit)
{
int count = 0;
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/GCToNativeCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/GCToNativeCommand.cs
index e3e46d6d9f..ca58be44cd 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/GCToNativeCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/GCToNativeCommand.cs
@@ -574,7 +574,7 @@ specified memory will be displayed instead of just a summary table.
Sample Output:
- 0:000> gctonative PAGE_READWRITE
+ {prompt}gctonative PAGE_READWRITE
Walking GC heap to find pointers...
Resolving object names...
================================================ PAGE_READWRITE Regions ================================================
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/GCWhereCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/GCWhereCommand.cs
index 2ae3193846..454ba3cd0b 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/GCWhereCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/GCWhereCommand.cs
@@ -79,6 +79,22 @@ public override void Invoke()
}
}
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"GCWhere displays the location in the GC heap of the argument passed in.
+
+ {prompt}gcwhere 02800038
+ Address Gen Heap segment begin allocated size
+ 02800038 2 0 02800000 02800038 0282b740 12
+
+When the argument lies in the managed heap, but is not a valid *object* address
+the ""size"" is displayed as 0:
+
+ {prompt}gcwhere 0280003c
+ Address Gen Heap segment begin allocated size
+ 0280003c 2 0 02800000 02800038 0282b740 0
+
+";
private IEnumerable FindSegments(ulong address)
{
// ClrHeap.GetSegmentByAddress doesn't search for reserve memory
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/Host/CommandFormatHelpers.cs b/src/Microsoft.Diagnostics.ExtensionCommands/Host/CommandFormatHelpers.cs
index 6ef5026f5c..8a7e2f69ef 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/Host/CommandFormatHelpers.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/Host/CommandFormatHelpers.cs
@@ -12,8 +12,18 @@ namespace Microsoft.Diagnostics.ExtensionCommands
{
public static class CommandFormatHelpers
{
- public const string DacTableSymbol = "g_dacTable";
- public const string DebugHeaderSymbol = "DotNetRuntimeDebugHeader";
+ public const string DacTableExport = "g_dacTable";
+ public const string DebugHeaderExport = "DotNetRuntimeDebugHeader";
+ public const string ContractDescriptorExport = "DotNetRuntimeContractDescriptor";
+
+ public static void DisplaySettingService(this CommandBase command)
+ {
+ ISettingsService settingsService = command.Services.GetService() ?? throw new DiagnosticsException("Settings service required");
+ command.Console.WriteLine("Settings:");
+ command.Console.WriteLine($"-> Use CDAC contract reader: {settingsService.UseContractReader}");
+ command.Console.WriteLine($"-> Force use CDAC contract reader: {settingsService.ForceUseContractReader}");
+ command.Console.WriteLine($"-> DAC signature verification check enabled: {settingsService.DacSignatureVerificationEnabled}");
+ }
///
/// Displays the special diagnostics info header memory block (.NET Core 8 or later on Linux/MacOS)
@@ -127,28 +137,28 @@ IConsoleService Console()
IExportSymbols symbols = module.Services.GetService();
if (symbols != null && symbols.TryGetSymbolAddress(RuntimeInfo.RUNTIME_INFO_SYMBOL, out ulong infoAddress))
{
- Console().Write($"{indent}{RuntimeInfo.RUNTIME_INFO_SYMBOL,-24}: {infoAddress:X16}");
+ Console().Write($"{indent}{RuntimeInfo.RUNTIME_INFO_SYMBOL}: {infoAddress:X16}");
if (RuntimeInfo.TryRead(command.Services, infoAddress, out RuntimeInfo info))
{
Console().WriteLine(info.IsValid ? "" : " ");
- Console().WriteLine($"{indent} Signature: {info.Signature}");
- Console().WriteLine($"{indent} Version: {info.Version}");
- Console().WriteLine($"{indent} RuntimeModuleIndex: {info.RawRuntimeModuleIndex.ToHex()}");
- Console().WriteLine($"{indent} DacModuleIndex: {info.RawDacModuleIndex.ToHex()}");
- Console().WriteLine($"{indent} DbiModuleIndex: {info.RawDbiModuleIndex.ToHex()}");
+ Console().WriteLine($"{indent} Signature: {info.Signature}");
+ Console().WriteLine($"{indent} Version: {info.Version}");
+ Console().WriteLine($"{indent} RuntimeModuleIndex: {info.RawRuntimeModuleIndex.ToHex()}");
+ Console().WriteLine($"{indent} DacModuleIndex: {info.RawDacModuleIndex.ToHex()}");
+ Console().WriteLine($"{indent} DbiModuleIndex: {info.RawDbiModuleIndex.ToHex()}");
if (module.IsPEImage)
{
- Console().WriteLine($"{indent} RuntimePEIndex: {info.RuntimePEIIndex.timeStamp:X8}/{info.RuntimePEIIndex.fileSize:X}");
- Console().WriteLine($"{indent} DacPEIndex: {info.DacPEIndex.timeStamp:X8}/{info.DacPEIndex.fileSize:X}");
- Console().WriteLine($"{indent} DbiPEIndex: {info.DbiPEIndex.timeStamp:X8}/{info.DbiPEIndex.fileSize:X}");
+ Console().WriteLine($"{indent} RuntimePEIndex: {info.RuntimePEIIndex.timeStamp:X8}/{info.RuntimePEIIndex.fileSize:X}");
+ Console().WriteLine($"{indent} DacPEIndex: {info.DacPEIndex.timeStamp:X8}/{info.DacPEIndex.fileSize:X}");
+ Console().WriteLine($"{indent} DbiPEIndex: {info.DbiPEIndex.timeStamp:X8}/{info.DbiPEIndex.fileSize:X}");
}
else
{
- Console().WriteLine($"{indent} RuntimeBuildId: {info.RuntimeBuildId.ToHex()}");
- Console().WriteLine($"{indent} DacBuildId: {info.DacBuildId.ToHex()}");
- Console().WriteLine($"{indent} DbiBuildId: {info.DbiBuildId.ToHex()}");
+ Console().WriteLine($"{indent} RuntimeBuildId: {info.RuntimeBuildId.ToHex()}");
+ Console().WriteLine($"{indent} DacBuildId: {info.DacBuildId.ToHex()}");
+ Console().WriteLine($"{indent} DbiBuildId: {info.DbiBuildId.ToHex()}");
}
- Console().WriteLine($"{indent} RuntimeVersion: {info.RuntimeVersion?.ToString() ?? ""}");
+ Console().WriteLine($"{indent} RuntimeVersion: {info.RuntimeVersion?.ToString() ?? ""}");
}
else
{
@@ -157,7 +167,7 @@ IConsoleService Console()
}
else if (error)
{
- Console().WriteLine($"{indent}{RuntimeInfo.RUNTIME_INFO_SYMBOL,-24}: ");
+ Console().WriteLine($"{indent}{RuntimeInfo.RUNTIME_INFO_SYMBOL}: ");
}
// Print the Windows runtime engine metrics (.NET Core and .NET Framework)
@@ -166,13 +176,13 @@ IConsoleService Console()
{
if (symbols != null && symbols.TryGetSymbolAddress(ClrEngineMetrics.Symbol, out ulong metricsAddress))
{
- Console().Write($"{indent}{ClrEngineMetrics.Symbol,-24}: ({metricsAddress:X16})");
+ Console().Write($"{indent}{ClrEngineMetrics.Symbol}: {metricsAddress:X16}");
if (ClrEngineMetrics.TryRead(command.Services, metricsAddress, out ClrEngineMetrics metrics))
{
Console().WriteLine();
- Console().WriteLine($"{indent} Size: {metrics.Size} (0x{metrics.Size:X2})");
- Console().WriteLine($"{indent} DbiVersion: {metrics.DbiVersion}");
- Console().WriteLine($"{indent} ContinueStartupEvent: {((ulong)metrics.ContinueStartupEvent):X16}");
+ Console().WriteLine($"{indent} Size: {metrics.Size} (0x{metrics.Size:X2})");
+ Console().WriteLine($"{indent} DbiVersion: {metrics.DbiVersion}");
+ Console().WriteLine($"{indent} ContinueStartupEvent: {((ulong)metrics.ContinueStartupEvent):X16}");
}
else
{
@@ -181,28 +191,38 @@ IConsoleService Console()
}
else if (error)
{
- Console().WriteLine($"{indent}{ClrEngineMetrics.Symbol,-24}: ");
+ Console().WriteLine($"{indent}{ClrEngineMetrics.Symbol}: ");
}
}
// Print the DAC table address (g_dacTable)
- if (symbols != null && symbols.TryGetSymbolAddress(DacTableSymbol, out ulong dacTableAddress))
+ if (symbols != null && symbols.TryGetSymbolAddress(DacTableExport, out ulong dacTableAddress))
{
- Console().WriteLine($"{indent}{DacTableSymbol,-24}: {dacTableAddress:X16}");
+ Console().WriteLine($"{indent}{DacTableExport}: {dacTableAddress:X16}");
}
else if (error)
{
- Console().WriteLine($"{indent}{DacTableSymbol,-24}: ");
+ Console().WriteLine($"{indent}{DacTableExport}: ");
}
// Print the Native AOT contract data address (DotNetRuntimeDebugHeader)
- if (symbols != null && symbols.TryGetSymbolAddress(DebugHeaderSymbol, out ulong debugHeaderAddress))
+ if (symbols != null && symbols.TryGetSymbolAddress(DebugHeaderExport, out ulong debugHeaderAddress))
+ {
+ Console().WriteLine($"{indent}{DebugHeaderExport}: {debugHeaderAddress:X16}");
+ }
+ else if (error)
+ {
+ Console().WriteLine($"{indent}{DebugHeaderExport}: ");
+ }
+
+ // Print the CDAC contract data address (DotNetRuntimeContractDescriptor)
+ if (symbols != null && symbols.TryGetSymbolAddress(ContractDescriptorExport, out ulong contractDescriptorAddress))
{
- Console().WriteLine($"{indent}{DebugHeaderSymbol,-24}: {debugHeaderAddress:X16}");
+ Console().WriteLine($"{indent}{ContractDescriptorExport}: {contractDescriptorAddress:X16}");
}
else if (error)
{
- Console().WriteLine($"{indent}{DebugHeaderSymbol,-24}: ");
+ Console().WriteLine($"{indent}{ContractDescriptorExport}: ");
}
}
}
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/Host/RuntimesCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/Host/RuntimesCommand.cs
index 751df3a496..83c70cef29 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/Host/RuntimesCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/Host/RuntimesCommand.cs
@@ -18,7 +18,7 @@ public class RuntimesCommand : CommandBase
[ServiceImport]
public IContextService ContextService { get; set; }
- [ServiceImport(Optional = true)]
+ [ServiceImport]
public ISettingsService SettingsService { get; set; }
[ServiceImport]
@@ -36,6 +36,12 @@ public class RuntimesCommand : CommandBase
[Option(Name = "--all", Aliases = new string[] { "-a" }, Help = "Forces all runtimes to be enumerated.")]
public bool All { get; set; }
+ [Option(Name = "--usecdac", Help = "Use the CDAC if available and requested (true/false).")]
+ public bool? UseContractReader { get; set; }
+
+ [Option(Name = "--forceusecdac", Help = "Always use the CDAC (true/false).")]
+ public bool? ForceUseContractReader { get; set; }
+
[Option(Name = "--DacSignatureVerification", Aliases = new string[] { "-v" }, Help = "Enforce the proper DAC certificate signing when loaded (true/false).")]
public bool? DacSignatureVerification { get; set; }
@@ -46,14 +52,24 @@ public override void Invoke()
throw new DiagnosticsException("Cannot specify both -netfx and -netcore options");
}
+ bool flush = false;
+ if (UseContractReader.HasValue)
+ {
+ SettingsService.UseContractReader = UseContractReader.Value;
+ flush = true;
+ }
+
+ if (ForceUseContractReader.HasValue)
+ {
+ SettingsService.UseContractReader = ForceUseContractReader.Value;
+ SettingsService.ForceUseContractReader = ForceUseContractReader.Value;
+ flush = true;
+ }
+
if (DacSignatureVerification.HasValue)
{
- if (SettingsService is null)
- {
- throw new DiagnosticsException("Changing the DAC signature verification setting not supported");
- }
SettingsService.DacSignatureVerificationEnabled = DacSignatureVerification.Value;
- Target.Flush();
+ flush = true;
}
RuntimeEnumerationFlags flags = RuntimeEnumerationFlags.Default;
@@ -61,6 +77,11 @@ public override void Invoke()
{
// Force all runtimes to be enumerated. This requires a target flush.
flags = RuntimeEnumerationFlags.All;
+ flush = true;
+ }
+
+ if (flush)
+ {
Target.Flush();
}
@@ -107,8 +128,8 @@ public override void Invoke()
}
this.DisplayResources(runtime.RuntimeModule, all: false, indent: " ");
this.DisplayRuntimeExports(runtime.RuntimeModule, error: true, indent: " ");
- WriteLine($"DAC signature verification check enabled: {SettingsService.DacSignatureVerificationEnabled}");
}
+ this.DisplaySettingService();
}
}
}
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/Host/SetSymbolServerCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/Host/SetSymbolServerCommand.cs
index 9c6ca4f4da..33a36157a7 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/Host/SetSymbolServerCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/Host/SetSymbolServerCommand.cs
@@ -123,28 +123,23 @@ public override void Invoke()
[HelpInvoke]
public static string GetDetailedHelp(IHost host)
{
- switch (host.HostType)
+ return host.HostType switch
{
- case HostType.DbgEng:
- return s_detailedHelpTextDbgEng;
- case HostType.Lldb:
- return s_detailedHelpTextLLDB;
- case HostType.DotnetDump:
- return s_detailedHelpTextDotNetDump;
- }
- return null;
+ HostType.DbgEng => s_detailedHelpTextDbgEng,
+ HostType.Lldb => s_detailedHelpTextLLDB,
+ HostType.DotnetDump => s_detailedHelpTextDotNetDump,
+ _ => null,
+ };
}
private const string s_detailedHelpTextDbgEng =
- @"
-This commands enables symbol server support for portable PDBs for managed assemblies and
+@"This commands enables symbol server support for portable PDBs for managed assemblies and
.NET Core native modules files (like the DAC) in SOS. If the .sympath is set, the symbol
server supported is automatically set and this command isn't necessary.
";
private const string s_detailedHelpTextLLDB =
- @"
-This commands enables symbol server support in SOS. The portable PDBs for managed assemblies
+@"This commands enables symbol server support in SOS. The portable PDBs for managed assemblies
and .NET Core native symbol and module (like the DAC) files are downloaded.
To enable downloading symbols from the Microsoft symbol server:
@@ -183,8 +178,7 @@ file structure that the core dump was generated.
";
private const string s_detailedHelpTextDotNetDump =
- @"
-This commands enables symbol server support in SOS. The portable PDBs for managed assemblies
+@"This commands enables symbol server support in SOS. The portable PDBs for managed assemblies
and .NET Core native module (like the DAC) files are downloaded.
To enable downloading symbols from the Microsoft symbol server:
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/Host/StatusCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/Host/StatusCommand.cs
index 6f7f343f12..3c425e251d 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/Host/StatusCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/Host/StatusCommand.cs
@@ -6,6 +6,7 @@
using System.Diagnostics;
using System.Linq;
using System.Reflection;
+using System.Runtime.InteropServices;
using Microsoft.Diagnostics.DebugServices;
namespace Microsoft.Diagnostics.ExtensionCommands
@@ -62,6 +63,7 @@ public override void Invoke()
}
}
this.DisplaySpecialInfo();
+ this.DisplaySettingService();
Write(SymbolService.ToString());
List extensions = new(ServiceManager.ExtensionsLoaded);
@@ -74,7 +76,7 @@ public override void Invoke()
FileVersionInfo versionInfo = FileVersionInfo.GetVersionInfo(path);
WriteLine($"-> {versionInfo.ProductVersion} {path}");
}
-
+ WriteLine($"Host runtime {RuntimeInformation.FrameworkDescription} on {RuntimeInformation.OSDescription} {RuntimeInformation.OSArchitecture}");
long memoryUsage = GC.GetTotalMemory(forceFullCollection: true);
WriteLine($"GC memory usage for managed SOS components: {memoryUsage:##,#} bytes");
}
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/ListNearObjCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/ListNearObjCommand.cs
index afeffcbfed..021a2add45 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/ListNearObjCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/ListNearObjCommand.cs
@@ -263,6 +263,38 @@ public override void Invoke()
}
}
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"ListNearObj is a diagnostic tool that displays the object preceeding and
+succeeding the address passed in:
+
+The command looks for the address in the GC heap that looks like a valid
+beginning of a managed object (based on a valid method table) and the object
+following the argument address.
+
+ {prompt}listnearobj 028000ec
+ Before: 0x28000a4 72 (0x48 ) System.StackOverflowException
+ After: 0x2800134 72 (0x48 ) System.Threading.ThreadAbortException
+ Heap local consistency confirmed.
+
+ {prompt}listnearobj 028000f0
+ Before: 0x28000ec 72 (0x48 ) System.ExecutionEngineException
+ After: 0x2800134 72 (0x48 ) System.Threading.ThreadAbortException
+ Heap local consistency confirmed.
+
+The command considers the heap as ""locally consistent"" if:
+ prev_obj_addr + prev_obj_size = arg_addr && arg_obj + arg_size = next_obj_addr
+OR
+ prev_obj_addr + prev_obj_size = next_obj_addr
+
+When the condition is not satisfied:
+
+ {prompt}listnearobj 028000ec
+ Before: 0x28000a4 72 (0x48 ) System.StackOverflowException
+ After: 0x2800134 72 (0x48 ) System.Threading.ThreadAbortException
+ Heap local consistency not confirmed.
+";
+
private void CheckEndOfSegment(ClrSegment segment, ulong expectedNextObject, ulong prevObjectAddress, ref bool localConsistency, ref bool foundLastObject)
{
if (!segment.ObjectRange.Contains(expectedNextObject) && !foundLastObject)
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/MAddressCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/MAddressCommand.cs
index 9a1774fdc6..674e62b9fa 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/MAddressCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/MAddressCommand.cs
@@ -202,7 +202,7 @@ orderby Size descending
[HelpInvoke]
public static string GetDetailedHelp() =>
$@"-------------------------------------------------------------------------------
-!maddress is a managed version of !address, which attempts to annotate all memory
+The maddress command is a managed version of !address, which attempts to annotate all memory
with information about CLR's heaps.
usage: !maddress [{SummaryFlag}] [{ImagesFlag}] [{ForceHandleTableFlag}] [{ReserveFlag} [{ReserveHeuristicFlag}]]
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/ObjSizeCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/ObjSizeCommand.cs
index 6906ff4f9e..2c81926831 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/ObjSizeCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/ObjSizeCommand.cs
@@ -86,6 +86,42 @@ public override void Invoke()
DumpHeap.PrintHeap(GetTransitiveClosure(objects), displayKind, Stat, printFragmentation: false);
}
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"With no parameters, 'objsize' lists the size of all objects found on managed
+threads. It also enumerates all GCHandles in the process, and totals the size
+of any objects pointed to by those handles. In calculating object size,
+!ObjSize includes the size of all child objects in addition to the parent.
+
+For example, 'dumpobj' lists a size of 20 bytes for this Customer object:
+
+ {prompt}dumpobj a79d40
+ Name: Customer
+ MethodTable: 009038ec
+ EEClass: 03ee1b84
+ Size: 20(0x14) bytes
+ (C:\pub\unittest.exe)
+ Fields:
+ MT Field Offset Type Attr Value Name
+ 009038ec 4000008 4 CLASS instance 00a79ce4 name
+ 009038ec 4000009 8 CLASS instance 00a79d2c bank
+ 009038ec 400000a c System.Boolean instance 1 valid
+
+but 'objsize' lists 152 bytes:
+
+ {prompt}objsize a79d40
+ sizeof(00a79d40) = 152 ( 0x98) bytes (Customer)
+
+This is because a Customer points to a Bank, has a name, and the Bank points to
+an Address string. You can use !ObjSize to identify any particularly large
+objects, such as a managed cache in a web server.
+
+While running ObjSize with no arguments may point to specific roots that hold
+onto large amounts of memory it does not provide information regarding the
+amount of managed memory that is still alive. This is due to the fact that a
+number of roots can share a common subgraph, and that part will be reported in
+the size of all the roots that reference the subgraph.
+";
private static IEnumerable GetTransitiveClosure(IEnumerable objects)
{
HashSet seen = new();
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/TaskStateCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/TaskStateCommand.cs
index 716fec0903..caad03cc99 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/TaskStateCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/TaskStateCommand.cs
@@ -60,15 +60,13 @@ public override void Invoke()
[HelpInvoke]
public static string GetDetailedHelp() =>
@"-------------------------------------------------------------------------------
-TaskState [hexa address] [-v ]
-
TaskState translates a Task m_stateFlags field value into human readable format.
It supports hexadecimal address corresponding to a task instance or -v .
-> tks 000001db16cf98f0
+{prompt}tks 000001db16cf98f0
Running
-> tks -v 73728
+{prompt}tks -v 73728
WaitingToRun
";
}
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/ThreadPoolCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/ThreadPoolCommand.cs
index a1ca305071..92d3cdcdab 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/ThreadPoolCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/ThreadPoolCommand.cs
@@ -133,6 +133,12 @@ It is the only option in .NET 6 and below. The UsePortableThreadPoolForIO field
}
}
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"This command lists basic information about the ThreadPool, including the number
+of work requests in the queue, number of completion port threads, and number of
+timers.
+";
private void DumpWorkItems()
{
Table output = null;
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/ThreadPoolQueueCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/ThreadPoolQueueCommand.cs
index 85b5d9b4ed..06b9965904 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/ThreadPoolQueueCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/ThreadPoolQueueCommand.cs
@@ -105,13 +105,11 @@ private static void UpdateStats(Dictionary stats, string statN
[HelpInvoke]
public static string GetDetailedHelp() =>
@"-------------------------------------------------------------------------------
-ThreadPoolQueue
-
ThreadPoolQueue lists the enqueued work items in the Clr Thread Pool followed by a summary of the different tasks/work items.
The global queue is first iterated before local per-thread queues.
The name of the method to be called (on which instance if any) is also provided when available.
-> tpq
+{prompt}tpq
global work item queue________________________________
0x000002AC3C1DDBB0 Work | (ASP.global_asax)System.Web.HttpApplication.ResumeStepsWaitCallback
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/TimersCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/TimersCommand.cs
index 2c576c7193..349a1c7a2f 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/TimersCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/TimersCommand.cs
@@ -107,12 +107,10 @@ private static string GetTimerString(TimerInfo timer)
[HelpInvoke]
public static string GetDetailedHelp() =>
@"-------------------------------------------------------------------------------
-TimerInfo
-
TimerInfo lists all the running timers followed by a summary of the different items.
The name of the method to be called (on which instance if any) is also provided when available.
-> ti
+{prompt}ti
0x000001E29BD45848 @ 964 ms every 1000 ms | 0x000001E29BD0C828 (Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.Heartbeat) ->
0x000001E19BD0F868 @ 1 ms every ------ ms | 0x000001E19BD0F800 (System.Threading.Tasks.Task+DelayPromise) -> System.Threading.Tasks.Task+<>c.b__260_1
0x000001E09BD09B40 @ 1 ms every ------ ms | 0x000001E09BD09AD8 (System.Threading.Tasks.Task+DelayPromise) -> System.Threading.Tasks.Task+<>c.b__260_1
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/TraverseHeapCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/TraverseHeapCommand.cs
index 57315180bb..d9ef4f1632 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/TraverseHeapCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/TraverseHeapCommand.cs
@@ -70,6 +70,47 @@ public override void Invoke()
}
}
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"TraverseHeap writes out a file in a format understood by the CLR Profiler.
+You can download the CLR Profiler from this link:
+
+http://www.microsoft.com/downloads/details.aspx?FamilyId=86CE6052-D7F4-4AEB-
+9B7A-94635BEEBDDA&displaylang=en
+
+It creates a graphical display of the GC heap to help you analyze the state of
+your application.
+
+If you pass the -verify option it will do more sanity checking of the heap
+as it dumps it. Use this option if heap corruption is suspected.
+
+If you pass the ""-xml"" flag, the file is instead written out in an easy to
+understand xml format:
+
+
+
+
+ ...
+
+
+
+
+ ...
+
+
+
+
+ ...
+
+
+
+You can break into your process, load SOS, take a snapshot of your heap with
+this function, then continue.
+";
+
private (MemoryStream Stream, Dictionary Types) WriteRootsAndObjects()
{
Dictionary types = new();
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/VerifyHeapCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/VerifyHeapCommand.cs
index a7645b4d82..4ce410e576 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/VerifyHeapCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/VerifyHeapCommand.cs
@@ -61,6 +61,44 @@ public override void Invoke()
VerifyHeap(filteredHeap.EnumerateFilteredObjects(Console.CancellationToken), verifySyncTable: filteredHeap.HasFilters);
}
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"VerifyHeap is a diagnostic tool that checks the garbage collected heap for
+signs of corruption. It walks objects one by one in a pattern like this:
+
+ o = firstobject;
+ while(o != endobject)
+ {
+ o.ValidateAllFields();
+ o = (Object *) o + o.Size();
+ }
+
+If an error is found, VerifyHeap will report it. I'll take a perfectly good
+object and corrupt it:
+
+ {prompt}dumpobj a79d40
+ Name: Customer
+ MethodTable: 009038ec
+ EEClass: 03ee1b84
+ Size: 20(0x14) bytes
+ (C:\pub\unittest.exe)
+ Fields:
+ MT Field Offset Type Attr Value Name
+ 009038ec 4000008 4 CLASS instance 00a79ce4 name
+ 009038ec 4000009 8 CLASS instance 00a79d2c bank
+ 009038ec 400000a c System.Boolean instance 1 valid
+
+ {prompt}ed a79d40+4 01 (change the name field to the bogus pointer value 1)
+ {prompt}verifyheap
+ object 01ee60dc: bad member 00000003 at 01EE6168
+ Last good object: 01EE60C4.
+
+If this gc heap corruption exists, there is a serious bug in your own code or
+in the CLR. In user code, an error in constructing PInvoke calls can cause
+this problem, and running with Managed Debugging Assistants is advised. If that
+possibility is eliminated, consider contacting Microsoft Product Support for
+help.
+";
private IEnumerable EnumerateWithCount(IEnumerable objs)
{
_totalObjects = 0;
diff --git a/src/Microsoft.Diagnostics.ExtensionCommands/VerifyObjectCommand.cs b/src/Microsoft.Diagnostics.ExtensionCommands/VerifyObjectCommand.cs
index a9779d337b..aaa37203e3 100644
--- a/src/Microsoft.Diagnostics.ExtensionCommands/VerifyObjectCommand.cs
+++ b/src/Microsoft.Diagnostics.ExtensionCommands/VerifyObjectCommand.cs
@@ -49,5 +49,17 @@ public override void Invoke()
Console.WriteLine();
Console.WriteLine($"{corruption.Length:n0} error{(corruption.Length == 1 ? "" : "s")} detected.");
}
+
+ [HelpInvoke]
+ public static string GetDetailedHelp() =>
+@"VerifyObj is a diagnostic tool that checks the object that is passed as an
+argument for signs of corruption.
+
+ {prompt}verifyobj 028000ec
+ object 0x28000ec does not have valid method table
+
+ {prompt}verifyobj 0680017c
+ object 0x680017c: bad member 00000001 at 06800184
+";
}
}
diff --git a/src/Microsoft.Diagnostics.Monitoring.EventPipe/Counters/MetricsPipeline.cs b/src/Microsoft.Diagnostics.Monitoring.EventPipe/Counters/MetricsPipeline.cs
index 65c2ba2882..b7b470c48b 100644
--- a/src/Microsoft.Diagnostics.Monitoring.EventPipe/Counters/MetricsPipeline.cs
+++ b/src/Microsoft.Diagnostics.Monitoring.EventPipe/Counters/MetricsPipeline.cs
@@ -68,10 +68,12 @@ protected override async Task OnEventSourceAvailable(EventPipeEventSource eventS
{
await ExecuteCounterLoggerActionAsync((metricLogger) => metricLogger.PipelineStarted(token)).ConfigureAwait(false);
+ CounterMetadataCache cache = new();
+
eventSource.Dynamic.All += traceEvent => {
try
{
- if (traceEvent.TryGetCounterPayload(_counterConfiguration, out ICounterPayload counterPayload))
+ if (traceEvent.TryGetCounterPayload(cache, _counterConfiguration, out ICounterPayload counterPayload))
{
ExecuteCounterLoggerAction((metricLogger) => metricLogger.Log(counterPayload));
}
diff --git a/src/Microsoft.Diagnostics.Monitoring.EventPipe/Counters/TraceEventExtensions.cs b/src/Microsoft.Diagnostics.Monitoring.EventPipe/Counters/TraceEventExtensions.cs
index 5300a73987..01bc977816 100644
--- a/src/Microsoft.Diagnostics.Monitoring.EventPipe/Counters/TraceEventExtensions.cs
+++ b/src/Microsoft.Diagnostics.Monitoring.EventPipe/Counters/TraceEventExtensions.cs
@@ -34,13 +34,20 @@ public CounterConfiguration(CounterFilter filter)
internal record struct ProviderAndCounter(string ProviderName, string CounterName);
+ internal sealed class CounterMetadataCache
+ {
+ public Dictionary CounterMetadataByName { get; } = new();
+ public Dictionary CounterMetadataById { get; } = new();
+ }
+
internal static partial class TraceEventExtensions
{
- private static Dictionary counterMetadataByName = new();
- private static Dictionary counterMetadataById = new();
+ // This cache is used to track shared sessions that have already been marked as inactive.
+ // It can be shared between processes.
private static HashSet inactiveSharedSessions = new(StringComparer.OrdinalIgnoreCase);
private static CounterMetadata AddCounterMetadata(
+ CounterMetadataCache counterMetadataCache,
string providerName,
string counterName,
int? id,
@@ -52,7 +59,7 @@ private static CounterMetadata AddCounterMetadata(
string counterDescription = null)
{
CounterMetadata metadata;
- if (id.HasValue && counterMetadataById.TryGetValue(id.Value, out metadata))
+ if (id.HasValue && counterMetadataCache.CounterMetadataById.TryGetValue(id.Value, out metadata))
{
return metadata;
}
@@ -61,7 +68,7 @@ private static CounterMetadata AddCounterMetadata(
// listening to it then.
// Its also possible that we previously indexed a counter with the same name as this one but with different tags or scope hash.
ProviderAndCounter providerAndCounter = new(providerName, counterName);
- if (counterMetadataByName.TryGetValue(providerAndCounter, out metadata))
+ if (counterMetadataCache.CounterMetadataByName.TryGetValue(providerAndCounter, out metadata))
{
// we found a counter that matches the name, but it might not match everything
if (metadata.MeterTags == meterTags && metadata.InstrumentTags == instrumentTags && metadata.ScopeHash == scopeHash)
@@ -69,7 +76,7 @@ private static CounterMetadata AddCounterMetadata(
// add the ID index if it didn't exist before
if (id.HasValue)
{
- counterMetadataById.TryAdd(id.Value, metadata);
+ counterMetadataCache.CounterMetadataById.TryAdd(id.Value, metadata);
}
return metadata;
}
@@ -79,45 +86,45 @@ private static CounterMetadata AddCounterMetadata(
metadata = new CounterMetadata(providerName, providerVersion, counterName, counterUnit, counterDescription, id, meterTags, instrumentTags, scopeHash);
if (id.HasValue)
{
- counterMetadataById.TryAdd(id.Value, metadata);
+ counterMetadataCache.CounterMetadataById.TryAdd(id.Value, metadata);
}
- counterMetadataByName.TryAdd(providerAndCounter, metadata);
+ counterMetadataCache.CounterMetadataByName.TryAdd(providerAndCounter, metadata);
return metadata;
}
- private static CounterMetadata GetCounterMetadata(string providerName, string counterName, int? id)
+ private static CounterMetadata GetCounterMetadata(CounterMetadataCache counterMetadataCache, string providerName, string counterName, int? id)
{
// Lookup by ID is preferred because it eliminates ambiguity in the case of duplicate provider/counter names.
// IDs are present starting in MetricsEventSource 9.0.
// Duplicate named providers/counters might still have different tags or scope hashes
CounterMetadata metadata;
- if (id.HasValue && counterMetadataById.TryGetValue(id.Value, out metadata))
+ if (id.HasValue && counterMetadataCache.CounterMetadataById.TryGetValue(id.Value, out metadata))
{
return metadata;
}
ProviderAndCounter providerAndCounter = new(providerName, counterName);
- if (counterMetadataByName.TryGetValue(providerAndCounter, out metadata))
+ if (counterMetadataCache.CounterMetadataByName.TryGetValue(providerAndCounter, out metadata))
{
return metadata;
}
// For EventCounter based events we expect to fall through here the first time a new counter is observed
// For MetricsEventSource events we should never reach here unless the BeginInstrumentRecording event was dropped.
- return AddCounterMetadata(providerName, counterName, id, null, null, null);
+ return AddCounterMetadata(counterMetadataCache, providerName, counterName, id, null, null, null);
}
- public static bool TryGetCounterMetadata(string providerName, string counterName, int? instrumentId, out CounterMetadata counterMetadata)
+ public static bool TryGetCounterMetadata(CounterMetadataCache counterMetadataCache, string providerName, string counterName, int? instrumentId, out CounterMetadata counterMetadata)
{
- if (instrumentId.HasValue && counterMetadataById.TryGetValue(instrumentId.Value, out counterMetadata))
+ if (instrumentId.HasValue && counterMetadataCache.CounterMetadataById.TryGetValue(instrumentId.Value, out counterMetadata))
{
return true;
}
ProviderAndCounter providerAndCounter = new(providerName, counterName);
- return counterMetadataByName.TryGetValue(providerAndCounter, out counterMetadata);
+ return counterMetadataCache.CounterMetadataByName.TryGetValue(providerAndCounter, out counterMetadata);
}
- public static bool TryGetCounterPayload(this TraceEvent traceEvent, CounterConfiguration counterConfiguration, out ICounterPayload payload)
+ public static bool TryGetCounterPayload(this TraceEvent traceEvent, CounterMetadataCache counterMetadataCache, CounterConfiguration counterConfiguration, out ICounterPayload payload)
{
payload = null;
@@ -183,23 +190,23 @@ public static bool TryGetCounterPayload(this TraceEvent traceEvent, CounterConfi
{
if (traceEvent.EventName == "BeginInstrumentReporting")
{
- HandleBeginInstrumentReporting(traceEvent, counterConfiguration, out payload);
+ HandleBeginInstrumentReporting(traceEvent, counterMetadataCache, counterConfiguration, out payload);
}
if (traceEvent.EventName == "HistogramValuePublished")
{
- HandleHistogram(traceEvent, counterConfiguration, out payload);
+ HandleHistogram(traceEvent, counterMetadataCache, counterConfiguration, out payload);
}
else if (traceEvent.EventName == "GaugeValuePublished")
{
- HandleGauge(traceEvent, counterConfiguration, out payload);
+ HandleGauge(traceEvent, counterMetadataCache, counterConfiguration, out payload);
}
else if (traceEvent.EventName == "CounterRateValuePublished")
{
- HandleCounterRate(traceEvent, counterConfiguration, out payload);
+ HandleCounterRate(traceEvent, counterMetadataCache, counterConfiguration, out payload);
}
else if (traceEvent.EventName == "UpDownCounterRateValuePublished")
{
- HandleUpDownCounterValue(traceEvent, counterConfiguration, out payload);
+ HandleUpDownCounterValue(traceEvent, counterMetadataCache, counterConfiguration, out payload);
}
else if (traceEvent.EventName == "TimeSeriesLimitReached")
{
@@ -232,7 +239,7 @@ public static bool TryGetCounterPayload(this TraceEvent traceEvent, CounterConfi
return false;
}
- private static void HandleGauge(TraceEvent obj, CounterConfiguration counterConfiguration, out ICounterPayload payload)
+ private static void HandleGauge(TraceEvent obj, CounterMetadataCache counterMetadataCache, CounterConfiguration counterConfiguration, out ICounterPayload payload)
{
payload = null;
@@ -261,7 +268,7 @@ private static void HandleGauge(TraceEvent obj, CounterConfiguration counterConf
return;
}
- CounterMetadata metadata = GetCounterMetadata(meterName, instrumentName, id);
+ CounterMetadata metadata = GetCounterMetadata(counterMetadataCache, meterName, instrumentName, id);
// the value might be an empty string indicating no measurement was provided this collection interval
if (double.TryParse(lastValueText, NumberStyles.Number | NumberStyles.Float, CultureInfo.InvariantCulture, out double lastValue))
{
@@ -275,7 +282,7 @@ private static void HandleGauge(TraceEvent obj, CounterConfiguration counterConf
}
}
- private static void HandleBeginInstrumentReporting(TraceEvent traceEvent, CounterConfiguration counterConfiguration, out ICounterPayload payload)
+ private static void HandleBeginInstrumentReporting(TraceEvent traceEvent, CounterMetadataCache counterMetadataCache, CounterConfiguration counterConfiguration, out ICounterPayload payload)
{
payload = null;
@@ -319,6 +326,7 @@ private static void HandleBeginInstrumentReporting(TraceEvent traceEvent, Counte
}
payload = new BeginInstrumentReportingPayload(
AddCounterMetadata(
+ counterMetadataCache,
meterName,
instrumentName,
instrumentID,
@@ -331,7 +339,7 @@ private static void HandleBeginInstrumentReporting(TraceEvent traceEvent, Counte
traceEvent.TimeStamp);
}
- private static void HandleCounterRate(TraceEvent traceEvent, CounterConfiguration counterConfiguration, out ICounterPayload payload)
+ private static void HandleCounterRate(TraceEvent traceEvent, CounterMetadataCache counterMetadataCache, CounterConfiguration counterConfiguration, out ICounterPayload payload)
{
payload = null;
@@ -364,7 +372,7 @@ private static void HandleCounterRate(TraceEvent traceEvent, CounterConfiguratio
{
return;
}
- CounterMetadata metadata = GetCounterMetadata(meterName, instrumentName, id);
+ CounterMetadata metadata = GetCounterMetadata(counterMetadataCache, meterName, instrumentName, id);
if (double.TryParse(rateText, NumberStyles.Number | NumberStyles.Float, CultureInfo.InvariantCulture, out double rate))
{
if (absoluteValueText != null &&
@@ -387,7 +395,7 @@ private static void HandleCounterRate(TraceEvent traceEvent, CounterConfiguratio
}
}
- private static void HandleUpDownCounterValue(TraceEvent traceEvent, CounterConfiguration configuration, out ICounterPayload payload)
+ private static void HandleUpDownCounterValue(TraceEvent traceEvent, CounterMetadataCache counterMetadataCache, CounterConfiguration configuration, out ICounterPayload payload)
{
payload = null;
@@ -422,7 +430,7 @@ private static void HandleUpDownCounterValue(TraceEvent traceEvent, CounterConfi
rateText = valueText;
}
- CounterMetadata metadata = GetCounterMetadata(meterName, instrumentName, id);
+ CounterMetadata metadata = GetCounterMetadata(counterMetadataCache, meterName, instrumentName, id);
if (double.TryParse(rateText, NumberStyles.Number | NumberStyles.Float, CultureInfo.InvariantCulture, out double rate)
&& double.TryParse(valueText, NumberStyles.Number | NumberStyles.Float, CultureInfo.InvariantCulture, out double value))
{
@@ -437,7 +445,7 @@ private static void HandleUpDownCounterValue(TraceEvent traceEvent, CounterConfi
}
}
- private static void HandleHistogram(TraceEvent obj, CounterConfiguration configuration, out ICounterPayload payload)
+ private static void HandleHistogram(TraceEvent obj, CounterMetadataCache counterMetadataCache, CounterConfiguration configuration, out ICounterPayload payload)
{
payload = null;
@@ -481,7 +489,7 @@ private static void HandleHistogram(TraceEvent obj, CounterConfiguration configu
//Note quantiles can be empty.
IList quantiles = ParseQuantiles(quantilesText);
- CounterMetadata metadata = GetCounterMetadata(meterName, instrumentName, id);
+ CounterMetadata metadata = GetCounterMetadata(counterMetadataCache, meterName, instrumentName, id);
payload = new AggregatePercentilePayload(metadata, displayName: null, displayUnits: null, tags, count, sum, quantiles, obj.TimeStamp);
}
diff --git a/src/Microsoft.Diagnostics.Monitoring.EventPipe/Triggers/EventCounter/EventCounterTrigger.cs b/src/Microsoft.Diagnostics.Monitoring.EventPipe/Triggers/EventCounter/EventCounterTrigger.cs
index a1b96bf004..8f32b66522 100644
--- a/src/Microsoft.Diagnostics.Monitoring.EventPipe/Triggers/EventCounter/EventCounterTrigger.cs
+++ b/src/Microsoft.Diagnostics.Monitoring.EventPipe/Triggers/EventCounter/EventCounterTrigger.cs
@@ -32,7 +32,11 @@ internal sealed class EventCounterTrigger :
private readonly CounterFilter _filter;
private readonly EventCounterTriggerImpl _impl;
private readonly string _providerName;
- private CounterConfiguration _counterConfiguration;
+ private readonly CounterConfiguration _counterConfiguration;
+
+ // CONSIDER It is likely that we could expand the scope of this cache across multiple triggers, but
+ // currently each trigger creates its own session.
+ private CounterMetadataCache _counterMetadataCache;
public EventCounterTrigger(EventCounterTriggerSettings settings)
{
@@ -47,6 +51,7 @@ public EventCounterTrigger(EventCounterTriggerSettings settings)
_filter.AddFilter(settings.ProviderName, new string[] { settings.CounterName });
_counterConfiguration = new CounterConfiguration(_filter);
+ _counterMetadataCache = new();
_impl = new EventCounterTriggerImpl(settings);
@@ -61,7 +66,7 @@ public IReadOnlyDictionary> GetProviderEvent
public bool HasSatisfiedCondition(TraceEvent traceEvent)
{
// Filter to the counter of interest before forwarding to the implementation
- if (traceEvent.TryGetCounterPayload(_counterConfiguration, out ICounterPayload payload))
+ if (traceEvent.TryGetCounterPayload(_counterMetadataCache, _counterConfiguration, out ICounterPayload payload))
{
return _impl.HasSatisfiedCondition(payload);
}
diff --git a/src/Microsoft.Diagnostics.Monitoring.EventPipe/Triggers/SystemDiagnosticsMetricsTrigger/SystemDiagnosticsMetricsTrigger.cs b/src/Microsoft.Diagnostics.Monitoring.EventPipe/Triggers/SystemDiagnosticsMetricsTrigger/SystemDiagnosticsMetricsTrigger.cs
index d9e9a1267a..495b4678fc 100644
--- a/src/Microsoft.Diagnostics.Monitoring.EventPipe/Triggers/SystemDiagnosticsMetricsTrigger/SystemDiagnosticsMetricsTrigger.cs
+++ b/src/Microsoft.Diagnostics.Monitoring.EventPipe/Triggers/SystemDiagnosticsMetricsTrigger/SystemDiagnosticsMetricsTrigger.cs
@@ -31,6 +31,10 @@ internal sealed class SystemDiagnosticsMetricsTrigger :
private readonly string _sessionId;
private CounterConfiguration _counterConfiguration;
+ // CONSIDER It is likely that we could expand the scope of this cache across multiple triggers, but
+ // currently each trigger creates its own session.
+ private readonly CounterMetadataCache _counterMetadataCache;
+
public SystemDiagnosticsMetricsTrigger(SystemDiagnosticsMetricsTriggerSettings settings)
{
if (null == settings)
@@ -54,6 +58,7 @@ public SystemDiagnosticsMetricsTrigger(SystemDiagnosticsMetricsTriggerSettings s
_clientId = settings.ClientId;
_counterConfiguration = new CounterConfiguration(_filter) { SessionId = _sessionId, ClientId = _clientId };
+ _counterMetadataCache = new();
}
public IReadOnlyDictionary> GetProviderEventMap()
@@ -64,7 +69,7 @@ public IReadOnlyDictionary> GetProviderEvent
public bool HasSatisfiedCondition(TraceEvent traceEvent)
{
// Filter to the counter of interest before forwarding to the implementation
- if (traceEvent.TryGetCounterPayload(_counterConfiguration, out ICounterPayload payload))
+ if (traceEvent.TryGetCounterPayload(_counterMetadataCache, _counterConfiguration, out ICounterPayload payload))
{
return _impl.HasSatisfiedCondition(payload);
}
diff --git a/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClient/DiagnosticsClientExceptions.cs b/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClient/DiagnosticsClientExceptions.cs
index e9c2f15c42..c49984572e 100644
--- a/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClient/DiagnosticsClientExceptions.cs
+++ b/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsClient/DiagnosticsClientExceptions.cs
@@ -21,6 +21,7 @@ public UnsupportedProtocolException(string msg) : base(msg) { }
public class ServerNotAvailableException : DiagnosticsClientException
{
public ServerNotAvailableException(string msg) : base(msg) { }
+ public ServerNotAvailableException(string msg, Exception exception) : base(msg, exception) { }
}
// When the runtime responded with an error
diff --git a/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcTcpSocketEndPoint.cs b/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcTcpSocketEndPoint.cs
index 7aaae390f5..abf3141534 100644
--- a/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcTcpSocketEndPoint.cs
+++ b/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcTcpSocketEndPoint.cs
@@ -38,7 +38,7 @@ public IpcTcpSocketEndPoint(string endPoint)
{
ParseTcpIpEndPoint(endPoint, out string host, out int port);
EndPoint = CreateEndPoint(host, port);
- DualMode = string.CompareOrdinal(host, "*") == 0;
+ DualMode = host == "*";
}
public static implicit operator EndPoint(IpcTcpSocketEndPoint endPoint) => endPoint.EndPoint;
@@ -98,7 +98,7 @@ private static void ParseTcpIpEndPoint(string endPoint, out string host, out int
port = int.Parse(segments[1]);
}
- if (string.CompareOrdinal(host, "*") != 0)
+ if (host == "*")
{
if (!IPAddress.TryParse(host, out _))
{
@@ -122,7 +122,7 @@ private static IPEndPoint CreateEndPoint(string host, int port)
IPAddress ipAddress = null;
try
{
- if (string.CompareOrdinal(host, "*") == 0)
+ if (host == "*")
{
if (Socket.OSSupportsIPv6)
{
diff --git a/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcTransport.cs b/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcTransport.cs
index 7d4fb0a909..de15869988 100644
--- a/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcTransport.cs
+++ b/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsIpc/IpcTransport.cs
@@ -6,6 +6,7 @@
using System.IO;
using System.IO.Pipes;
using System.Linq;
+using System.Net.Sockets;
using System.Runtime.InteropServices;
using System.Security.Principal;
using System.Threading;
@@ -20,6 +21,7 @@ internal abstract class IpcEndpoint
///
/// The amount of time to block attempting to connect
/// A stream used for writing and reading data to and from the target .NET process
+ /// ServerNotAvailableException
public abstract Stream Connect(TimeSpan timeout);
///
@@ -29,6 +31,7 @@ internal abstract class IpcEndpoint
///
/// A task that completes with a stream used for writing and reading data to and from the target .NET process.
///
+ /// ServerNotAvailableException
public abstract Task ConnectAsync(CancellationToken token);
///
@@ -51,66 +54,81 @@ internal static class IpcEndpointHelper
{
public static Stream Connect(IpcEndpointConfig config, TimeSpan timeout)
{
- if (config.Transport == IpcEndpointConfig.TransportType.NamedPipe)
- {
- NamedPipeClientStream namedPipe = new(
- ".",
- config.Address,
- PipeDirection.InOut,
- PipeOptions.None,
- TokenImpersonationLevel.Impersonation);
- namedPipe.Connect((int)timeout.TotalMilliseconds);
- return namedPipe;
- }
- else if (config.Transport == IpcEndpointConfig.TransportType.UnixDomainSocket)
+ try
{
- IpcUnixDomainSocket socket = new();
- socket.Connect(new IpcUnixDomainSocketEndPoint(config.Address), timeout);
- return new ExposedSocketNetworkStream(socket, ownsSocket: true);
- }
+ if (config.Transport == IpcEndpointConfig.TransportType.NamedPipe)
+ {
+ NamedPipeClientStream namedPipe = new(
+ ".",
+ config.Address,
+ PipeDirection.InOut,
+ PipeOptions.None,
+ TokenImpersonationLevel.Impersonation);
+ namedPipe.Connect((int)timeout.TotalMilliseconds);
+ return namedPipe;
+ }
+ else if (config.Transport == IpcEndpointConfig.TransportType.UnixDomainSocket)
+ {
+ IpcUnixDomainSocket socket = new();
+ socket.Connect(new IpcUnixDomainSocketEndPoint(config.Address), timeout);
+ return new ExposedSocketNetworkStream(socket, ownsSocket: true);
+ }
#if DIAGNOSTICS_RUNTIME
- else if (config.Transport == IpcEndpointConfig.TransportType.TcpSocket)
- {
- var tcpClient = new TcpClient ();
- var endPoint = new IpcTcpSocketEndPoint(config.Address);
- tcpClient.Connect(endPoint.EndPoint);
- return tcpClient.GetStream();
- }
+ else if (config.Transport == IpcEndpointConfig.TransportType.TcpSocket)
+ {
+ var tcpClient = new TcpClient ();
+ var endPoint = new IpcTcpSocketEndPoint(config.Address);
+ tcpClient.Connect(endPoint.EndPoint);
+ return tcpClient.GetStream();
+ }
#endif
- else
+ else
+ {
+ throw new ArgumentException($"Unsupported IpcEndpointConfig transport type {config.Transport}");
+ }
+
+ }
+ catch (SocketException ex)
{
- throw new ArgumentException($"Unsupported IpcEndpointConfig transport type {config.Transport}");
+ throw new ServerNotAvailableException($"Unable to connect to the server. {ex.Message}", ex);
}
}
public static async Task ConnectAsync(IpcEndpointConfig config, CancellationToken token)
{
- if (config.Transport == IpcEndpointConfig.TransportType.NamedPipe)
- {
- NamedPipeClientStream namedPipe = new(
- ".",
- config.Address,
- PipeDirection.InOut,
- PipeOptions.Asynchronous,
- TokenImpersonationLevel.Impersonation);
-
- // Pass non-infinite timeout in order to cause internal connection algorithm
- // to check the CancellationToken periodically. Otherwise, if the named pipe
- // is waited using WaitNamedPipe with an infinite timeout, then the
- // CancellationToken cannot be observed.
- await namedPipe.ConnectAsync(int.MaxValue, token).ConfigureAwait(false);
-
- return namedPipe;
- }
- else if (config.Transport == IpcEndpointConfig.TransportType.UnixDomainSocket)
+ try
{
- IpcUnixDomainSocket socket = new();
- await socket.ConnectAsync(new IpcUnixDomainSocketEndPoint(config.Address), token).ConfigureAwait(false);
- return new ExposedSocketNetworkStream(socket, ownsSocket: true);
+ if (config.Transport == IpcEndpointConfig.TransportType.NamedPipe)
+ {
+ NamedPipeClientStream namedPipe = new(
+ ".",
+ config.Address,
+ PipeDirection.InOut,
+ PipeOptions.Asynchronous,
+ TokenImpersonationLevel.Impersonation);
+
+ // Pass non-infinite timeout in order to cause internal connection algorithm
+ // to check the CancellationToken periodically. Otherwise, if the named pipe
+ // is waited using WaitNamedPipe with an infinite timeout, then the
+ // CancellationToken cannot be observed.
+ await namedPipe.ConnectAsync(int.MaxValue, token).ConfigureAwait(false);
+
+ return namedPipe;
+ }
+ else if (config.Transport == IpcEndpointConfig.TransportType.UnixDomainSocket)
+ {
+ IpcUnixDomainSocket socket = new();
+ await socket.ConnectAsync(new IpcUnixDomainSocketEndPoint(config.Address), token).ConfigureAwait(false);
+ return new ExposedSocketNetworkStream(socket, ownsSocket: true);
+ }
+ else
+ {
+ throw new ArgumentException($"Unsupported IpcEndpointConfig transport type {config.Transport}");
+ }
}
- else
+ catch (SocketException ex)
{
- throw new ArgumentException($"Unsupported IpcEndpointConfig transport type {config.Transport}");
+ throw new ServerNotAvailableException($"Unable to connect to the server. {ex.Message}", ex);
}
}
}
@@ -221,10 +239,13 @@ internal class PidIpcEndpoint : IpcEndpoint
{
public static string IpcRootPath { get; } = RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ? @"\\.\pipe\" : Path.GetTempPath();
public static string DiagnosticsPortPattern { get; } = RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ? @"^dotnet-diagnostic-(\d+)$" : @"^dotnet-diagnostic-(\d+)-(\d+)-socket$";
-
+ // Format strings as private const members
+ private const string _defaultAddressFormatWindows = "dotnet-diagnostic-{0}";
+ private const string _dsrouterAddressFormatWindows = "dotnet-diagnostic-dsrouter-{0}";
+ private const string _defaultAddressFormatNonWindows = "dotnet-diagnostic-{0}-{1}-socket";
+ private const string _dsrouterAddressFormatNonWindows = "dotnet-diagnostic-dsrouter-{0}-{1}-socket";
private int _pid;
private IpcEndpointConfig _config;
-
///
/// Creates a reference to a .NET process's IPC Transport
/// using the default rules for a given pid
@@ -271,11 +292,11 @@ private static bool TryGetDefaultAddress(int pid, out string defaultAddress)
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
- defaultAddress = $"dotnet-diagnostic-{pid}";
+ defaultAddress = string.Format(_defaultAddressFormatWindows, pid);
try
{
- string dsrouterAddress = Directory.GetFiles(IpcRootPath, $"dotnet-diagnostic-dsrouter-{pid}").FirstOrDefault();
+ string dsrouterAddress = Directory.GetFiles(IpcRootPath, string.Format(_dsrouterAddressFormatWindows, pid)).FirstOrDefault();
if (!string.IsNullOrEmpty(dsrouterAddress))
{
defaultAddress = dsrouterAddress;
@@ -287,11 +308,11 @@ private static bool TryGetDefaultAddress(int pid, out string defaultAddress)
{
try
{
- defaultAddress = Directory.GetFiles(IpcRootPath, $"dotnet-diagnostic-{pid}-*-socket") // Try best match.
+ defaultAddress = Directory.GetFiles(IpcRootPath, string.Format(_defaultAddressFormatNonWindows, pid, "*")) // Try best match.
.OrderByDescending(f => new FileInfo(f).LastWriteTime)
.FirstOrDefault();
- string dsrouterAddress = Directory.GetFiles(IpcRootPath, $"dotnet-diagnostic-dsrouter-{pid}-*-socket") // Try best match.
+ string dsrouterAddress = Directory.GetFiles(IpcRootPath, string.Format(_dsrouterAddressFormatNonWindows, pid, "*")) // Try best match.
.OrderByDescending(f => new FileInfo(f).LastWriteTime)
.FirstOrDefault();
@@ -332,8 +353,15 @@ public static string GetDefaultAddress(int pid)
string msg = $"Unable to connect to Process {pid}.";
if (!RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
+ int total_length = IpcRootPath.Length + string.Format(_defaultAddressFormatNonWindows, pid, "##########").Length;
+ if (total_length > 108) // This isn't perfect as we don't know the disambiguation key length. However it should catch most cases.
+ {
+ msg += "The total length of the diagnostic socket path may exceed 108 characters. " +
+ "Try setting the TMPDIR environment variable to a shorter path";
+ }
msg += $" Please verify that {IpcRootPath} is writable by the current user. "
+ "If the target process has environment variable TMPDIR set, please set TMPDIR to the same directory. "
+ + "Please also ensure that the target process has {TMPDIR}/dotnet-diagnostic-{pid}-{disambiguation_key}-socket shorter than 108 characters. "
+ "Please see https://aka.ms/dotnet-diagnostics-port for more information";
}
throw new ServerNotAvailableException(msg);
@@ -349,7 +377,7 @@ public static bool IsDefaultAddressDSRouter(int pid, string address)
address = address.Substring(IpcRootPath.Length);
}
- string dsrouterAddress = $"dotnet-diagnostic-dsrouter-{pid}";
+ string dsrouterAddress = string.Format(_dsrouterAddressFormatWindows, pid);
return address.StartsWith(dsrouterAddress, StringComparison.OrdinalIgnoreCase);
}
diff --git a/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsServerRouter/DiagnosticsServerRouterFactory.cs b/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsServerRouter/DiagnosticsServerRouterFactory.cs
index c79bd3c15e..e7287c0fd9 100644
--- a/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsServerRouter/DiagnosticsServerRouterFactory.cs
+++ b/src/Microsoft.Diagnostics.NETCore.Client/DiagnosticsServerRouter/DiagnosticsServerRouterFactory.cs
@@ -356,9 +356,11 @@ public static WebSocketServerRouterFactory CreateDefaultInstance(string webSocke
public WebSocketServerRouterFactory(string webSocketURL, int runtimeTimeoutMs, ILogger logger) : base(runtimeTimeoutMs, logger)
{
- _webSocketURL = string.IsNullOrEmpty(webSocketURL) ? "ws://127.0.0.1:8088/diagnostics" : webSocketURL;
+ Debug.Assert(!string.IsNullOrEmpty(webSocketURL));
- _webSocketServer = new ReversedDiagnosticsServer(_webSocketURL, ReversedDiagnosticsServer.Kind.WebSocket);
+ _webSocketURL = webSocketURL;
+
+ _webSocketServer = new ReversedDiagnosticsServer(_webSocketURL, ReversedDiagnosticsServer.Kind.WebSocket, TimeSpan.FromMilliseconds(750));
_webSocketServer.TransportCallback = this;
}
diff --git a/src/Microsoft.Diagnostics.NETCore.Client/Microsoft.Diagnostics.NETCore.Client.csproj b/src/Microsoft.Diagnostics.NETCore.Client/Microsoft.Diagnostics.NETCore.Client.csproj
index 1f5414270e..c006330b90 100644
--- a/src/Microsoft.Diagnostics.NETCore.Client/Microsoft.Diagnostics.NETCore.Client.csproj
+++ b/src/Microsoft.Diagnostics.NETCore.Client/Microsoft.Diagnostics.NETCore.Client.csproj
@@ -13,8 +13,11 @@
truetruefalse
- true
-
+ true
+
+
diff --git a/src/Microsoft.Diagnostics.NETCore.Client/ReversedServer/ReversedDiagnosticsServer.cs b/src/Microsoft.Diagnostics.NETCore.Client/ReversedServer/ReversedDiagnosticsServer.cs
index b0ea799412..72c690c49e 100644
--- a/src/Microsoft.Diagnostics.NETCore.Client/ReversedServer/ReversedDiagnosticsServer.cs
+++ b/src/Microsoft.Diagnostics.NETCore.Client/ReversedServer/ReversedDiagnosticsServer.cs
@@ -21,7 +21,7 @@ internal sealed class ReversedDiagnosticsServer : IAsyncDisposable
{
// The amount of time to allow parsing of the advertise data before cancelling. This allows the server to
// remain responsive in case the advertise data is incomplete and the stream is not closed.
- private static readonly TimeSpan ParseAdvertiseTimeout = TimeSpan.FromMilliseconds(250);
+ private readonly TimeSpan ParseAdvertiseTimeout;
private readonly CancellationTokenSource _disposalSource = new();
private readonly HandleableCollection _endpointInfos = new();
@@ -52,6 +52,7 @@ public enum Kind
public ReversedDiagnosticsServer(string address)
{
_address = address;
+ ParseAdvertiseTimeout = TimeSpan.FromMilliseconds(250);
}
///
@@ -69,12 +70,38 @@ public ReversedDiagnosticsServer(string address)
/// Otherwise if kind is TcpIp as a supported protocol for ReversedDiagnosticServer. When Kind is Tcp, address will
/// be analyzed and if on format host:port, ReversedDiagnosticServer will try to bind
/// a TcpIp listener to host and port, otherwise it will use a Unix domain socket or a Windows named pipe.
- ///
///
public ReversedDiagnosticsServer(string address, Kind kind)
{
_address = address;
_kind = kind;
+ ParseAdvertiseTimeout = TimeSpan.FromMilliseconds(250);
+ }
+
+ ///
+ /// Constructs the instance with an endpoint bound
+ /// to the location specified by .
+ ///
+ ///
+ /// The server endpoint.
+ /// On Windows, this can be a full pipe path or the name without the "\\.\pipe\" prefix.
+ /// On all other systems, this must be the full file path of the socket.
+ ///
+ ///
+ /// If kind is WebSocket, start a Kestrel web server.
+ /// Otherwise if kind is TcpIp as a supported protocol for ReversedDiagnosticServer. When Kind is Tcp, address will
+ /// be analyzed and if on format host:port, ReversedDiagnosticServer will try to bind
+ /// a TcpIp listener to host and port, otherwise it will use a Unix domain socket or a Windows named pipe.
+ ///
+ ///
+ /// The amount of time to allow parsing of the advertise data before cancelling. This allows the server to
+ /// remain responsive in case the advertise data is incomplete and the stream is not closed.
+ ///
+ public ReversedDiagnosticsServer(string address, Kind kind, TimeSpan timeout)
+ {
+ _address = address;
+ _kind = kind;
+ ParseAdvertiseTimeout = timeout;
}
public async ValueTask DisposeAsync()
diff --git a/src/Microsoft.Diagnostics.TestHelpers/SdkPrebuiltDebuggeeCompiler.cs b/src/Microsoft.Diagnostics.TestHelpers/SdkPrebuiltDebuggeeCompiler.cs
index d397dd809e..ebc4105d43 100644
--- a/src/Microsoft.Diagnostics.TestHelpers/SdkPrebuiltDebuggeeCompiler.cs
+++ b/src/Microsoft.Diagnostics.TestHelpers/SdkPrebuiltDebuggeeCompiler.cs
@@ -15,6 +15,15 @@ public class SdkPrebuiltDebuggeeCompiler : IDebuggeeCompiler
public SdkPrebuiltDebuggeeCompiler(TestConfiguration config, string debuggeeName)
{
+ if (string.IsNullOrEmpty(config.TargetConfiguration))
+ {
+ throw new System.ArgumentException("TargetConfiguration must be set in the TestConfiguration");
+ }
+ if (string.IsNullOrEmpty(config.BuildProjectFramework))
+ {
+ throw new System.ArgumentException("BuildProjectFramework must be set in the TestConfiguration");
+ }
+
// The layout is how the current .NET Core SDK layouts the binaries out:
// Source Path: //[]
// Binary Path: /bin///
diff --git a/src/Microsoft.Diagnostics.TestHelpers/TestConfiguration.cs b/src/Microsoft.Diagnostics.TestHelpers/TestConfiguration.cs
index 2ee0c2ddc5..01821e3244 100644
--- a/src/Microsoft.Diagnostics.TestHelpers/TestConfiguration.cs
+++ b/src/Microsoft.Diagnostics.TestHelpers/TestConfiguration.cs
@@ -79,7 +79,8 @@ private void ParseConfigFile(string path)
["IsAlpine"] = OS.IsAlpine.ToString().ToLowerInvariant(),
["TargetRid"] = GetRid(),
["TargetArchitecture"] = OS.TargetArchitecture.ToString().ToLowerInvariant(),
- ["NuGetPackageCacheDir"] = nugetPackages
+ ["NuGetPackageCacheDir"] = nugetPackages,
+ ["TestCDAC"] = Environment.GetEnvironmentVariable("SOS_TEST_CDAC")
};
if (OS.Kind == OSKind.Windows)
{
@@ -461,6 +462,10 @@ private string GetStringViewWithVersion(string version)
{
sb.Append(".singlefile");
}
+ if (TestCDAC)
+ {
+ sb.Append(".cdac");
+ }
if (!string.IsNullOrEmpty(version))
{
sb.Append('.');
@@ -555,6 +560,14 @@ public bool IsDesktop
get { return TestProduct.Equals("desktop"); }
}
+ ///
+ /// Returns true if test should use the cDAC.
+ ///
+ public bool TestCDAC
+ {
+ get { return string.Equals(GetValue("TestCDAC"), "true", StringComparison.InvariantCultureIgnoreCase); }
+ }
+
///
/// The test runner script directory
///
diff --git a/src/Microsoft.Diagnostics.TestHelpers/TestHost/TestDataWriter.cs b/src/Microsoft.Diagnostics.TestHelpers/TestHost/TestDataWriter.cs
index 700b028659..7612d3d07c 100644
--- a/src/Microsoft.Diagnostics.TestHelpers/TestHost/TestDataWriter.cs
+++ b/src/Microsoft.Diagnostics.TestHelpers/TestHost/TestDataWriter.cs
@@ -84,7 +84,7 @@ public void Build(IServiceProvider services)
{
XElement runtimeElement = new("Runtime");
runtimesElement.Add(runtimeElement);
- AddMembers(runtimeElement, typeof(IRuntime), runtime, nameof(IRuntime.GetDacFilePath), nameof(IRuntime.GetDbiFilePath));
+ AddMembers(runtimeElement, typeof(IRuntime), runtime, nameof(IRuntime.GetDacFilePath), nameof(IRuntime.GetCDacFilePath), nameof(IRuntime.GetDbiFilePath));
XElement runtimeModuleElement = new("RuntimeModule");
runtimeElement.Add(runtimeModuleElement);
diff --git a/src/Microsoft.Diagnostics.TestHelpers/TestHost/TestDump.cs b/src/Microsoft.Diagnostics.TestHelpers/TestHost/TestDump.cs
index 271ca304d9..96a136d98c 100644
--- a/src/Microsoft.Diagnostics.TestHelpers/TestHost/TestDump.cs
+++ b/src/Microsoft.Diagnostics.TestHelpers/TestHost/TestDump.cs
@@ -8,7 +8,7 @@
namespace Microsoft.Diagnostics.TestHelpers
{
- public class TestDump : TestHost, ISettingsService
+ public class TestDump : TestHost
{
private readonly Host _host;
private readonly ContextService _contextService;
@@ -26,7 +26,6 @@ public TestDump(TestConfiguration config)
_contextService = new(_host);
serviceContainer.AddService(_contextService);
- serviceContainer.AddService(this);
_commandService = new();
serviceContainer.AddService(_commandService);
@@ -53,11 +52,5 @@ protected override ITarget GetTarget()
_symbolService.AddDirectoryPath(Path.GetDirectoryName(DumpFile));
return _dumpTargetFactory.OpenDump(DumpFile);
}
-
- #region ISettingsService
-
- public bool DacSignatureVerificationEnabled { get; set; }
-
- #endregion
}
}
diff --git a/src/Microsoft.Diagnostics.WebSocketServer/WebSocketServerImpl.cs b/src/Microsoft.Diagnostics.WebSocketServer/WebSocketServerImpl.cs
index f34e5907ce..e7e53cf986 100644
--- a/src/Microsoft.Diagnostics.WebSocketServer/WebSocketServerImpl.cs
+++ b/src/Microsoft.Diagnostics.WebSocketServer/WebSocketServerImpl.cs
@@ -40,9 +40,18 @@ public async Task StartServer(string endpoint, CancellationToken cancellationTok
ParseWebSocketURL(endpoint, out Uri uri);
+ string scheme = uri.Scheme switch
+ {
+ "ws" => "http",
+ "http" => "http",
+ "wss" => "https",
+ "https" => "https",
+ _ => throw new ArgumentException(string.Format("Unsupported Uri schema, \"{0}\"", uri.Scheme))
+ };
+
EmbeddedWebSocketServer.Options options = new()
{
- Scheme = uri.Scheme,
+ Scheme = scheme,
Host = uri.Host,
Port = uri.Port.ToString(),
Path = uri.PathAndQuery,
diff --git a/src/SOS/CMakeLists.txt b/src/SOS/CMakeLists.txt
index e1de7aff33..901a26e428 100644
--- a/src/SOS/CMakeLists.txt
+++ b/src/SOS/CMakeLists.txt
@@ -25,5 +25,4 @@ add_compile_definitions(SOS_INCLUDE)
add_compile_definitions(DISABLE_CONTRACTS)
add_subdirectory(extensions)
-add_subdirectory(SOS.Extensions)
add_subdirectory(Strike)
diff --git a/src/SOS/SOS.Extensions/CMakeLists.txt b/src/SOS/SOS.Extensions/CMakeLists.txt
deleted file mode 100644
index b6aa9bf8c3..0000000000
--- a/src/SOS/SOS.Extensions/CMakeLists.txt
+++ /dev/null
@@ -1,21 +0,0 @@
-project(SOS.Extensions)
-
-if(NOT ${NUGET_PACKAGES} STREQUAL "")
- set(DIASYMREADER_ARCH ${CLR_CMAKE_TARGET_ARCH})
-
- if(NOT(CLR_CMAKE_TARGET_ARCH STREQUAL CLR_CMAKE_HOST_ARCH))
- set(DIASYMREADER_ARCH ${CLR_CMAKE_HOST_ARCH})
- endif()
-
- if (DIASYMREADER_ARCH STREQUAL x64)
- set(DIASYMREADER_ARCH amd64)
- endif()
-
- install(FILES ${NUGET_PACKAGES}/microsoft.diasymreader.native/17.10.0-beta1.24272.1/runtimes/win/native/Microsoft.DiaSymReader.Native.${DIASYMREADER_ARCH}.dll DESTINATION . )
-endif()
-
-if(NOT ${CLR_MANAGED_BINARY_DIR} STREQUAL "")
- set(MANAGED_BINDIR ${CLR_MANAGED_BINARY_DIR}/SOS.Extensions/${CLR_BUILD_TYPE}/netstandard2.0/publish)
- file(GLOB installfiles ${MANAGED_BINDIR}/*.dll ${MANAGED_BINDIR}/*.pdb)
- install(FILES ${installfiles} DESTINATION . )
-endif()
diff --git a/src/SOS/SOS.Extensions/DebuggerServices.cs b/src/SOS/SOS.Extensions/DebuggerServices.cs
index 380a87e546..4b1ab060a2 100644
--- a/src/SOS/SOS.Extensions/DebuggerServices.cs
+++ b/src/SOS/SOS.Extensions/DebuggerServices.cs
@@ -259,11 +259,11 @@ public HResult GetThreadIdsByIndex(uint start, uint count, uint[] ids, uint[] sy
}
}
- public HResult GetThreadContext(uint threadId, uint contextFlags, uint contextSize, byte[] context)
+ public HResult GetThreadContext(uint threadId, uint contextFlags, byte[] context)
{
fixed (byte* contextPtr = context)
{
- return VTable.GetThreadContextBySystemId(Self, threadId, contextFlags, contextSize, contextPtr);
+ return VTable.GetThreadContextBySystemId(Self, threadId, contextFlags, (uint)context.Length, contextPtr);
}
}
diff --git a/src/SOS/SOS.Extensions/HostServices.cs b/src/SOS/SOS.Extensions/HostServices.cs
index e8155b7be6..04b401c647 100644
--- a/src/SOS/SOS.Extensions/HostServices.cs
+++ b/src/SOS/SOS.Extensions/HostServices.cs
@@ -22,7 +22,7 @@ namespace SOS.Extensions
///
/// The extension services Wrapper the native hosts are given
///
- public sealed unsafe class HostServices : COMCallableIUnknown, SOSLibrary.ISOSModule, ISettingsService
+ public sealed unsafe class HostServices : COMCallableIUnknown, SOSLibrary.ISOSModule
{
private static readonly Guid IID_IHostServices = new("27B2CB8D-BDEE-4CBD-B6EF-75880D76D46F");
@@ -130,7 +130,7 @@ private HostServices(string extensionPath, IntPtr extensionsLibrary)
SOSPath = Path.GetDirectoryName(extensionPath);
SOSHandle = extensionsLibrary;
- _host = new Host(RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ? HostType.DbgEng : HostType.Lldb);
+ _host = new HostForHostServices(this);
_commandService = new CommandService(RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ? ">!sos" : null);
_host.ServiceManager.NotifyExtensionLoad.Register(_commandService.AddCommands);
@@ -228,7 +228,6 @@ private int RegisterDebuggerServices(
// Add all the global services to the global service container
serviceContainer.AddService(this);
- serviceContainer.AddService(this);
serviceContainer.AddService(DebuggerServices);
serviceContainer.AddService(_commandService);
serviceContainer.AddService(_symbolService);
@@ -357,6 +356,10 @@ private int DispatchCommand(
{
return HResult.E_INVALIDARG;
}
+ if (_commandService is null)
+ {
+ return HResult.E_FAIL;
+ }
try
{
_commandService.Execute(commandName, commandArguments, string.Equals(commandName, "help", StringComparison.OrdinalIgnoreCase) ? _contextServiceFromDebuggerServices.Services : _servicesWithManagedOnlyFilter);
@@ -414,23 +417,38 @@ private void Uninitialize(
#endregion
- #region ISettingsService
+ #region HostForHostServices
- public bool DacSignatureVerificationEnabled
+ internal sealed class HostForHostServices : Host
{
- get
+ private readonly HostServices _hostServices;
+
+ public HostForHostServices(HostServices hostServices)
+ : base(RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ? HostType.DbgEng : HostType.Lldb)
{
- HResult hr = DebuggerServices.GetDacSignatureVerificationSettings(out bool value);
- if (hr.IsOK)
- {
- return value;
- }
- // Return true (verify DAC signature) if any errors. Secure by default.
- return true;
+ _hostServices = hostServices;
}
- set
+
+ public override bool DacSignatureVerificationEnabled
{
- throw new NotSupportedException("Changing the DacSignatureVerificationEnabled setting is not supported.");
+ get
+ {
+ if (_hostServices.DebuggerServices is null)
+ {
+ throw new InvalidOperationException("DacSignatureVerificationEnabled called too soon in initialization");
+ }
+ HResult hr = _hostServices.DebuggerServices.GetDacSignatureVerificationSettings(out bool value);
+ if (hr.IsOK)
+ {
+ return value;
+ }
+ // Return true (verify DAC signature) if any errors. Secure by default.
+ return true;
+ }
+ set
+ {
+ throw new NotSupportedException("Changing the DacSignatureVerificationEnabled setting is not supported.");
+ }
}
}
diff --git a/src/SOS/SOS.Extensions/SOS.Extensions.csproj b/src/SOS/SOS.Extensions/SOS.Extensions.csproj
index a1ecf2ce33..f61bf5d269 100644
--- a/src/SOS/SOS.Extensions/SOS.Extensions.csproj
+++ b/src/SOS/SOS.Extensions/SOS.Extensions.csproj
@@ -12,7 +12,6 @@
-
@@ -23,4 +22,15 @@
+
+
+
+
+
+
+
+
+
diff --git a/src/SOS/SOS.Extensions/ThreadServiceFromDebuggerServices.cs b/src/SOS/SOS.Extensions/ThreadServiceFromDebuggerServices.cs
index 694f61dac4..2a8c5da98b 100644
--- a/src/SOS/SOS.Extensions/ThreadServiceFromDebuggerServices.cs
+++ b/src/SOS/SOS.Extensions/ThreadServiceFromDebuggerServices.cs
@@ -52,7 +52,7 @@ protected override IEnumerable GetThreadsInner()
protected override bool GetThreadContext(uint threadId, uint contextFlags, byte[] context)
{
- return _debuggerServices.GetThreadContext(threadId, contextFlags, (uint)context.Length, context).IsOK;
+ return _debuggerServices.GetThreadContext(threadId, contextFlags, context).IsOK;
}
protected override ulong GetThreadTeb(uint threadId)
diff --git a/src/SOS/SOS.Hosting/DataTargetWrapper.cs b/src/SOS/SOS.Hosting/DataTargetWrapper.cs
index 3feade5d10..0a01ceea22 100644
--- a/src/SOS/SOS.Hosting/DataTargetWrapper.cs
+++ b/src/SOS/SOS.Hosting/DataTargetWrapper.cs
@@ -6,6 +6,7 @@
using System.Linq;
using System.Runtime.InteropServices;
using Microsoft.Diagnostics.DebugServices;
+using Microsoft.Diagnostics.Runtime;
using Microsoft.Diagnostics.Runtime.Utilities;
using SOS.Hosting.DbgEng.Interop;
@@ -17,6 +18,7 @@ internal sealed unsafe class DataTargetWrapper : COMCallableIUnknown
private static readonly Guid IID_ICLRDataTarget4 = new("E799DC06-E099-4713-BDD9-906D3CC02CF2");
private static readonly Guid IID_ICLRMetadataLocator = new("aa8fa804-bc05-4642-b2c5-c353ed22fc63");
private static readonly Guid IID_ICLRRuntimeLocator = new("b760bf44-9377-4597-8be7-58083bdc5146");
+ private static readonly Guid IID_ICLRContractLocator = new("17d5b8c6-34a9-407f-af4f-a930201d4e02");
// For ClrMD's magic hand shake
private const ulong MagicCallbackConstant = 0x43;
@@ -67,6 +69,10 @@ public DataTargetWrapper(IServiceProvider services, IRuntime runtime)
builder.AddMethod(new GetRuntimeBaseDelegate(GetRuntimeBase));
builder.Complete();
+ builder = AddInterface(IID_ICLRContractLocator, false);
+ builder.AddMethod(new GetContractDescriptorDelegate(GetContractDescriptor));
+ builder.Complete();
+
AddRef();
}
@@ -350,6 +356,28 @@ private int GetRuntimeBase(
#endregion
+ #region ICLRContractLocator
+
+ private int GetContractDescriptor(
+ IntPtr self,
+ out ulong address)
+ {
+ address = 0;
+ ClrInfo clrInfo = _runtime.Services.GetService();
+ if (clrInfo is null)
+ {
+ return HResult.E_FAIL;
+ }
+ address = clrInfo.ContractDescriptorAddress;
+ if (address == 0)
+ {
+ return HResult.E_FAIL;
+ }
+ return HResult.S_OK;
+ }
+
+ #endregion
+
#region ICLRDataTarget delegates
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
@@ -485,5 +513,14 @@ private delegate int GetRuntimeBaseDelegate(
[Out] out ulong address);
#endregion
+
+ #region ICLRContractLocator delegate
+
+ [UnmanagedFunctionPointer(CallingConvention.Winapi)]
+ private delegate int GetContractDescriptorDelegate(
+ [In] IntPtr self,
+ [Out] out ulong address);
+
+ #endregion
}
}
diff --git a/src/SOS/SOS.Hosting/DbgEng/DebugClient.cs b/src/SOS/SOS.Hosting/DbgEng/DebugClient.cs
index f6a26520b1..35ef20cd8f 100644
--- a/src/SOS/SOS.Hosting/DbgEng/DebugClient.cs
+++ b/src/SOS/SOS.Hosting/DbgEng/DebugClient.cs
@@ -274,12 +274,12 @@ private delegate int ExitDispatchDelegate(
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int CreateClientDelegate(
IntPtr self,
- [Out][MarshalAs(UnmanagedType.Interface)] IntPtr Client); // out IDebugClient
+ [Out] IntPtr Client); // out IDebugClient
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetInputCallbacksDelegate(
IntPtr self,
- [Out][MarshalAs(UnmanagedType.Interface)] IntPtr Callbacks); // out IDebugInputCallbacks
+ [Out] IntPtr Callbacks); // out IDebugInputCallbacks
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int SetInputCallbacksDelegate(
diff --git a/src/SOS/SOS.Hosting/DbgEng/DebugControl.cs b/src/SOS/SOS.Hosting/DbgEng/DebugControl.cs
index 097e8c1ac7..faa1ec8f93 100644
--- a/src/SOS/SOS.Hosting/DbgEng/DebugControl.cs
+++ b/src/SOS/SOS.Hosting/DbgEng/DebugControl.cs
@@ -158,7 +158,7 @@ private delegate int GetLogFileDelegate(
[Out][MarshalAs(UnmanagedType.LPStr)] StringBuilder Buffer,
[In] int BufferSize,
[Out] uint* FileSize,
- [Out][MarshalAs(UnmanagedType.Bool)] bool* Append);
+ [Out] bool* Append);
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int OpenLogFileDelegate(
@@ -554,13 +554,13 @@ private delegate int GetNumberBreakpointsDelegate(
private delegate int GetBreakpointByIndexDelegate(
IntPtr self,
[In] uint Index,
- [Out][MarshalAs(UnmanagedType.Interface)] IntPtr bp); // out IDebugBreakpoint
+ [Out] IntPtr bp); // out IDebugBreakpoint
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetBreakpointByIdDelegate(
IntPtr self,
[In] uint Id,
- [Out][MarshalAs(UnmanagedType.Interface)] IntPtr bp); // out IDebugBreakpoint
+ [Out] IntPtr bp); // out IDebugBreakpoint
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetBreakpointParametersDelegate(
@@ -575,7 +575,7 @@ private delegate int AddBreakpointDelegate(
IntPtr self,
[In] DEBUG_BREAKPOINT_TYPE Type,
[In] uint DesiredId,
- [Out][MarshalAs(UnmanagedType.Interface)] IntPtr bp); // out IDebugBreakpoint
+ [Out] IntPtr bp); // out IDebugBreakpoint
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int RemoveBreakpointDelegate(
diff --git a/src/SOS/SOS.Hosting/DbgEng/DebugSymbols.cs b/src/SOS/SOS.Hosting/DbgEng/DebugSymbols.cs
index e8da6aacf4..63e2ec43d7 100644
--- a/src/SOS/SOS.Hosting/DbgEng/DebugSymbols.cs
+++ b/src/SOS/SOS.Hosting/DbgEng/DebugSymbols.cs
@@ -412,12 +412,12 @@ private delegate int GetScopeSymbolGroupDelegate(
IntPtr self,
[In] DEBUG_SCOPE_GROUP Flags,
[In][MarshalAs(UnmanagedType.Interface)] IDebugSymbolGroup Update,
- [Out][MarshalAs(UnmanagedType.Interface)] IntPtr Symbols); // out IDebugSymbolGroup
+ [Out] IntPtr Symbols); // out IDebugSymbolGroup
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int CreateSymbolGroupDelegate(
IntPtr self,
- [Out][MarshalAs(UnmanagedType.Interface)] IntPtr Group); // out IDebugSymbolGroup
+ [Out] IntPtr Group); // out IDebugSymbolGroup
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int StartSymbolMatchDelegate(
@@ -682,12 +682,12 @@ private delegate int GetScopeSymbolGroup2Delegate(
IntPtr self,
[In] DEBUG_SCOPE_GROUP Flags,
[In][MarshalAs(UnmanagedType.Interface)] IDebugSymbolGroup2 Update,
- [Out][MarshalAs(UnmanagedType.Interface)] IntPtr Symbols); // out IDebugSymbolGroup2
+ [Out] IntPtr Symbols); // out IDebugSymbolGroup2
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int CreateSymbolGroup2Delegate(
IntPtr self,
- [Out][MarshalAs(UnmanagedType.Interface)] IntPtr Group); // out IDebugSymbolGroup2
+ [Out] IntPtr Group); // out IDebugSymbolGroup2
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int StartSymbolMatchWideDelegate(
@@ -975,7 +975,7 @@ [Out] DEBUG_MODULE_AND_ID* Id
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int RemoveSyntheticSymbolDelegate(
IntPtr self,
- [In][MarshalAs(UnmanagedType.LPStruct)] DEBUG_MODULE_AND_ID Id
+ [In] DEBUG_MODULE_AND_ID* Id
);
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
@@ -1020,14 +1020,14 @@ [Out] DEBUG_MODULE_AND_ID* Id
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetSymbolEntryInformationDelegate(
IntPtr self,
- [In][MarshalAs(UnmanagedType.LPStruct)] DEBUG_MODULE_AND_ID Id,
+ [In] DEBUG_MODULE_AND_ID* Id,
[Out] DEBUG_SYMBOL_ENTRY* Info
);
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetSymbolEntryStringDelegate(
IntPtr self,
- [In][MarshalAs(UnmanagedType.LPStruct)] DEBUG_MODULE_AND_ID Id,
+ [In] DEBUG_MODULE_AND_ID* Id,
[In] uint Which,
[Out][MarshalAs(UnmanagedType.LPStr)] StringBuilder Buffer,
[In] int BufferSize,
@@ -1037,7 +1037,7 @@ [Out] uint* StringSize
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetSymbolEntryStringWideDelegate(
IntPtr self,
- [In][MarshalAs(UnmanagedType.LPStruct)] DEBUG_MODULE_AND_ID Id,
+ [In] DEBUG_MODULE_AND_ID* Id,
[In] uint Which,
[Out][MarshalAs(UnmanagedType.LPWStr)] StringBuilder Buffer,
[In] int BufferSize,
@@ -1047,7 +1047,7 @@ [Out] uint* StringSize
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetSymbolEntryOffsetRegionsDelegate(
IntPtr self,
- [In][MarshalAs(UnmanagedType.LPStruct)] DEBUG_MODULE_AND_ID Id,
+ [In] DEBUG_MODULE_AND_ID* Id,
[In] uint Flags,
[Out] DEBUG_OFFSET_REGION* Regions,
[In] uint RegionsCount,
@@ -1057,7 +1057,7 @@ [Out] uint* RegionsAvail
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetSymbolEntryBySymbolEntryDelegate(
IntPtr self,
- [In][MarshalAs(UnmanagedType.LPStruct)] DEBUG_MODULE_AND_ID FromId,
+ [In] DEBUG_MODULE_AND_ID* FromId,
[In] uint Flags,
[Out] DEBUG_MODULE_AND_ID* ToId
);
@@ -1097,7 +1097,7 @@ [Out] uint* EntriesAvail
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetSourceEntryStringDelegate(
IntPtr self,
- [In][MarshalAs(UnmanagedType.LPStruct)] DEBUG_SYMBOL_SOURCE_ENTRY Entry,
+ [In] DEBUG_SYMBOL_SOURCE_ENTRY Entry,
[In] uint Which,
[Out][MarshalAs(UnmanagedType.LPStr)] StringBuilder Buffer,
[In] int BufferSize,
@@ -1107,7 +1107,7 @@ [Out] uint* StringSize
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetSourceEntryStringWideDelegate(
IntPtr self,
- [In][MarshalAs(UnmanagedType.LPStruct)] DEBUG_SYMBOL_SOURCE_ENTRY Entry,
+ [In] DEBUG_SYMBOL_SOURCE_ENTRY Entry,
[In] uint Which,
[Out][MarshalAs(UnmanagedType.LPWStr)] StringBuilder Buffer,
[In] int BufferSize,
@@ -1117,7 +1117,7 @@ [Out] uint* StringSize
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetSourceEntryOffsetRegionsDelegate(
IntPtr self,
- [In][MarshalAs(UnmanagedType.LPStruct)] DEBUG_SYMBOL_SOURCE_ENTRY Entry,
+ [In] DEBUG_SYMBOL_SOURCE_ENTRY* Entry,
[In] uint Flags,
[Out] DEBUG_OFFSET_REGION* Regions,
[In] uint RegionsCount,
@@ -1127,7 +1127,7 @@ [Out] uint* RegionsAvail
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetSourceEntryBySourceEntryDelegate(
IntPtr self,
- [In][MarshalAs(UnmanagedType.LPStruct)] DEBUG_SYMBOL_SOURCE_ENTRY FromEntry,
+ [In] DEBUG_SYMBOL_SOURCE_ENTRY* FromEntry,
[In] uint Flags,
[Out] DEBUG_SYMBOL_SOURCE_ENTRY* ToEntry
);
diff --git a/src/SOS/SOS.Hosting/RuntimeWrapper.cs b/src/SOS/SOS.Hosting/RuntimeWrapper.cs
index a2c64cfd8a..bfca28db91 100644
--- a/src/SOS/SOS.Hosting/RuntimeWrapper.cs
+++ b/src/SOS/SOS.Hosting/RuntimeWrapper.cs
@@ -9,6 +9,7 @@
using Microsoft.Diagnostics.DebugServices;
using Microsoft.Diagnostics.Runtime;
using Microsoft.Diagnostics.Runtime.Utilities;
+using Microsoft.FileFormats.ELF;
using SOS.Hosting.DbgEng.Interop;
namespace SOS.Hosting
@@ -28,6 +29,22 @@ private enum RuntimeConfiguration
Unknown = 4
}
+ ///
+ /// Flags to GetClrDataProcess when creating the DAC instance
+ ///
+ private enum ClrDataProcessFlags
+ {
+ ///
+ /// No flags
+ ///
+ None,
+
+ ///
+ /// Use the cdac if available and enabled by global setting
+ ///
+ UseCDac
+ }
+
public static Guid IID_IXCLRDataProcess = new("5c552ab6-fc09-4cb3-8e36-22fa03c798b7");
public static Guid IID_ICorDebugProcess = new("3d6f5f64-7538-11d3-8d5b-00104b35e7ef");
private static readonly Guid IID_IRuntime = new("A5F152B9-BA78-4512-9228-5091A4CB7E35");
@@ -84,8 +101,10 @@ private delegate IntPtr LoadLibraryWDelegate(
private readonly IServiceProvider _services;
private readonly IRuntime _runtime;
private IntPtr _clrDataProcess = IntPtr.Zero;
+ private IntPtr _cdacDataProcess = IntPtr.Zero;
private IntPtr _corDebugProcess = IntPtr.Zero;
private IntPtr _dacHandle = IntPtr.Zero;
+ private IntPtr _cdacHandle = IntPtr.Zero;
private IntPtr _dbiHandle = IntPtr.Zero;
public IntPtr IRuntime { get; }
@@ -132,11 +151,21 @@ protected override void Destroy()
ComWrapper.ReleaseWithCheck(_clrDataProcess);
_clrDataProcess = IntPtr.Zero;
}
+ if (_cdacDataProcess != IntPtr.Zero)
+ {
+ ComWrapper.ReleaseWithCheck(_cdacDataProcess);
+ _cdacDataProcess = IntPtr.Zero;
+ }
if (_dacHandle != IntPtr.Zero)
{
DataTarget.PlatformFunctions.FreeLibrary(_dacHandle);
_dacHandle = IntPtr.Zero;
}
+ if (_cdacHandle != IntPtr.Zero)
+ {
+ DataTarget.PlatformFunctions.FreeLibrary(_cdacHandle);
+ _cdacHandle = IntPtr.Zero;
+ }
if (_dbiHandle != IntPtr.Zero)
{
DataTarget.PlatformFunctions.FreeLibrary(_dbiHandle);
@@ -200,24 +229,45 @@ private string GetRuntimeDirectory(
private int GetClrDataProcess(
IntPtr self,
+ ClrDataProcessFlags flags,
IntPtr* ppClrDataProcess)
{
if (ppClrDataProcess == null)
{
return HResult.E_INVALIDARG;
}
- if (_clrDataProcess == IntPtr.Zero)
+ *ppClrDataProcess = IntPtr.Zero;
+ if ((flags & ClrDataProcessFlags.UseCDac) != 0)
{
- try
+ if (_cdacDataProcess == IntPtr.Zero)
{
- _clrDataProcess = CreateClrDataProcess();
+ try
+ {
+ _cdacDataProcess = CreateClrDataProcess(GetCDacHandle());
+ }
+ catch (Exception ex)
+ {
+ Trace.TraceError(ex.ToString());
+ }
}
- catch (Exception ex)
+ *ppClrDataProcess = _cdacDataProcess;
+ }
+ // Fallback to regular DAC instance if CDac isn't enabled or there where errors creating the instance
+ if (*ppClrDataProcess == IntPtr.Zero)
+ {
+ if (_clrDataProcess == IntPtr.Zero)
{
- Trace.TraceError(ex.ToString());
+ try
+ {
+ _clrDataProcess = CreateClrDataProcess(GetDacHandle());
+ }
+ catch (Exception ex)
+ {
+ Trace.TraceError(ex.ToString());
+ }
}
+ *ppClrDataProcess = _clrDataProcess;
}
- *ppClrDataProcess = _clrDataProcess;
if (*ppClrDataProcess == IntPtr.Zero)
{
return HResult.E_NOINTERFACE;
@@ -301,9 +351,8 @@ private int GetEEVersion(
#endregion
- private IntPtr CreateClrDataProcess()
+ private IntPtr CreateClrDataProcess(IntPtr dacHandle)
{
- IntPtr dacHandle = GetDacHandle();
if (dacHandle == IntPtr.Zero)
{
return IntPtr.Zero;
@@ -334,10 +383,9 @@ private IntPtr CreateClrDataProcess()
private IntPtr CreateCorDebugProcess()
{
string dbiFilePath = _runtime.GetDbiFilePath();
- string dacFilePath = _runtime.GetDacFilePath();
- if (dbiFilePath == null || dacFilePath == null)
+ if (dbiFilePath == null)
{
- Trace.TraceError($"Could not find matching DBI {dbiFilePath ?? ""} or DAC {dacFilePath ?? ""} for this runtime: {_runtime.RuntimeModule.FileName}");
+ Trace.TraceError($"Could not find matching DBI {dbiFilePath ?? ""} for this runtime: {_runtime.RuntimeModule.FileName}");
return IntPtr.Zero;
}
if (_dbiHandle == IntPtr.Zero)
@@ -366,6 +414,16 @@ private IntPtr CreateCorDebugProcess()
int hresult = 0;
try
{
+ // This will verify the DAC signature if needed before DBI is passed the DAC path or handle
+ IntPtr dacHandle = GetDacHandle();
+ if (dacHandle == IntPtr.Zero)
+ {
+ return IntPtr.Zero;
+ }
+
+ // The DAC was verified in the GetDacHandle call above. Ignore the verifySignature parameter here.
+ string dacFilePath = _runtime.GetDacFilePath(out bool _);
+
OpenVirtualProcessImpl2Delegate openVirtualProcessImpl2 = SOSHost.GetDelegateFunction(_dbiHandle, "OpenVirtualProcessImpl2");
if (openVirtualProcessImpl2 != null)
{
@@ -387,12 +445,6 @@ private IntPtr CreateCorDebugProcess()
return corDebugProcess;
}
- IntPtr dacHandle = GetDacHandle();
- if (dacHandle == IntPtr.Zero)
- {
- return IntPtr.Zero;
- }
-
// On Linux/MacOS the DAC module handle needs to be re-created using the DAC PAL instance
// before being passed to DBI's OpenVirtualProcess* implementation. The DBI and DAC share
// the same PAL where dbgshim has it's own.
@@ -465,49 +517,65 @@ private IntPtr GetDacHandle()
{
if (_dacHandle == IntPtr.Zero)
{
- string dacFilePath = _runtime.GetDacFilePath();
- if (dacFilePath == null)
- {
- Trace.TraceError($"Could not find matching DAC {dacFilePath ?? ""} for this runtime: {_runtime.RuntimeModule.FileName}");
- return IntPtr.Zero;
- }
- ISettingsService settingsService = _services.GetService();
- IDisposable fileLock = null;
- try
+ _dacHandle = GetDacHandle(useCDac: false);
+ }
+ return _dacHandle;
+ }
+
+ private IntPtr GetCDacHandle()
+ {
+ if (_cdacHandle == IntPtr.Zero)
+ {
+ _cdacHandle = GetDacHandle(useCDac: true);
+ }
+ return _cdacHandle;
+ }
+
+ private IntPtr GetDacHandle(bool useCDac)
+ {
+ bool verifySignature = false;
+ string dacFilePath = useCDac ? _runtime.GetCDacFilePath() : _runtime.GetDacFilePath(out verifySignature);
+ if (dacFilePath == null)
+ {
+ Trace.TraceError($"Could not find matching DAC {dacFilePath ?? ""} {useCDac} for this runtime: {_runtime.RuntimeModule.FileName}");
+ return IntPtr.Zero;
+ }
+ IntPtr dacHandle = IntPtr.Zero;
+ IDisposable fileLock = null;
+ try
+ {
+ if (verifySignature)
{
- if (settingsService is null || settingsService.DacSignatureVerificationEnabled)
- {
- Trace.TraceInformation($"Verifying DAC signing and cert {dacFilePath}");
+ Trace.TraceInformation($"Verifying DAC signing and cert {dacFilePath} {useCDac}");
- // Check if the DAC cert is valid before loading
- if (!AuthenticodeUtil.VerifyDacDll(dacFilePath, out fileLock))
- {
- return IntPtr.Zero;
- }
- }
- try
- {
- _dacHandle = DataTarget.PlatformFunctions.LoadLibrary(dacFilePath);
- }
- catch (Exception ex) when (ex is DllNotFoundException or BadImageFormatException)
+ // Check if the DAC cert is valid before loading
+ if (!AuthenticodeUtil.VerifyDacDll(dacFilePath, out fileLock))
{
- Trace.TraceError($"LoadLibrary({dacFilePath}) FAILED {ex}");
return IntPtr.Zero;
}
}
- finally
+ try
{
- // Keep DAC file locked until it loaded
- fileLock?.Dispose();
+ dacHandle = DataTarget.PlatformFunctions.LoadLibrary(dacFilePath);
}
- Debug.Assert(_dacHandle != IntPtr.Zero);
- if (!RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
+ catch (Exception ex) when (ex is DllNotFoundException or BadImageFormatException)
{
- DllMainDelegate dllmain = SOSHost.GetDelegateFunction(_dacHandle, "DllMain");
- dllmain?.Invoke(_dacHandle, 1, IntPtr.Zero);
+ Trace.TraceError($"LoadLibrary({dacFilePath}) {useCDac} FAILED {ex}");
+ return IntPtr.Zero;
}
}
- return _dacHandle;
+ finally
+ {
+ // Keep DAC file locked until it loaded
+ fileLock?.Dispose();
+ }
+ Debug.Assert(dacHandle != IntPtr.Zero);
+ if (!RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
+ {
+ DllMainDelegate dllmain = SOSHost.GetDelegateFunction(dacHandle, "DllMain");
+ dllmain?.Invoke(dacHandle, 1, IntPtr.Zero);
+ }
+ return dacHandle;
}
#region IRuntime delegates
@@ -537,6 +605,7 @@ private delegate string GetRuntimeDirectoryDelegate(
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
private delegate int GetClrDataProcessDelegate(
[In] IntPtr self,
+ [In] ClrDataProcessFlags flags,
[Out] IntPtr* ppClrDataProcess);
[UnmanagedFunctionPointer(CallingConvention.Winapi)]
diff --git a/src/SOS/SOS.Hosting/SOS.Hosting.csproj b/src/SOS/SOS.Hosting/SOS.Hosting.csproj
index 645c0ec264..bb438ecf78 100644
--- a/src/SOS/SOS.Hosting/SOS.Hosting.csproj
+++ b/src/SOS/SOS.Hosting/SOS.Hosting.csproj
@@ -3,7 +3,10 @@
netstandard2.0SOS.Hosting$(NoWarn);1591;1701;IDE0060
- SOS Hosting support
+ true
+ Diagnostic SOS hosting support
+ $(Description)
+ SOStruetruefalse
diff --git a/src/SOS/SOS.UnitTests/ConfigFiles/Windows/Debugger.Tests.Config.txt b/src/SOS/SOS.UnitTests/ConfigFiles/Windows/Debugger.Tests.Config.txt
index 7fe2f9f26b..885c235399 100644
--- a/src/SOS/SOS.UnitTests/ConfigFiles/Windows/Debugger.Tests.Config.txt
+++ b/src/SOS/SOS.UnitTests/ConfigFiles/Windows/Debugger.Tests.Config.txt
@@ -200,7 +200,7 @@
diff --git a/src/SOS/SOS.UnitTests/Debuggees/DesktopClrHost/CMakeLists.txt b/src/SOS/SOS.UnitTests/Debuggees/DesktopClrHost/CMakeLists.txt
index 6d1aeff17a..567493fa1e 100644
--- a/src/SOS/SOS.UnitTests/Debuggees/DesktopClrHost/CMakeLists.txt
+++ b/src/SOS/SOS.UnitTests/Debuggees/DesktopClrHost/CMakeLists.txt
@@ -29,8 +29,11 @@ set(DESKTOPCLRHOST_LIBRARY
add_library_clr(DesktopClrHost SHARED ${DESKTOPCLRHOST_SOURCES})
+# When building with ninja we need to explicitly link against the 4.8.1 version of the .NET Framework SDK because it is the only one that supports ARM64.
+if(CLR_CMAKE_TARGET_ARCH_ARM64)
+ target_link_directories(DesktopClrHost PRIVATE "$ENV{WindowsSdkDir}/../NETFXSDK/4.8.1/Lib/um/arm64")
+endif(CLR_CMAKE_TARGET_ARCH_ARM64)
+
target_link_libraries(DesktopClrHost ${DESKTOPCLRHOST_LIBRARY})
-install(TARGETS DesktopClrHost DESTINATION ${CLR_MANAGED_BINARY_DIR}/WebApp3/${CLR_BUILD_TYPE}/net8.0)
-install(TARGETS DesktopClrHost DESTINATION ${CLR_MANAGED_BINARY_DIR}/WebApp3/${CLR_BUILD_TYPE}/net9.0)
-install(TARGETS DesktopClrHost DESTINATION ${CLR_MANAGED_BINARY_DIR}/WebApp3/${CLR_BUILD_TYPE}/net10.0)
+install_clr(TARGETS DesktopClrHost DESTINATIONS .)
diff --git a/src/SOS/SOS.UnitTests/Debuggees/WebApp3/WebApp3.csproj b/src/SOS/SOS.UnitTests/Debuggees/WebApp3/WebApp3.csproj
index 2ec653657d..e7d9777bc6 100644
--- a/src/SOS/SOS.UnitTests/Debuggees/WebApp3/WebApp3.csproj
+++ b/src/SOS/SOS.UnitTests/Debuggees/WebApp3/WebApp3.csproj
@@ -5,4 +5,14 @@
$(SupportedSubProcessTargetFrameworks)
+
+
+
+
+
+
+
+
diff --git a/src/SOS/SOS.UnitTests/SOS.UnitTests.csproj b/src/SOS/SOS.UnitTests/SOS.UnitTests.csproj
index f36ff9a527..c0ab49ec2d 100644
--- a/src/SOS/SOS.UnitTests/SOS.UnitTests.csproj
+++ b/src/SOS/SOS.UnitTests/SOS.UnitTests.csproj
@@ -61,5 +61,4 @@
-
diff --git a/src/SOS/SOS.UnitTests/SOS.cs b/src/SOS/SOS.UnitTests/SOS.cs
index fb9d74886d..578e6f609b 100644
--- a/src/SOS/SOS.UnitTests/SOS.cs
+++ b/src/SOS/SOS.UnitTests/SOS.cs
@@ -32,6 +32,14 @@ internal static void SkipIfArm(TestConfiguration config)
}
}
+ internal static void SkipIfWinX86(TestConfiguration config)
+ {
+ if (config.TargetArchitecture == "x86" && OS.Kind == OSKind.Windows)
+ {
+ throw new SkipTestException("Test does not support x86 on Windows");
+ }
+ }
+
internal static async Task RunTest(
string scriptName,
SOSRunner.TestInformation information,
@@ -192,6 +200,39 @@ public SOS(ITestOutputHelper output)
public static IEnumerable Configurations => SOSTestHelpers.GetConfigurations("TestName", value: null);
+ [SkippableTheory, MemberData(nameof(Configurations)), Trait("Category", "CDACCompatible")]
+ public async Task StackTraceSoftwareExceptionFrame(TestConfiguration config)
+ {
+ if (config.RuntimeFrameworkVersionMajor < 10)
+ {
+ throw new SkipTestException("This test validates SoftwareExceptionFrame handling, before .NET10, these aren't used in this debuggee scenario.");
+ }
+
+ SOSTestHelpers.SkipIfWinX86(config);
+
+ await SOSTestHelpers.RunTest(
+ config,
+ debuggeeName: "SimpleThrow",
+ scriptName: "StackTraceSoftwareExceptionFrame.script",
+ Output,
+ testName: "SOS.StackTraceSoftwareExceptionFrame",
+ testTriage: true);
+ }
+
+ [SkippableTheory, MemberData(nameof(Configurations)), Trait("Category", "CDACCompatible")]
+ public async Task StackTraceFaultingExceptionFrame(TestConfiguration config)
+ {
+ SOSTestHelpers.SkipIfWinX86(config);
+
+ await SOSTestHelpers.RunTest(
+ config,
+ debuggeeName: "DivZero",
+ scriptName: "StackTraceFaultingExceptionFrame.script",
+ Output,
+ testName: "SOS.StackTraceFaultingExceptionFrame",
+ testTriage: true);
+ }
+
[SkippableTheory, MemberData(nameof(Configurations))]
public async Task DivZero(TestConfiguration config)
{
@@ -312,6 +353,12 @@ await SOSTestHelpers.RunTest(
[SkippableTheory, MemberData(nameof(Configurations))]
public async Task StackTests(TestConfiguration config)
{
+ if (config.RuntimeFrameworkVersionMajor == 10)
+ {
+ // The clrstack -i command regressed on .NET 10 win-x86, so skip this test for now.
+ SOSTestHelpers.SkipIfWinX86(config);
+ }
+
await SOSTestHelpers.RunTest(
config,
debuggeeName: "NestedExceptionTest",
@@ -447,6 +494,11 @@ await SOSTestHelpers.RunTest(
[SkippableTheory, MemberData(nameof(Configurations))]
public async Task ConcurrentDictionaries(TestConfiguration config)
{
+ if (OS.Kind != OSKind.Windows && config.RuntimeFrameworkVersionMajor == 10)
+ {
+ throw new SkipTestException("Dumping concurrent dict objects in dumps hits unavailable memory on linux dumps. Tracking: dotnet/diagnostics#5491");
+ }
+
await SOSTestHelpers.RunTest(
scriptName: "ConcurrentDictionaries.script",
new SOSRunner.TestInformation
diff --git a/src/SOS/SOS.UnitTests/SOSRunner.cs b/src/SOS/SOS.UnitTests/SOSRunner.cs
index e6deb33295..647bb08fb7 100644
--- a/src/SOS/SOS.UnitTests/SOSRunner.cs
+++ b/src/SOS/SOS.UnitTests/SOSRunner.cs
@@ -378,9 +378,10 @@ public static async Task CreateDump(TestInformation information)
if (pipeServer != null)
{
dotnetDumpOutputHelper.WriteLine("Waiting for connection on pipe {0}", pipeName);
- CancellationTokenSource source = new(TimeSpan.FromMinutes(5));
+ using CancellationTokenSource source = new(TimeSpan.FromMinutes(5));
// Wait for debuggee to connect/write to pipe or if the process exits on some other failure/abnormally
+ // TODO: This is a resiliency issue - we'll try to collect the dump even if the debuggee fails to connect.
await Task.WhenAny(pipeServer.WaitForConnectionAsync(source.Token), processRunner.WaitForExit());
}
@@ -691,6 +692,11 @@ public static async Task StartDebugger(TestInformation information, D
WithLog(scriptLogger).
WithTimeout(TimeSpan.FromMinutes(10));
+ if (config.TestCDAC)
+ {
+ processRunner.WithEnvironmentVariable("DOTNET_ENABLE_CDAC", "1");
+ }
+
// Exit codes on Windows should always be 0, but not on Linux/OSX for the faulting debuggees.
if (OS.Kind == OSKind.Windows)
{
@@ -988,7 +994,7 @@ public async Task LoadSosExtension()
if (_config.PublishSingleFile)
{
string appRootDir = ReplaceVariables(_variables, "%DEBUG_ROOT%");
- commands.Add($"!SetSymbolServer -ms -directory {appRootDir}");
+ commands.Add($"!SetSymbolServer -ms -timeout 10 -directory {appRootDir}");
}
if (!string.IsNullOrEmpty(setSymbolServer))
{
@@ -1022,7 +1028,7 @@ public async Task LoadSosExtension()
if (_config.PublishSingleFile)
{
string appRootDir = ReplaceVariables(_variables, "%DEBUG_ROOT%");
- commands.Add($"setsymbolserver -ms -directory {appRootDir}");
+ commands.Add($"setsymbolserver -ms -timeout 10 -directory {appRootDir}");
}
if (!string.IsNullOrEmpty(setSymbolServer))
{
diff --git a/src/SOS/SOS.UnitTests/Scripts/OtherCommands.script b/src/SOS/SOS.UnitTests/Scripts/OtherCommands.script
index e7655c6c8d..86f950c084 100644
--- a/src/SOS/SOS.UnitTests/Scripts/OtherCommands.script
+++ b/src/SOS/SOS.UnitTests/Scripts/OtherCommands.script
@@ -106,13 +106,10 @@ SOSCOMMAND:DumpClass \s*Class:\s+()\s+
VERIFY:\s*Class Name:\s+SymbolTestApp.Program\s+
VERIFY:\s*File:\s+.*SymbolTestApp\.(dll|exe)\s+
-# Issue: https://github.com/dotnet/diagnostics/issues/4654
-!IFDEF:ALPINE
# Verify DumpMT
SOSCOMMAND:DumpMT \s*Method Table:\s+()\s+
VERIFY:\s*Name:\s+SymbolTestApp.Program\s+
VERIFY:\s*File:\s+.*SymbolTestApp\.(dll|exe)\s+
-ENDIF:ALPINE
SOSCOMMAND:FinalizeQueue
VERIFY:\s*SyncBlocks to be cleaned up:\s+\s+
diff --git a/src/SOS/SOS.UnitTests/Scripts/StackTests.script b/src/SOS/SOS.UnitTests/Scripts/StackTests.script
index 002a019e0c..485774da01 100644
--- a/src/SOS/SOS.UnitTests/Scripts/StackTests.script
+++ b/src/SOS/SOS.UnitTests/Scripts/StackTests.script
@@ -26,7 +26,7 @@ VERIFY:.*\s+\s+\s+NestedExceptionTest\.Program\.Main\(.*\)\s+\[(
ENDIF:64BIT
!IFDEF:WINDOWS
-SOSCOMMAND:SetSymbolServer -ms -loadsymbols
+SOSCOMMAND:SetSymbolServer -ms -timeout 10 -loadsymbols
ENDIF:WINDOWS
!IFDEF:DOTNETDUMP
diff --git a/src/SOS/SOS.UnitTests/Scripts/StackTraceFaultingExceptionFrame.script b/src/SOS/SOS.UnitTests/Scripts/StackTraceFaultingExceptionFrame.script
new file mode 100644
index 0000000000..b9752a27ff
--- /dev/null
+++ b/src/SOS/SOS.UnitTests/Scripts/StackTraceFaultingExceptionFrame.script
@@ -0,0 +1,45 @@
+# Divide By Zero debugging scenario
+# 1) Load the executable
+# 2) Run the executable and wait for it to crash
+# 3) Take a dump of the executable.
+# 4) Open the dump and compare the output
+
+CONTINUE
+
+LOADSOS
+
+# Verifying that PrintException gives us the right exception in the format above.
+SOSCOMMAND:PrintException
+VERIFY:Exception object:\s+\s+
+VERIFY:Exception type:\s+System\.DivideByZeroException\s+
+VERIFY:Message:\s+(|Attempted to divide by zero\.)\s+
+VERIFY:InnerException:\s+\s+
+VERIFY:StackTrace \(generated\):\s+
+VERIFY:\s+SP\s+IP\s+\s+Function\s+
+VERIFY:\s+\s+\s+[Dd]iv[Zz]ero.*!C\.DivideByZero(\(.*\))?\+0x\s+
+VERIFY:\s+\s+\s+[Dd]iv[Zz]ero.*!C\.F3(\(.*\))?\+0x\s+
+VERIFY:\s+\s+\s+[Dd]iv[Zz]ero.*!C\.F2(\(.*\))?\+0x\s+
+VERIFY:\s+\s+\s+[Dd]iv[Zz]ero.*!C\.Main(\(.*\))?\+0x\s+
+VERIFY:(StackTraceString: \s+)?\s+
+VERIFY:HResult:\s+80020012\s+
+
+# Verify that Threads (clrthreads) works
+SOSCOMMAND:clrthreads
+VERIFY:\s*ThreadCount:\s+\s+
+VERIFY:\s+UnstartedThread:\s+\s+
+VERIFY:\s+BackgroundThread:\s+\s+
+VERIFY:\s+PendingThread:\s+\s+
+VERIFY:\s+DeadThread:\s+\s+
+VERIFY:\s+Hosted Runtime:\s+no\s+
+VERIFY:\s+ID\s+OSID\s+ThreadOBJ\s+State.*\s+
+VERIFY:\s+\s+\s+\s+.*\s+
+
+# Verify that ClrStack with no options works
+SOSCOMMAND:ClrStack
+VERIFY:.*OS Thread Id:\s+0x\s+.*
+VERIFY:\s+Child\s+SP\s+IP\s+Call Site\s+
+VERIFY:\s+\s+\s+\[FaultingExceptionFrame: \]\s+
+VERIFY:\s+\s+\s+(\*\*\* WARNING: Unable to verify checksum for DivZero.exe\s*)?
+VERIFY:\s+\s+\s+C\.F3(\(.*\))?\s+
+VERIFY:\s+\s+\s+C\.F2(\(.*\))?\s+
+VERIFY:\s+\s+\s+C\.Main(\(.*\))?\s+
diff --git a/src/SOS/SOS.UnitTests/Scripts/StackTraceSoftwareExceptionFrame.script b/src/SOS/SOS.UnitTests/Scripts/StackTraceSoftwareExceptionFrame.script
new file mode 100644
index 0000000000..de81ee7f15
--- /dev/null
+++ b/src/SOS/SOS.UnitTests/Scripts/StackTraceSoftwareExceptionFrame.script
@@ -0,0 +1,42 @@
+# Simple Throw debugging scenario
+# 1) Load the executable
+# 2) Run the executable and wait for it to crash
+# 3) Take a dump of the executable.
+# 4) Open the dump and compare the output
+
+CONTINUE
+
+LOADSOS
+
+# Verifying that PrintException gives us the right exception in the format above.
+SOSCOMMAND:PrintException
+VERIFY:Exception object:\s+\s+
+VERIFY:Exception type:\s+System\.InvalidOperationException\s+
+VERIFY:Message:\s+(|Throwing an invalid operation\.\.\.\.)\s+
+VERIFY:InnerException:\s+\s+
+VERIFY:StackTrace \(generated\):\s+
+VERIFY:\s+SP\s+IP\s+\s+Function\s+
+VERIFY:\s+\s+\s+.*!UserObject\.UseObject(\(.*\))?\+0x\s+
+VERIFY:\s+\s+\s+.*!Simple\.Main(\(.*\))?\+0x\s+
+VERIFY:(StackTraceString: \s+)?\s+
+VERIFY:HResult:\s+80131509\s+
+
+# Verify that Threads (clrthreads) works
+SOSCOMMAND:clrthreads
+VERIFY:\s*ThreadCount:\s+\s+
+VERIFY:\s+UnstartedThread:\s+\s+
+VERIFY:\s+BackgroundThread:\s+\s+
+VERIFY:\s+PendingThread:\s+\s+
+VERIFY:\s+DeadThread:\s+\s+
+VERIFY:\s+Hosted Runtime:\s+no\s+
+VERIFY:\s+ID\s+OSID\s+ThreadOBJ\s+State.*\s+
+VERIFY:\s+\s+\s+\s+.*\s+
+
+# Verify that ClrStack with no options works
+SOSCOMMAND:ClrStack
+VERIFY:.*OS Thread Id:\s+0x\s+.*
+VERIFY:\s+Child\s+SP\s+IP\s+Call Site\s+
+VERIFY:\s+\s+\s+\[InlinedCallFrame: \]\s+
+VERIFY:\s+\s+\s+\[SoftwareExceptionFrame: \]\s+
+VERIFY:\s+\s+\s+UserObject\.UseObject(\(.*\))?\s+
+VERIFY:\s+\s+\s+Simple\.Main(\(.*\))?\s+
diff --git a/src/SOS/Strike/clrma/managedanalysis.cpp b/src/SOS/Strike/clrma/managedanalysis.cpp
index 0483ff9269..24250b4de6 100644
--- a/src/SOS/Strike/clrma/managedanalysis.cpp
+++ b/src/SOS/Strike/clrma/managedanalysis.cpp
@@ -260,7 +260,7 @@ ClrmaManagedAnalysis::AssociateClient(
TraceError("AssociateClient GetRuntime FAILED %08x\n", hr);
return hr;
}
- if (FAILED(hr = runtime->GetClrDataProcess(&m_clrData)))
+ if (FAILED(hr = runtime->GetClrDataProcess(IRuntime::ClrDataProcessFlags::UseCDac, &m_clrData)))
{
m_clrData = GetClrDataFromDbgEng();
if (m_clrData == nullptr)
diff --git a/src/SOS/Strike/disasm.h b/src/SOS/Strike/disasm.h
index 82ad583780..f50bbd768f 100644
--- a/src/SOS/Strike/disasm.h
+++ b/src/SOS/Strike/disasm.h
@@ -77,7 +77,7 @@ struct SOSEHInfo
UINT EHCount;
CLRDATA_ADDRESS methodStart;
- SOSEHInfo() { ZeroMemory(this, sizeof(SOSEHInfo)); }
+ SOSEHInfo() { ZeroMemory(static_cast(this), sizeof(SOSEHInfo)); }
~SOSEHInfo() { if (m_pInfos) { delete [] m_pInfos; } }
void FormatForDisassembly(CLRDATA_ADDRESS offSet);
diff --git a/src/SOS/Strike/exts.cpp b/src/SOS/Strike/exts.cpp
index 1bb392ba47..f1c3c9fdaf 100644
--- a/src/SOS/Strike/exts.cpp
+++ b/src/SOS/Strike/exts.cpp
@@ -107,6 +107,8 @@ ExtInit(PDEBUG_CLIENT client)
g_bDacBroken = TRUE;
g_clrData = NULL;
g_sos = NULL;
+ g_sos15 = NULL;
+ g_sos16 = NULL;
// Flush here only on Windows under dbgeng. The lldb sos plugin handles it for Linux/MacOS.
#ifndef FEATURE_PAL
diff --git a/src/SOS/Strike/exts.h b/src/SOS/Strike/exts.h
index 5ee3a0c70d..6f1285c6ac 100644
--- a/src/SOS/Strike/exts.h
+++ b/src/SOS/Strike/exts.h
@@ -241,6 +241,8 @@ class __ExtensionCleanUp
/* We may reconsider caching g_clrData in the future */ \
ToRelease spIDP(g_clrData); \
ToRelease spISD(g_sos); \
+ ToRelease spISD15(g_sos15); \
+ ToRelease spISD16(g_sos16); \
ResetGlobals();
#define INIT_API_PROBE_MANAGED(name) \
@@ -275,8 +277,10 @@ class __ExtensionCleanUp
} \
/* If LoadClrDebugDll() succeeded make sure we release g_clrData. */ \
/* We may reconsider caching g_clrData in the future */ \
+ ToRelease spIDP(g_clrData); \
ToRelease spISD(g_sos); \
- ToRelease spIDP(g_clrData);
+ ToRelease spISD15(g_sos15); \
+ ToRelease spISD16(g_sos16);
#ifdef FEATURE_PAL
diff --git a/src/SOS/Strike/metadata.cpp b/src/SOS/Strike/metadata.cpp
index 95d12f9aa0..f5a2b80789 100644
--- a/src/SOS/Strike/metadata.cpp
+++ b/src/SOS/Strike/metadata.cpp
@@ -2,9 +2,9 @@
// The .NET Foundation licenses this file to you under the MIT license.
// ==++==
-//
-
-//
+//
+
+//
// ==--==
#include "strike.h"
#include "util.h"
@@ -12,7 +12,7 @@
/**********************************************************************\
* Routine Description: *
* *
-* This function is called to find the name of a TypeDef using *
+* This function is called to find the name of a TypeDef using *
* metadata API. *
* *
\**********************************************************************/
@@ -20,9 +20,14 @@
static HRESULT NameForTypeDef_s(mdTypeDef tkTypeDef, IMetaDataImport *pImport,
__out_ecount (capacity_mdName) WCHAR *mdName, size_t capacity_mdName)
{
+ if (pImport == NULL)
+ {
+ int res = swprintf_s(mdName, capacity_mdName, W("0x%08x"), tkTypeDef);
+ return res != -1 ? S_OK : E_FAIL;
+ }
+
DWORD flags;
ULONG nameLen;
-
HRESULT hr = pImport->GetTypeDefProps(tkTypeDef, mdName,
(ULONG)capacity_mdName, &nameLen,
&flags, NULL);
@@ -68,13 +73,13 @@ IMetaDataImport* MDImportForModule(DacpModuleData* pModule)
IMetaDataImport *pRet = NULL;
ToRelease module;
HRESULT hr = g_sos->GetModule(pModule->Address, &module);
-
+
if (SUCCEEDED(hr))
hr = module->QueryInterface(IID_IMetaDataImport, (LPVOID *) &pRet);
if (SUCCEEDED(hr))
return pRet;
-
+
return NULL;
}
@@ -104,9 +109,9 @@ HRESULT NameForToken_s(mdTypeDef mb, IMetaDataImport *pImport, __out_ecount (cap
//ExtOut("unsupported\n");
return E_FAIL;
}
-
+
HRESULT hr = E_FAIL;
-
+
PAL_CPP_TRY
{
static WCHAR name[MAX_CLASSNAME_LENGTH];
@@ -168,7 +173,7 @@ HRESULT NameForToken_s(mdTypeDef mb, IMetaDataImport *pImport, __out_ecount (cap
/**********************************************************************\
* Routine Description: *
* *
-* This function is called to find the name of a metadata token *
+* This function is called to find the name of a metadata token *
* using metadata API. *
* *
\**********************************************************************/
@@ -206,7 +211,7 @@ void NameForToken_s(DacpModuleData *pModule, mdTypeDef mb, __out_ecount (capacit
{
hr = NameForToken_s (mb, pImport, mdName, capacity_mdName, bClassName);
}
-
+
if (!pImport || !SUCCEEDED (hr))
{
const SIZE_T capacity_moduleName = mdNameLen+19;
@@ -218,7 +223,7 @@ void NameForToken_s(DacpModuleData *pModule, mdTypeDef mb, __out_ecount (capacit
if (assembly.isDynamic) {
wcscpy_s(moduleName, capacity_moduleName, W("Dynamic "));
}
- wcscat_s (moduleName, capacity_moduleName, W("Module in "));
+ wcscat_s (moduleName, capacity_moduleName, W("Module in "));
if(g_sos->GetAssemblyName(pModule->Assembly, mdNameLen, g_mdName, NULL)==S_OK)
{
wcscat_s(moduleName, capacity_moduleName, g_mdName);
@@ -238,9 +243,16 @@ class MDInfo
public:
MDInfo (DWORD_PTR ModuleAddr)
{
- m_pImport = MDImportForModule(ModuleAddr);
- if (!m_pImport)
- ExtOut("Unable to get IMetaDataImport for module %p\n", ModuleAddr);
+ if (ModuleAddr == 0)
+ {
+ m_pImport = NULL;
+ }
+ else
+ {
+ m_pImport = MDImportForModule(ModuleAddr);
+ if (!m_pImport)
+ ExtOut("Unable to get IMetaDataImport for module %p\n", ModuleAddr);
+ }
m_pSigBuf = NULL;
}
@@ -251,7 +263,7 @@ class MDInfo
m_pSigBuf = NULL;
}
- void GetMethodName(mdTypeDef token, CQuickBytes *fullName);
+ void GetMethodName(mdMethodDef token, CQuickBytes *fullName);
GetSignatureStringResults GetMethodSignature(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, CQuickBytes *fullName);
GetSignatureStringResults GetSignature(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, CQuickBytes *fullName);
@@ -308,46 +320,47 @@ void GetMethodName(mdMethodDef methodDef, IMetaDataImport * pImport, CQuickBytes
// Tables for mapping element type to text
-const WCHAR *g_wszMapElementType[] =
+const WCHAR *g_wszMapElementType[] =
{
W("End"), // 0x0
W("Void"), // 0x1
W("Boolean"),
- W("Char"),
+ W("Char"),
W("I1"),
- W("UI1"),
+ W("U1"),
W("I2"), // 0x6
- W("UI2"),
+ W("U2"),
W("I4"),
- W("UI4"),
+ W("U4"),
W("I8"),
- W("UI8"),
+ W("U8"),
W("R4"),
W("R8"),
W("String"),
W("Ptr"), // 0xf
W("ByRef"), // 0x10
- W("ValueClass"),
+ W("ValueType"),
W("Class"),
- W("CopyCtor"),
+ W("Var"),
W("MDArray"), // 0x14
- W("GENArray"),
+ W("GenericInst"),
W("TypedByRef"),
- W("VALUEARRAY"),
- W("I"),
- W("U"),
- W("R"), // 0x1a
- W("FNPTR"),
+ W("UNUSED"),
+ W("IntPtr"),
+ W("UIntPtr"),
+ W("UNUSED"), // 0x1a
+ W("FnPtr"),
W("Object"),
W("SZArray"),
- W("GENERICArray"),
+ W("MVar"),
W("CMOD_REQD"),
W("CMOD_OPT"),
W("INTERNAL"),
+ W("CMOD_INTERNAL"),
};
-
-const WCHAR *g_wszCalling[] =
-{
+
+const WCHAR *g_wszCalling[] =
+{
W("[DEFAULT]"),
W("[C]"),
W("[STDCALL]"),
@@ -358,36 +371,44 @@ const WCHAR *g_wszCalling[] =
W("[LOCALSIG]"),
W("[PROPERTY]"),
W("[UNMANAGED]"),
+ W("[GENERICINST]"),
+ W("[NATIVEVARARG]"),
+ W("[UNKNOWN]"),
+ W("[UNKNOWN]"),
+ W("[UNKNOWN]"),
+ W("[UNKNOWN]"),
};
-void MDInfo::GetMethodName(mdTypeDef token, CQuickBytes *fullName)
+void MDInfo::GetMethodName(mdMethodDef token, CQuickBytes *fullName)
{
- if (m_pImport == NULL) {
- return;
- }
-
- HRESULT hr;
- mdTypeDef memTypeDef;
- ULONG nameLen;
- DWORD flags;
- PCCOR_SIGNATURE pbSigBlob;
- ULONG ulSigBlob;
- ULONG ulCodeRVA;
- ULONG ulImplFlags;
+ HRESULT hr = E_FAIL;
+ mdTypeDef memTypeDef = mdTypeDefNil;
+ ULONG nameLen = 0;
+ DWORD flags = 0;
+ PCCOR_SIGNATURE pbSigBlob = NULL;
+ ULONG ulSigBlob = 0;
+ ULONG ulCodeRVA = 0;
+ ULONG ulImplFlags = 0;
m_pSigBuf = fullName;
InitSigBuffer();
WCHAR szFunctionName[1024];
- hr = m_pImport->GetMethodProps(token, &memTypeDef,
- szFunctionName, ARRAY_SIZE(szFunctionName), &nameLen,
- &flags, &pbSigBlob, &ulSigBlob, &ulCodeRVA, &ulImplFlags);
- if (FAILED (hr))
+ if (m_pImport != NULL)
{
+ hr = m_pImport->GetMethodProps(token, &memTypeDef,
+ szFunctionName, ARRAY_SIZE(szFunctionName), &nameLen,
+ &flags, &pbSigBlob, &ulSigBlob, &ulCodeRVA, &ulImplFlags);
+ }
+
+ if (FAILED(hr))
+ {
+ swprintf_s(szFunctionName, ARRAY_SIZE(szFunctionName), W("0x%08x"), token);
+ AddToSigBuffer(szFunctionName);
return;
}
-
+
szFunctionName[nameLen] = L'\0';
m_szName[0] = L'\0';
if (memTypeDef != mdTypeDefNil)
@@ -420,9 +441,6 @@ void MDInfo::GetMethodName(mdTypeDef token, CQuickBytes *fullName)
GetSignatureStringResults MDInfo::GetMethodSignature(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, CQuickBytes *fullName)
{
- if (!m_pImport)
- return GSS_ERROR;
-
m_pSigBuf = fullName;
InitSigBuffer();
@@ -431,7 +449,7 @@ GetSignatureStringResults MDInfo::GetMethodSignature(PCCOR_SIGNATURE pbSigBlob,
LONG lSigBlobRemaining;
if (FAILED(GetFullNameForMD(pbSigBlob, ulSigBlob, &lSigBlobRemaining)))
return GSS_ERROR;
-
+
if (lSigBlobRemaining < 0)
return GSS_INSUFFICIENT_DATA;
@@ -441,9 +459,6 @@ GetSignatureStringResults MDInfo::GetMethodSignature(PCCOR_SIGNATURE pbSigBlob,
GetSignatureStringResults MDInfo::GetSignature(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, CQuickBytes *fullName)
{
- if (!m_pImport)
- return GSS_ERROR;
-
m_pSigBuf = fullName;
InitSigBuffer();
@@ -461,10 +476,9 @@ GetSignatureStringResults MDInfo::GetSignature(PCCOR_SIGNATURE pbSigBlob, ULONG
return GSS_SUCCESS;
}
-
inline bool isCallConv(unsigned sigByte, CorCallingConvention conv)
{
- return ((sigByte & IMAGE_CEE_CS_CALLCONV_MASK) == (unsigned) conv);
+ return ((sigByte & IMAGE_CEE_CS_CALLCONV_MASK) == (unsigned) conv);
}
#undef IfFailGoto
@@ -492,16 +506,11 @@ HRESULT MDInfo::GetFullNameForMD(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, LON
// goto ErrExit;
AddToSigBuffer (g_wszCalling[ulData & IMAGE_CEE_CS_CALLCONV_MASK]);
- if (cb>ulSigBlob)
+ if (cb>ulSigBlob)
goto ErrExit;
cbCur += cb;
ulSigBlob -= cb;
- if (ulData & IMAGE_CEE_CS_CALLCONV_HASTHIS)
- AddToSigBuffer ( W(" [hasThis]"));
- if (ulData & IMAGE_CEE_CS_CALLCONV_EXPLICITTHIS)
- AddToSigBuffer ( W(" [explicit]"));
-
AddToSigBuffer (W(" "));
if ( isCallConv(ulData,IMAGE_CEE_CS_CALLCONV_FIELD) )
{
@@ -510,15 +519,36 @@ HRESULT MDInfo::GetFullNameForMD(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, LON
goto ErrExit;
AddToSigBuffer ( W(" "));
AddToSigBuffer ( m_szName);
- if (cb>ulSigBlob)
+ if (cb>ulSigBlob)
goto ErrExit;
cbCur += cb;
ulSigBlob -= cb;
}
- else
+ else
{
+ if (ulData & IMAGE_CEE_CS_CALLCONV_HASTHIS)
+ AddToSigBuffer ( W("[hasThis] "));
+ if (ulData & IMAGE_CEE_CS_CALLCONV_EXPLICITTHIS)
+ AddToSigBuffer ( W("[explicit] "));
+
+ if (ulData & IMAGE_CEE_CS_CALLCONV_GENERIC)
+ {
+ ULONG ulGenericCount;
+ cb = CorSigUncompressData(&pbSigBlob[cbCur], &ulGenericCount);
+ if (cb>ulSigBlob)
+ goto ErrExit;
+ AddToSigBuffer (W("[generic:"));
+
+ WCHAR buffer[16];
+ _itow_s(ulGenericCount, buffer, ARRAY_SIZE(buffer), 10);
+ AddToSigBuffer (buffer);
+ AddToSigBuffer (W("] "));
+ cbCur += cb;
+ ulSigBlob -= cb;
+ }
+
cb = CorSigUncompressData(&pbSigBlob[cbCur], &ulArgs);
- if (cb>ulSigBlob)
+ if (cb>ulSigBlob)
goto ErrExit;
cbCur += cb;
ulSigBlob -= cb;
@@ -531,7 +561,7 @@ HRESULT MDInfo::GetFullNameForMD(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, LON
AddToSigBuffer (W(" "));
AddToSigBuffer (m_szName);
AddToSigBuffer ( W("("));
- if (cb>ulSigBlob)
+ if (cb>ulSigBlob)
goto ErrExit;
cbCur += cb;
ulSigBlob -= cb;
@@ -551,7 +581,7 @@ HRESULT MDInfo::GetFullNameForMD(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, LON
if (i != ulArgs) {
AddToSigBuffer ( W(","));
}
- if (cb>ulSigBlob)
+ if (cb>ulSigBlob)
goto ErrExit;
cbCur += cb;
@@ -573,39 +603,42 @@ HRESULT MDInfo::GetFullNameForMD(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, LON
LPCWSTR MDInfo::TypeDefName(mdTypeDef inTypeDef)
{
- if (m_pImport == NULL) {
- return W("");
+ HRESULT hr = E_FAIL;
+ if (m_pImport != NULL)
+ {
+ hr = m_pImport->GetTypeDefProps(
+ // [IN] The import scope.
+ inTypeDef, // [IN] TypeDef token for inquiry.
+ m_szTempBuf, // [OUT] Put name here.
+ MAX_CLASSNAME_LENGTH , // [IN] size of name buffer in wide chars.
+ NULL, // [OUT] put size of name (wide chars) here.
+ NULL, // [OUT] Put flags here.
+ NULL); // [OUT] Put base class TypeDef/TypeRef here.
+ }
+ if (FAILED(hr))
+ {
+ swprintf_s(m_szTempBuf, MAX_CLASSNAME_LENGTH, W("0x%08x"), inTypeDef);
}
- HRESULT hr;
-
- hr = m_pImport->GetTypeDefProps(
- // [IN] The import scope.
- inTypeDef, // [IN] TypeDef token for inquiry.
- m_szTempBuf, // [OUT] Put name here.
- MAX_CLASSNAME_LENGTH , // [IN] size of name buffer in wide chars.
- NULL, // [OUT] put size of name (wide chars) here.
- NULL, // [OUT] Put flags here.
- NULL); // [OUT] Put base class TypeDef/TypeRef here.
-
- if (FAILED(hr)) return (W("NoName"));
return (m_szTempBuf);
} // LPCWSTR MDInfo::TypeDefName()
+
LPCWSTR MDInfo::TypeRefName(mdTypeRef tr)
{
- if (m_pImport == NULL) {
- return W("");
+ HRESULT hr = E_FAIL;
+ if (m_pImport != NULL)
+ {
+ hr = m_pImport->GetTypeRefProps(
+ tr, // The class ref token.
+ NULL, // Resolution scope.
+ m_szTempBuf, // Put the name here.
+ MAX_CLASSNAME_LENGTH, // Size of the name buffer, wide chars.
+ NULL); // Put actual size of name here.
+ }
+ if (FAILED(hr))
+ {
+ swprintf_s(m_szTempBuf, MAX_CLASSNAME_LENGTH, W("0x%08x"), tr);
}
-
- HRESULT hr;
-
- hr = m_pImport->GetTypeRefProps(
- tr, // The class ref token.
- NULL, // Resolution scope.
- m_szTempBuf, // Put the name here.
- MAX_CLASSNAME_LENGTH, // Size of the name buffer, wide chars.
- NULL); // Put actual size of name here.
- if (FAILED(hr)) return (W("NoName"));
return (m_szTempBuf);
} // LPCWSTR MDInfo::TypeRefName()
@@ -643,7 +676,6 @@ HRESULT MDInfo::GetOneElementType(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, UL
ULONG ulTemp;
int iTemp = 0;
mdToken tk;
- const size_t capacity_buffer = 9;
cb = CorSigUncompressData(pbSigBlob, &ulData);
@@ -673,11 +705,12 @@ HRESULT MDInfo::GetOneElementType(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, UL
}
// Handle the underlying element types.
- if (ulData >= ELEMENT_TYPE_MAX)
+ if (ulData >= ELEMENT_TYPE_MAX)
{
hr = E_FAIL;
goto ErrExit;
}
+
while (ulData == ELEMENT_TYPE_PTR || ulData == ELEMENT_TYPE_BYREF)
{
IfFailGo(AddToSigBuffer(g_wszMapElementType[ulData]));
@@ -687,14 +720,24 @@ HRESULT MDInfo::GetOneElementType(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, UL
}
// Generics
- if (ulData == ELEMENT_TYPE_VAR)
+ if (ulData == ELEMENT_TYPE_VAR || ulData == ELEMENT_TYPE_MVAR)
{
- IfFailGo(AddToSigBuffer(W("__Canon")));
+ if (ulData == ELEMENT_TYPE_VAR)
+ {
+ IfFailGo(AddToSigBuffer(W("!")));
+ }
+ else
+ {
+ IfFailGo(AddToSigBuffer(W("!!")));
+ }
+
+ ULONG varIndex = 0;
+ IfFailGo(CorSigUncompressData(&pbSigBlob[cbCur], ulSigBlob-cbCur, &varIndex, &cb));
+ cbCur += cb;
- // The next byte represents which generic parameter is referred to. We
- // do not currently use this information, so just bypass this byte.
- cbCur++;
-
+ WCHAR buffer[16];
+ _itow_s(varIndex, buffer, ARRAY_SIZE(buffer), 10);
+ AddToSigBuffer(buffer);
goto ErrExit;
}
@@ -707,26 +750,32 @@ HRESULT MDInfo::GetOneElementType(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, UL
// Get the number of generic arguments.
ULONG numParams = 0;
- IfFailGo(CorSigUncompressData(&pbSigBlob[cbCur], 1, &numParams, &cb));
+ IfFailGo(CorSigUncompressData(&pbSigBlob[cbCur], ulSigBlob-cbCur, &numParams, &cb));
cbCur += cb;
// Print out the list of arguments
IfFailGo(AddToSigBuffer(W("<")));
for (ULONG i = 0; i < numParams; i++)
{
- if (i > 0)
+ if (i > 0)
IfFailGo(AddToSigBuffer(W(",")));
-
+
IfFailGo(GetOneElementType(&pbSigBlob[cbCur], ulSigBlob-cbCur, &cb));
cbCur += cb;
}
IfFailGo(AddToSigBuffer(W(">")));
goto ErrExit;
}
-
+
// Past this point we must have something which directly maps to a value in g_wszMapElementType.
+ if (ulData >= ARRAY_SIZE(g_wszMapElementType))
+ {
+ IfFailGo(AddToSigBuffer(W("INVALID_ELEMENT_TYPE")));
+ return E_FAIL;
+ }
+
IfFailGo(AddToSigBuffer(g_wszMapElementType[ulData]));
- if (CorIsPrimitiveType((CorElementType)ulData) ||
+ if (CorIsPrimitiveType((CorElementType)ulData) ||
ulData == ELEMENT_TYPE_TYPEDBYREF ||
ulData == ELEMENT_TYPE_OBJECT ||
ulData == ELEMENT_TYPE_I ||
@@ -737,8 +786,8 @@ HRESULT MDInfo::GetOneElementType(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, UL
}
AddToSigBuffer(W(" "));
- if (ulData == ELEMENT_TYPE_VALUETYPE ||
- ulData == ELEMENT_TYPE_CLASS ||
+ if (ulData == ELEMENT_TYPE_VALUETYPE ||
+ ulData == ELEMENT_TYPE_CLASS ||
ulData == ELEMENT_TYPE_CMOD_REQD ||
ulData == ELEMENT_TYPE_CMOD_OPT)
{
@@ -753,8 +802,8 @@ HRESULT MDInfo::GetOneElementType(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, UL
else
{
_ASSERTE(TypeFromToken(tk) == mdtTypeSpec);
- WCHAR buffer[capacity_buffer];
- _itow_s (tk, buffer, capacity_buffer, 16);
+ WCHAR buffer[16];
+ _itow_s (tk, buffer, ARRAY_SIZE(buffer), 16);
IfFailGo(AddToSigBuffer(buffer));
}
if (ulData == ELEMENT_TYPE_CMOD_REQD ||
@@ -770,13 +819,13 @@ HRESULT MDInfo::GetOneElementType(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, UL
}
if (ulData == ELEMENT_TYPE_SZARRAY)
{
- // display the base type of SZARRAY or GENERICARRAY
+ // display the base type of SZARRAY
if (FAILED(GetOneElementType(&pbSigBlob[cbCur], ulSigBlob-cbCur, &cb)))
goto ErrExit;
cbCur += cb;
goto ErrExit;
}
- if (ulData == ELEMENT_TYPE_FNPTR)
+ if (ulData == ELEMENT_TYPE_FNPTR)
{
cb = CorSigUncompressData(&pbSigBlob[cbCur], &ulData);
cbCur += cb;
@@ -798,7 +847,7 @@ HRESULT MDInfo::GetOneElementType(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, UL
cbCur += cb;
IfFailGo(AddToSigBuffer(W("(")));
- while (numArgs > 0)
+ while (numArgs > 0)
{
if (cbCur > ulSigBlob)
goto ErrExit;
@@ -806,7 +855,7 @@ HRESULT MDInfo::GetOneElementType(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, UL
goto ErrExit;
cbCur += cb;
--numArgs;
- if (numArgs > 0)
+ if (numArgs > 0)
IfFailGo(AddToSigBuffer(W(",")));
}
IfFailGo(AddToSigBuffer(W(")")));
@@ -821,9 +870,8 @@ HRESULT MDInfo::GetOneElementType(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, UL
cb = CorSigUncompressPointer(&pbSigBlob[cbCur], (void**)&pvMethodTable);
cbCur += cb;
- const size_t capacity_szMethodTableValue = 10;
- WCHAR szMethodTableValue[10];
- itow_s_ptr((INT_PTR)pvMethodTable, szMethodTableValue, capacity_szMethodTableValue, 16);
+ WCHAR szMethodTableValue[32];
+ itow_s_ptr((INT_PTR)pvMethodTable, szMethodTableValue, ARRAY_SIZE(szMethodTableValue), 16);
IfFailGo(AddToSigBuffer(szMethodTableValue));
IfFailGo(AddToSigBuffer(W(" ")));
@@ -834,60 +882,68 @@ HRESULT MDInfo::GetOneElementType(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, UL
goto ErrExit;
}
-
- if(ulData != ELEMENT_TYPE_ARRAY) return E_FAIL;
+ if(ulData != ELEMENT_TYPE_ARRAY)
+ return E_FAIL;
+
+ // Since MDARRAY has extra data, we will use a visual indication
+ // to group the base type and the ArrayShape.
+ IfFailGo(AddToSigBuffer(W("{")));
- // display the base type of SDARRAY
+ // Display the base type of MDARRAY
if (FAILED(GetOneElementType(&pbSigBlob[cbCur], ulSigBlob-cbCur, &cb)))
goto ErrExit;
cbCur += cb;
- IfFailGo(AddToSigBuffer(W(" ")));
- // display the rank of MDARRAY
+ // Print the ArrayShape - ECMA-335 II.23.2.13
+ AddToSigBuffer(W(", "));
+
+ // Display the rank
cb = CorSigUncompressData(&pbSigBlob[cbCur], &ulData);
cbCur += cb;
- WCHAR buffer[capacity_buffer];
- _itow_s (ulData, buffer, capacity_buffer, 10);
+ WCHAR buffer[16];
+ _itow_s (ulData, buffer, ARRAY_SIZE(buffer), 10);
IfFailGo(AddToSigBuffer(buffer));
+
+ // we are done if no rank specified
if (ulData == 0)
- // we are done if no rank specified
goto ErrExit;
IfFailGo(AddToSigBuffer(W(" ")));
+
// how many dimensions have size specified?
cb = CorSigUncompressData(&pbSigBlob[cbCur], &ulData);
cbCur += cb;
- _itow_s (ulData, buffer, capacity_buffer, 10);
+ _itow_s (ulData, buffer, ARRAY_SIZE(buffer), 10);
IfFailGo(AddToSigBuffer(buffer));
- if (ulData == 0) {
+ if (ulData == 0)
IfFailGo(AddToSigBuffer(W(" ")));
- }
- while (ulData)
- {
+ for (;ulData != 0; ulData--)
+ {
cb = CorSigUncompressData(&pbSigBlob[cbCur], &ulTemp);
- _itow_s (ulTemp, buffer, capacity_buffer, 10);
+ _itow_s (ulTemp, buffer, ARRAY_SIZE(buffer), 10);
IfFailGo(AddToSigBuffer(buffer));
IfFailGo(AddToSigBuffer(W(" ")));
cbCur += cb;
- ulData--;
}
+
// how many dimensions have lower bounds specified?
cb = CorSigUncompressData(&pbSigBlob[cbCur], &ulData);
cbCur += cb;
- _itow_s (ulData, buffer, capacity_buffer, 10);
+ _itow_s (ulData, buffer, ARRAY_SIZE(buffer), 10);
IfFailGo(AddToSigBuffer(buffer));
- while (ulData)
- {
+ IfFailGo(AddToSigBuffer(W(" ")));
+ for (;ulData != 0; ulData--)
+ {
cb = CorSigUncompressSignedInt(&pbSigBlob[cbCur], &iTemp);
- _itow_s (iTemp, buffer, capacity_buffer, 10);
+ _itow_s (iTemp, buffer, ARRAY_SIZE(buffer), 10);
IfFailGo(AddToSigBuffer(buffer));
IfFailGo(AddToSigBuffer(W(" ")));
cbCur += cb;
- ulData--;
}
-
+ IfFailGo(AddToSigBuffer(W("}")));
+
ErrExit:
if (cbCur > ulSigBlob)
hr = E_FAIL;
@@ -900,17 +956,17 @@ HRESULT MDInfo::GetOneElementType(PCCOR_SIGNATURE pbSigBlob, ULONG ulSigBlob, UL
//*****************************************************************************
typedef struct tagCOR_ILMETHOD_TINY : IMAGE_COR_ILMETHOD_TINY
{
- bool IsTiny() const { return((Flags_CodeSize & (CorILMethod_FormatMask >> 1)) == CorILMethod_TinyFormat); }
- DWORD GetLocalVarSigTok() const { return(0); }
+ bool IsTiny() const { return((Flags_CodeSize & (CorILMethod_FormatMask >> 1)) == CorILMethod_TinyFormat); }
+ DWORD GetLocalVarSigTok() const { return(0); }
} COR_ILMETHOD_TINY;
//*****************************************************************************
-// This strucuture is the 'fat' layout, where no compression is attempted.
+// This strucuture is the 'fat' layout, where no compression is attempted.
// Note that this structure can be added on at the end, thus making it extensible
//*****************************************************************************
typedef struct tagCOR_ILMETHOD_FAT : IMAGE_COR_ILMETHOD_FAT
{
- bool IsFat() const { return((Flags & CorILMethod_FormatMask) == CorILMethod_FatFormat); }
- mdToken GetLocalVarSigTok() const { return(LocalVarSigTok); }
+ bool IsFat() const { return((Flags & CorILMethod_FormatMask) == CorILMethod_FatFormat); }
+ mdToken GetLocalVarSigTok() const { return(LocalVarSigTok); }
} COR_ILMETHOD_FAT;
diff --git a/src/SOS/Strike/platform/runtimeimpl.cpp b/src/SOS/Strike/platform/runtimeimpl.cpp
index 546c28ddad..3f356315a9 100644
--- a/src/SOS/Strike/platform/runtimeimpl.cpp
+++ b/src/SOS/Strike/platform/runtimeimpl.cpp
@@ -421,7 +421,7 @@ LPCSTR Runtime::GetRuntimeDirectory()
/**********************************************************************\
* Creates an instance of the DAC clr data process
\**********************************************************************/
-HRESULT Runtime::GetClrDataProcess(IXCLRDataProcess** ppClrDataProcess)
+HRESULT Runtime::GetClrDataProcess(ClrDataProcessFlags flags, IXCLRDataProcess** ppClrDataProcess)
{
if (m_clrDataProcess == nullptr)
{
diff --git a/src/SOS/Strike/platform/runtimeimpl.h b/src/SOS/Strike/platform/runtimeimpl.h
index 27b3069407..6d8994eaeb 100644
--- a/src/SOS/Strike/platform/runtimeimpl.h
+++ b/src/SOS/Strike/platform/runtimeimpl.h
@@ -180,7 +180,7 @@ class Runtime : public IRuntime
LPCSTR STDMETHODCALLTYPE GetRuntimeDirectory();
- HRESULT STDMETHODCALLTYPE GetClrDataProcess(IXCLRDataProcess** ppClrDataProcess);
+ HRESULT STDMETHODCALLTYPE GetClrDataProcess(ClrDataProcessFlags flags, IXCLRDataProcess** ppClrDataProcess);
HRESULT STDMETHODCALLTYPE GetCorDebugInterface(ICorDebugProcess** ppCorDebugProcess);
diff --git a/src/SOS/Strike/sosdocs.txt b/src/SOS/Strike/sosdocs.txt
index efde40e62e..af25301273 100644
--- a/src/SOS/Strike/sosdocs.txt
+++ b/src/SOS/Strike/sosdocs.txt
@@ -27,38 +27,33 @@ DumpObj (do) Threads (clrthreads)
DumpALC (dumpalc) ThreadState
DumpArray (da) IP2MD
DumpDelegate U
-DumpStackObjects (dso) DumpStack
-DumpHeap EEStack
-DumpVC ClrStack
-GCRoot GCInfo
-ObjSize EHInfo
-FinalizeQueue BPMD (bpmd)
-PrintException (pe) COMState
-TraverseHeap
+DumpVC DumpStack
+PrintException (pe) EEStack
+ ClrStack
+ GCInfo
+ EHInfo
+ BPMD (bpmd)
+ COMState
Examining CLR data structures Diagnostic Utilities
----------------------------- -----------------------------
-DumpDomain VerifyHeap
-EEHeap VerifyObj
-Name2EE FindRoots
-SyncBlk HeapStat
-DumpMT GCWhere
-DumpClass ListNearObj (lno)
-DumpMD GCHandles
-Token2EE GCHandleLeaks
-EEVersion FinalizeQueue (fq)
-DumpModule FindAppDomain
-ThreadPool (tp) SaveModule
-DumpAssembly SaveAllModules
-DumpSigElem ProcInfo
-DumpRuntimeTypes StopOnException (soe)
-DumpSig DumpLog
-RCWCleanupList VMMap
-DumpIL VMStat
-DumpRCW MinidumpMode
-DumpCCW AnalyzeOOM (ao)
- SuppressJitOptimization
-
+DumpDomain FindRoots
+Name2EE HeapStat
+SyncBlk GCHandles
+DumpMT GCHandleLeaks
+DumpClass FindAppDomain
+DumpMD SaveModule
+Token2EE SaveAllModules
+EEVersion ProcInfo
+DumpModule StopOnException (soe)
+DumpAssembly DumpLog
+DumpSigElem VMMap
+DumpSig VMStat
+RCWCleanupList MinidumpMode
+DumpIL SuppressJitOptimization
+DumpRCW
+DumpCCW
+
Examining the GC history Other
----------------------------- -----------------------------
HistInit SetHostRuntime (sethostruntime)
@@ -361,23 +356,6 @@ specified object was loaded.
\\
-COMMAND: dso.
-COMMAND: dumpstackobjects.
-!DumpStackObjects [-verify] [top stack [bottom stack]]
-
-This command will display any managed objects it finds within the bounds of
-the current stack. Combined with the stack tracing commands like K and
-!ClrStack, it is a good aid to determining the values of locals and
-parameters.
-
-If you use the -verify option, each non-static CLASS field of an object
-candidate is validated. This helps to eliminate false positives. It is not
-on by default because very often in a debugging scenario, you are
-interested in objects with invalid fields.
-
-The abbreviation !dso can be used for brevity.
-\\
-
COMMAND: dumpdelegate.
!DumpDelegate
@@ -391,149 +369,6 @@ For example:
000001461bacb098 00007ffc5c894b68 ConsoleApp16.Program.StaticMethod()
\\
-COMMAND: dumpheap.
-!DumpHeap [-stat]
- [-strings]
- [-short]
- [-min ]
- [-max ]
- [-live]
- [-dead]
- [-thinlock]
- [-startAtLowerBound]
- [-mt ]
- [-type ]
- [start [end]]
-
-!DumpHeap is a powerful command that traverses the garbage collected heap,
-collection statistics about objects. With it's various options, it can look for
-particular types, restrict to a range, or look for ThinLocks (see !SyncBlk
-documentation). Finally, it will provide a warning if it detects excessive
-fragmentation in the GC heap.
-
-When called without options, the output is first a list of objects in the heap,
-followed by a report listing all the types found, their size and number:
-
- 0:000> !dumpheap
- Address MT Size
- 00a71000 0015cde8 12 Free
- 00a7100c 0015cde8 12 Free
- 00a71018 0015cde8 12 Free
- 00a71024 5ba58328 68
- 00a71068 5ba58380 68
- 00a710ac 5ba58430 68
- 00a710f0 5ba5dba4 68
- ...
- total 619 objects
- Statistics:
- MT Count TotalSize Class Name
- 5ba7607c 1 12 System.Security.Permissions.HostProtectionResource
- 5ba75d54 1 12 System.Security.Permissions.SecurityPermissionFlag
- 5ba61f18 1 12 System.Collections.CaseInsensitiveComparer
- ...
- 0015cde8 6 10260 Free
- 5ba57bf8 318 18136 System.String
- ...
-
-"Free" objects are simply regions of space the garbage collector can use later.
-If 30% or more of the heap contains "Free" objects, the process may suffer from
-heap fragmentation. This is usually caused by pinning objects for a long time
-combined with a high rate of allocation. Here is example output where !DumpHeap
-provides a warning about fragmentation:
-
-
- Fragmented blocks larger than 1MB:
- Addr Size Followed by
- 00a780c0 1.5MB 00bec800 System.Byte[]
- 00da4e38 1.2MB 00ed2c00 System.Byte[]
- 00f16df0 1.2MB 01044338 System.Byte[]
-
-The arguments in detail:
-
--stat Restrict the output to the statistical type summary
--strings Restrict the output to a statistical string value summary
--short Limits output to just the address of each object. This allows you
- to easily pipe output from the command to another debugger
- command for automation.
--min Ignore objects less than the size given in bytes (hex)
--max Ignore objects larger than the size given in bytes (hex)
--live Only print live objects
--dead Only print dead objects (objects which will be collected in the
- next full GC)
--thinlock Report on any ThinLocks (an efficient locking scheme, see !SyncBlk
- documentation for more info)
--startAtLowerBound
- Force heap walk to begin at lower bound of a supplied address range.
- (During plan phase, the heap is often not walkable because objects
- are being moved. In this case, DumpHeap may report spurious errors,
- in particular bad objects. It may be possible to traverse more of
- the heap after the reported bad object. Even if you specify an
- address range, !DumpHeap will start its walk from the beginning of
- the heap by default. If it finds a bad object before the specified
- range, it will stop before displaying the part of the heap in which
- you are interested. This switch will force !DumpHeap to begin its
- walk at the specified lower bound. You must supply the address of a
- good object as the lower bound for this to work. Display memory at
- the address of the bad object to manually find the next method
- table (use !dumpmt to verify). If the GC is currently in a call to
- memcopy, You may also be able to find the next object's address by
- adding the size to the start address given as parameters.)
--mt List only those objects with the MethodTable given
--type List only those objects whose type name is a substring match of the
- string provided.
-start Begin listing from this address
-end Stop listing at this address
-
-A special note about -type: Often, you'd like to find not only Strings, but
-System.Object arrays that are constrained to contain Strings. ("new
-String[100]" actually creates a System.Object array, but it can only hold
-System.String object pointers). You can use -type in a special way to find
-these arrays. Just pass "-type System.String[]" and those Object arrays will
-be returned. More generally, "-type []".
-
-The start/end parameters can be obtained from the output of !EEHeap -gc. For
-example, if you only want to list objects in the large heap segment:
-
- 0:000> !eeheap -gc
- Number of GC Heaps: 1
- generation 0 starts at 0x00c32754
- generation 1 starts at 0x00c32748
- generation 2 starts at 0x00a71000
- segment begin allocated size
- 00a70000 00a71000 010443a8 005d33a8(6108072)
- Large object heap starts at 0x01a71000
- segment begin allocated size
- 01a70000 01a71000 01a75000 0x00004000(16384)
- Total Size 0x5d73a8(6124456)
- ------------------------------
- GC Heap Size 0x5d73a8(6124456)
-
- 0:000> !dumpheap 1a71000 1a75000
- Address MT Size
- 01a71000 5ba88bd8 2064
- 01a71810 0019fe48 2032 Free
- 01a72000 5ba88bd8 4096
- 01a73000 0019fe48 4096 Free
- 01a74000 5ba88bd8 4096
- total 5 objects
- Statistics:
- MT Count TotalSize Class Name
- 0019fe48 2 6128 Free
- 5ba88bd8 3 10256 System.Object[]
- Total 5 objects
-
-Finally, if GC heap corruption is present, you may see an error like this:
-
- 0:000> !dumpheap -stat
- object 00a73d24: does not have valid MT
- curr_object : 00a73d24
- Last good object: 00a73d14
- ----------------
-
-That indicates a serious problem. See the help for !VerifyHeap for more
-information on diagnosing the cause.
-\\
-
COMMAND: dumpvc.
!DumpVC
@@ -575,135 +410,6 @@ MethodTable for it, and the Value column provides the start address:
of value classes, while others do not.
\\
-COMMAND: gcroot.
-!GCRoot [-nostacks] [-all]