Add documentation on EventPipe

docs/core/diagnostics/dotnet-gcdump.md

@@ -23,7 +23,7 @@ dotnet-gcdump [-h|--help] [--version] <command>
 
 ## Description
 
-The `dotnet-gcdump` global tool is a way to collect GC (Garbage Collector) dumps of live .NET processes. It uses the EventPipe technology, which is a cross-platform alternative to ETW on Windows. GC dumps are created by triggering a GC in the target process, turning on special events, and regenerating the graph of object roots from the event stream. This process allows for GC dumps to be collected while the process is running and with minimal overhead. These dumps are useful for several scenarios:
+The `dotnet-gcdump` global tool collects GC (Garbage Collector) dumps of live .NET processes using [EventPipe](./eventpipe.md). GC dumps are created by triggering a GC in the target process, turning on special events, and regenerating the graph of object roots from the event stream. This process allows for GC dumps to be collected while the process is running and with minimal overhead. These dumps are useful for several scenarios:
 
 - Comparing the number of objects on the heap at several points in time.
 - Analyzing roots of objects (answering questions like, "what still has a reference to this type?").

docs/core/diagnostics/dotnet-trace.md

@@ -27,7 +27,7 @@ The `dotnet-trace` tool:
 
 * Is a cross-platform .NET Core tool.
 * Enables the collection of .NET Core traces of a running process without a native profiler.
-* Is built around the cross-platform `EventPipe` technology of the .NET Core runtime.
+* Is built on [`EventPipe`](./eventpipe.md) of the .NET Core runtime.
 * Delivers the same experience on Windows, Linux, or macOS.
 
 ## Options

docs/core/diagnostics/event-counters.md

@@ -12,7 +12,7 @@ EventCounters are .NET Core APIs used for lightweight, cross-platform, and near
 
 The .NET Core runtime and a few .NET libraries publish basic diagnostics information using EventCounters starting in .NET Core 3.0. Apart from the EventCounters that are provided by the .NET runtime, you may choose to implement your own EventCounters. EventCounters can be used to track various metrics.
 
-EventCounters live as a part of an <xref:System.Diagnostics.Tracing.EventSource>, and are automatically pushed to listener tools on a regular basis. Like all other events on an <xref:System.Diagnostics.Tracing.EventSource>, they can be consumed both in-proc and out-of-proc via <xref:System.Diagnostics.Tracing.EventListener> and EventPipe. This article focuses on the cross-platform capabilities of EventCounters, and intentionally excludes PerfView and ETW (Event Tracing for Windows) - although both can be used with EventCounters.
+EventCounters live as a part of an <xref:System.Diagnostics.Tracing.EventSource>, and are automatically pushed to listener tools on a regular basis. Like all other events on an <xref:System.Diagnostics.Tracing.EventSource>, they can be consumed both in-proc and out-of-proc via <xref:System.Diagnostics.Tracing.EventListener> and [EventPipe](./eventpipe.md). This article focuses on the cross-platform capabilities of EventCounters, and intentionally excludes PerfView and ETW (Event Tracing for Windows) - although both can be used with EventCounters.
 
 ![EventCounters in-proc and out-of-proc diagram image](media/event-counters.svg)
 

docs/core/diagnostics/eventpipe.md

@@ -0,0 +1,91 @@
+---
+title: EventPipe Overview
+description: Learn about EventPipe and how to use it for tracing your .NET applications to diagnose performance issues.
+ms.date: 11/09/2020
+ms.topic: overview
+---
+
+# EventPipe
+
+EventPipe is a runtime component that can be used to collect tracing data, similar to ETW or LTTng. The goal of EventPipe is to allow .NET developers to easily trace their .NET applications without having to rely on platform-specific OS-native components such as ETW or LTTng.
+
+EventPipe is the mechanism behind many of the diagnostic tools and can be used for consuming events emitted by the runtime as well as custom events written with [EventSource](xref:System.Diagnostics.Tracing.EventSource).
+
+This article is a high-level overview of EventPipe describing when and how to use EventPipe, and how to configure it to best suit your needs.
+
+## EventPipe basics
+
+EventPipe aggregates events emitted by runtime components - for example, the Just-In-Time compiler or the garbage collector - and events written from [EventSource](xref:System.Diagnostics.Tracing.EventSource) instances in the libraries and user code.
+
+The events are then serialized and can be written directly to a file or consumed through a Diagnostics Port from out-of-proces. On Windows, Diagnostic Ports are implemented as `NamedPipe`s. On non-Windows platforms, such as Linux or macOS, it is implemented using Unix Domain Sockets. For more information about the Diagnostics Port and how to interact with it via its custom inter-process communication protocol, see the [diagnostics IPC protocol documentation](https://github.com/dotnet/diagnostics/blob/master/documentation/design-docs/ipc-protocol.md).
+
+EventPipe then writes the serialized events in the `.nettrace` file format, either as a stream via Diagnostic Ports or directly to a file. To learn more about the EventPipe serialization format, refer to the [EventPipe format documentation](https://github.com/microsoft/perfview/blob/master/src/TraceEvent/EventPipe/EventPipeFormat.md).
+
+## EventPipe vs. ETW/LTTng
+
+EventPipe is part of the .NET runtime (CoreCLR) and is designed to work the same way across all the platforms .NET Core supports. This allows tracing tools based on EventPipe, such as `dotnet-counters`, `dotnet-gcdump`, and `dotnet-trace`, to work seamlessly across platforms.
+
+However, because EventPipe is a runtime built-in component, its scope is limited to managed code and the runtime itself. EventPipe cannot be used for tracking some lower-level events, such as resolving native code stack or getting various kernel events. If you use C/C++ interop in your app or you want to trace the runtime itself (which is written in C++), or want deeper diagnostics into the behavior of the app that requires kernel events (that is, native-thread context-switching events) you should use ETW or [perf/LTTng](./trace-perfcollect-lttng.md).
+
+Another major difference between EventPipe and ETW/LTTng is admin/root privilege requirement. To trace an application using ETW or LTTng you need to be an admin/root. Using EventPipe, you can trace applications as long as the tracer (for example, `dotnet-trace`) is run as the same user as the user that launched the application.
+
+The following table is a summary of the differences between EventPipe and ETW/LTTng.
+
+|Feature|EventPipe|ETW|LTTng|
+|-------|---------|---|-----------|
+|Cross-platform|Yes|No (only on Windows)|No (only on supported Linux distros)|
+|Require admin/root privilege|No|Yes|Yes|
+|Can get OS/kernel events|No|Yes|Yes|
+|Can resolve native callstacks|No|Yes|Yes|
+
+## Use EventPipe to trace your .NET application
+
+You can use EventPipe to trace your .NET application in many ways:
+
+* Use one of the [diagnostics tools](#tools-using-eventpipe) that are built on top of EventPipe.
+
+* Use [Microsoft.Diagnostics.NETCore.Client](https://github.com/dotnet/diagnostics/blob/master/documentation/diagnostics-client-library-instructions.md) library to write your own tool to configure and start EventPipe sessions yourself.
+
+* Use [environment variables](#trace-using-environment-variables) to start EventPipe.
+
+After you've produced a `nettrace` file that contains your EventPipe events, you can view the file in [`PerfView`](https://github.com/Microsoft/perfview#perfview-overview) or Visual Studio. On non-Windows platforms, you can convert the `nettrace` file to a `speedscope` or `Chromium` trace format by using [`dotnet-trace convert`](./dotnet-trace.md#dotnet-trace-convert) command and view it with [`speedscope`](https://www.speedscope.app/) or Chrome DevTools.
+
+You can also analyze EventPipe traces programmatically with [TraceEvent](https://github.com/Microsoft/perfview/blob/master/documentation/TraceEvent/TraceEventLibrary.md).
+
+### Tools that use EventPipe
+
+This is the easiest way to use EventPipe to trace your application. To learn more about how to use each of these tools, refer to each tool's documentation.
+
+* [dotnet-counters](./dotnet-counters.md) lets you monitor and collect various metrics emitted by the .NET runtime and core libraries, as well as custom metrics you can write.
+
+* [dotnet-gcdump](./dotnet-gcdump.md) lets you collect GC heap dumps of live processes for analyzing an application's managed heap.
+
+* [dotnet-trace](./dotnet-trace.md) lets you collect traces of applications to analyze for performance.
+
+## Trace using environment variables
+
+The preferred mechanism for using EventPipe is to use `dotnet-trace` or the `Microsoft.Diagnostics.NETCore.Client` library.
+
+However, you can use the following environment variables to set up an EventPipe session on an app and have it write the trace directly to a file. To stop tracing, exit the application.
+
+* `COMPlus_EnableEventPipe`: Set this to `1` to start an EventPipe session that writes directly to a file. The default value is `0`.
+
+* `COMPlus_EventPipeOutputPath`: The path to the output EventPipe trace file when it's configured to run via `COMPlus_EnableEventPipe`. The default value is `trace.nettrace`, which will be created in the same directory that the app is running from.
+
+* `COMPlus_CircularBufferMB`: The size of the internal buffer that is used by EventPipe when it's configured to run via `COMPlus_EnableEventPipe`.
+
+* `COMPlus_EventPipeConfig`: Sets up the EventPipe session configuration when starting an EventPipe session with `COMPlus_EnableEventPipe`.
+
+  The syntax is as follows:
+
+  `<provider>:<keyword>:<level>`
+
+  You can also specify multiple providers by concatenating them with a comma:
+
+  `<provider1>:<keyword1>:<level1>,<provider2>:<keyword2>:<level2>`
+
+  If this environment variable is not set but EventPipe is enabled by `COMPlus_EnableEventPipe`, it will start tracing by enabling the following providers with the following keywords and levels:
+
+  - `Microsoft-Windows-DotNETRuntime:4c14fccbd:5`
+  - `Microsoft-Windows-DotNETRuntimePrivate:4002000b:5`
+  - `Microsoft-DotNETCore-SampleProfiler:0:5`

docs/core/diagnostics/logging-tracing.md

@@ -49,7 +49,7 @@ The following APIs are more event oriented. Rather than logging simple strings t
   - Only allows tracing serializable objects.
   - Can be consumed in-process via any [EventListener](xref:System.Diagnostics.Tracing.EventListener) instances configured to consume the EventSource.
   - Can be consumed out-of-process via:
-    - .NET Core's EventPipe on all platforms
+    - [.NET Core's EventPipe](./eventpipe.md) on all platforms
     - [Event Tracing for Windows (ETW)](/windows/win32/etw/event-tracing-portal)
     - [LTTng tracing framework for Linux](https://lttng.org/)
       - Walkthrough: [Collect an LTTng trace using PerfCollect](trace-perfcollect-lttng.md).

docs/fundamentals/toc.yml

@@ -370,6 +370,8 @@ items:
       href: ../core/diagnostics/debug-linux-dumps.md
     - name: EventCounters
       href: ../core/diagnostics/event-counters.md
+    - name: EventPipe
+      href: ../core/diagnostics/eventpipe.md
     - name: Logging and tracing
       href: ../core/diagnostics/logging-tracing.md
     - name: Collect diagnostics in containers