Add some more documentation (DataDog#2317)

* Move docs to _docs_ folder

Fix links

* Clarify return types in instrumentation docs

* Mention that integration methods are optional

* Add small section on constraints duck-typing.

Author: andrewlock

tracer/src/Datadog.Trace/ClrProfiler/README.md renamed to docs/development/AutomaticInstrumentation.md

@@ -8,12 +8,12 @@ Creating a new instrumentation implementation typically uses the following proce
 
 1. Identify the operation of interest that we want to measure. Also gather the tags, resource names that we will need to set. Don't forget to check what has been implemented by other tracers.
 2. Find an appropriate instrumentation point in the target library. You may need to use multiple instrumentation points, and you may need to use different targets for different _versions_ of the library
-3. Create an instrumentation class using one of the standard "shapes" (described below), and place it in the [ClrProfiler/AutoInstrumentation folder](./AutoInstrumentation). If the methods you need to instrument have different prototypes (especially the number of parameters), you will need multiple class to instrument them.
+3. Create an instrumentation class using one of the standard "shapes" (described below), and place it in the [ClrProfiler/AutoInstrumentation folder](../../tracer/src/Datadog.Trace/ClrProfiler/AutoInstrumentation). If the methods you need to instrument have different prototypes (especially the number of parameters), you will need multiple class to instrument them.
 4. Add an `[InstrumentMethod]` attribute to the instrumentation class, as described below. Alternatively, add an assembly-level `[AdoNetClientInstrumentMethods]` attribute
 5. (Optional) Create duck-typing unit tests in _Datadog.Trace.Tests_ to confirm any duck types are valid. This can make the feedback cycle much faster than relying on integration tests
 6. Create integration tests for your instrumentation 
    1. Create (or reuse) a sample application that uses the target library, which ideally exercises all the code paths in your new instrumentation. Use an `$(ApiVersion)` MSBuild variables to allow testing against multiple package versions in CI. 
-   2. Add an entry in [tracer/build/PackageVersionsGeneratorDefinitions.json](../../../build/PackageVersionsGeneratorDefinitions.json) defining the range of all supported versions. See the existing definitions for examples
+   2. Add an entry in [tracer/build/PackageVersionsGeneratorDefinitions.json](../../tracer/build/PackageVersionsGeneratorDefinitions.json) defining the range of all supported versions. See the existing definitions for examples
    3. Run `./tracer/build.ps1 GeneratePackageVersions`. This generates the xunit test data for package versions in the `TestData` that you can use as `[MemberData]` for your `[Theory]` tests. 
    4. Use the `MockTracerAgent` to confirm your instrumentation is working as expected.
 7. After testing locally, push to GitHub, and do a manual run in Azure Devops for your branch
@@ -40,14 +40,16 @@ public class ClientQueryIteratorsIntegrations
 
     // Include ONE of the following:
     
-    // 👇 Async method
+    // 👇 Async method, with different behavior based on the return type
+    //   - Task<TReturn>: returnValue is set to the Result
+    //   - Task: TReturn is set to typeof(object) and returnValue is set to null
     internal static TReturn OnAsyncMethodEnd<TTarget, TReturn>(TTarget instance, TReturn returnValue, Exception exception, in CallTargetState state)
     {
         state.Scope?.DisposeWithException(exception);
         return returnValue;
     }
 
-    // 👇 Method with return value TReturn
+    // 👇 Method with return type TReturn
     internal static CallTargetReturn<TReturn> OnMethodEnd<TTarget, TReturn>(TTarget instance, TReturn returnValue, Exception exception, in CallTargetState state)
     {
         // Run your "method end" code here
@@ -61,12 +63,42 @@ public class ClientQueryIteratorsIntegrations
 }
 ```
 
+> Note that both `OnMethodBegin` and `OnMethodEnd` are optional. If you only need one of the integration points, you can omit the others
+ 
+The first parameter passed to the method is the instance on which the method is called (for `static` methods, this parameter should be omitted), and should be a _generic parameter_ type.  
+ 
+For parameters that are well-known types like `string`, `object`, or `Exception`, you can use the type directly in the `OnMethodBegin` or `OnMethodEnd` methods. For other types that can't be directly referenced, such as types in the target-library, you should use generic parameters. If you need to manipulate the generic parameters, for example to access values, use the duck-typing approach described below.
+
+### Duck-typing, instrumentation classes, and constraints
+
+When creating instrumentation classes you often need to work with `Type`s in the target library that you can't reference directly. Rather than using reflection directly to manipulate these types, the .NET Tracer has an optimised solution for working with them called _Duck Typing_.
+
+> See [the Duck Typing document](./DuckTyping.md) for a detailed description of duck-typing, use cases, best practices,  and benchmarks.
+ 
+You can use duck-typing imperatively at runtime to "cast" any object to a type you can manipulate directly, but if you know at call time that you need to work with one of the generic parameters passed to an `OnMethodBegin` or `OnMethodEnd`, you can use a more performant approach leveraging _constraints_. 
+
+Add a constraint to your method that the generic type implements your duck type. The value passed to your method will then be pre-duck-typed, and will have better performance than using duck-typing manually at a later point. For example, the integration below [(A GraphQL integration)](../../tracer/src/Datadog.Trace/ClrProfiler/AutoInstrumentation/GraphQL/ExecuteAsyncIntegration.cs) takes a single parameter which is duck-typed using constraints to implement the type IExecutionContext
+
+```csharp
+public class ExecuteAsyncIntegration
+{
+    internal static CallTargetState OnMethodBegin<TTarget, TContext>(TTarget instance, TContext context)
+        where TContext : IExecutionContext
+    {
+        // ...
+    }
+```
+
+> ⚠ Note that pre duck-typed parameters that use constraints will never be `null`. If you need to check the parameter for `null`, add the `IDuckType` constraint too, and check the value of `IDuckType.Instance`. 
+
+For more information on duck-typing, see [the documentation](./DuckTyping.md).
+
 ### Instrumentation attributes
 
 A source generator is used to automatically "find" all custom instrumentation classes in the app and generate a list of them to pass to the native CLR profiler. We do this by using one of two attributes:
 
-- [`[InstrumentMethod]`](./InstrumentMethodAttribute.cs)
-- [`[AdoNetClientInstrumentMethods]`](./AutoInstrumentation/AdoNet/AdoNetClientInstrumentMethodsAttribute.cs)
+- [`[InstrumentMethod]`](../../tracer/src/Datadog.Trace/ClrProfiler/InstrumentMethodAttribute.cs)
+- [`[AdoNetClientInstrumentMethods]`](../../tracer/src/Datadog.Trace/ClrProfiler/AutoInstrumentation/AdoNet/AdoNetClientInstrumentMethodsAttribute.cs)
 
 > Alternatively, you can _manually_ call `NativeMethods.InitializeProfiler()`, passing in `NativeCallTargetDefinition[]`. This is not the "normal" approach, but may be necessary when you need to dynamically generate definitions, for example in serverless scenarios
 
@@ -88,7 +120,7 @@ public class HttpClientHandlerIntegration
 }
 ```
 
-We also have a special case for ADO.NET method instrumentation, as this is generally more convoluted, and requires a lot of duplication. All new ADO.NET implementations will likely reuse existing instrumentation classes, such as [`CommandExecuteReaderIntegration`](./AutoInstrumentation/AdoNet/CommandExecuteReaderIntegration.cs) for example. To save having to specify many `[InstrumentMethod]` attributes, you can instead use the  `[AdoNetClientInstrumentMethods]` _assembly_ attribute, to define some standard types, as well as which of the standard ADO.NET signatures to implement. For example:
+We also have a special case for ADO.NET method instrumentation, as this is generally more convoluted, and requires a lot of duplication. All new ADO.NET implementations will likely reuse existing instrumentation classes, such as [`CommandExecuteReaderIntegration`](../../tracer/src/Datadog.Trace/ClrProfiler/AutoInstrumentation/AdoNet/CommandExecuteReaderIntegration.cs) for example. To save having to specify many `[InstrumentMethod]` attributes, you can instead use the  `[AdoNetClientInstrumentMethods]` _assembly_ attribute, to define some standard types, as well as which of the standard ADO.NET signatures to implement. For example:
 
 ```csharp
 [assembly: AdoNetClientInstrumentMethods(
@@ -114,6 +146,6 @@ We also have a special case for ADO.NET method instrumentation, as this is gener
     })]
 ```
 
-The above attribute shows how to select which signatures to implement, via the  `TargetMethodAttributes` property. These attributes are nested types defined inside [`AdoNetClientInstrumentMethodsAttribute`](./AutoInstrumentation/AdoNet/AdoNetClientInstrumentMethodsAttribute.cs), each of which are associated with a given signature + instrumentation class (via the `[AdoNetClientInstrumentMethodsAttribute.AdoNetTargetSignature]` attribute)
+The above attribute shows how to select which signatures to implement, via the  `TargetMethodAttributes` property. These attributes are nested types defined inside [`AdoNetClientInstrumentMethodsAttribute`](../../tracer/src/Datadog.Trace/ClrProfiler/AutoInstrumentation/AdoNet/AdoNetClientInstrumentMethodsAttribute.cs), each of which are associated with a given signature + instrumentation class (via the `[AdoNetClientInstrumentMethodsAttribute.AdoNetTargetSignature]` attribute)
 
 > Note that there are separate target method attributes if you are using the new abstract/interface instrumentation feature.

tracer/src/Datadog.Trace/DuckTyping/README.md renamed to docs/development/DuckTyping.md

@@ -372,7 +372,7 @@ In summary:
 <details>
 <summary>Best practices benchmarks</summary>
         
-[Benchmark Code](../../../test/benchmarks/Benchmarks.Trace/DuckTyping/DuckTypeMethodCallComparisonBenchmark.cs)
+[Benchmark Code](../../tracer/test/benchmarks/Benchmarks.Trace/DuckTyping/DuckTypeMethodCallComparisonBenchmark.cs)
 
 ``` ini
 BenchmarkDotNet=v0.12.1, OS=Windows 10.0.22000

tracer/README.MD

@@ -37,7 +37,7 @@ You can develop the tracer on various environments.
 - Optional: [nuget.exe CLI](https://www.nuget.org/downloads) v5.3 or newer
 - Optional: [WiX Toolset 3.11.1](http://wixtoolset.org/releases/) or newer to build Windows installer (msi)
   - [WiX Toolset Visual Studio Extension](https://wixtoolset.org/releases/) to build installer from Visual Studio
-- Optional: [Docker for Windows](https://docs.docker.com/docker-for-windows/) to build Linux binaries and run integration tests on Linux containers. See [section on Docker Compose](#building-and-running-tests-with-docker-compose).
+- Optional: [Docker for Windows](https://docs.docker.com/docker-for-windows/) to build Linux binaries and run integration tests on Linux containers.
   - Requires Windows 10 (1607 Anniversary Update, Build 14393 or newer)
 
 Microsoft provides [evaluation developer VMs](https://developer.microsoft.com/en-us/windows/downloads/virtual-machines) with Windows and Visual Studio pre-installed.
@@ -101,8 +101,8 @@ You can use Rider and CLion to develop on macOS. Building and testing can be don
 
 ## Additional Technical Documentation
 
-* [Implementing an automatic instrumentation](./src/Datadog.Trace/ClrProfiler/README.md)
-* [Duck typing: usages, best practices, and benchmarks](./src/Datadog.Trace/DuckTyping/README.md)
+* [Implementing an automatic instrumentation](../docs/development/AutomaticInstrumentation.md)
+* [Duck typing: usages, best practices, and benchmarks](../docs/development/DuckTyping.md)
 * [Datadog.Trace NuGet package README](../docs/Datadog.Trace/README.md)
 
 ## Further Reading