INTRODUCTION

diginsight telemetry is a set .Net packages that that provides automatic observability for dotnet applications.
In particular, the full application flow is made available to local text based streams such as traditional file logs, the Console log or the Azure Streaming log and also to remote analysis tools such as Azure Monitor and Prometheus/Grafana.

The image below shows the text based stream associated to to a Web API call.

The following image shows the same call on the Azure Monitor Transaction Detail where the call structure is shown as a hierarchy of activities (also called spans) and trace details:

Performance information gathered by diginsight can be analyzed in the form of metrics.
The following image shows the Azure Monitor Metrics dashboard where method invocations and latencies can be analized in value and frequency:

ADDITIONAL INFORMATION

Application flow observability is provided by means of .Net ILogger, System Diagnostics classes so that diginsight telemetry can be mixed and analyzed with other components telemetry, as long as they rely on the same standard framework classes.
Observability for remote tools is provided by means of OpenTelemetry so that telemetry data can be targeted to Azure Monitor and also other analysis tools such as Prometheus/Graphana.

The following image shows diginsight metrics such as span durations and frequencies on a custom Grafana dashboard receiving data by means of Opentelemetry Prometheus stack.

Diginsight application flow is:

• consistent across tools: every information or metric visible on the local text based streams can be published and observed on the remote analysis tools (eg. on Appinsight Transaction detail or Appinsight Metrics).

• consistent with code: the application flow is published with information about classes, method names and call nesting so the 'gap' from telemetry and code is shortened for application developers and site reliability engineers.

• consistent across applications application flow published in the same way for all applications. so it is easily readable for peopble without background knowledge on the application logic.
  
  

Diginsight uses dynamic logging, smart sampling, automatic truncation and other strategies to maximize applications efficiency and minimize telemetry cost so that local analysis and analysis on the remote tools can be supported without compromises on performance and without compromises on cost of telemetry.

Diginsight log layout and automatic rendering for entities can be fully customized to ensure best readability of the application flow.

Paragraph GETTING STARTED discusses basic steps we can follow to integrate diginsight telemetry.

The following articles explain in details:

• HowTo: use dynamic logging to manage loglevel dynamically, at runtime.

• HowTo: use Dynamic-Configuration to manage configurations and feature flags dynamically, at runtime.

• HowTo: Configure diginsight telemetry to the local text based streams.

• HowTo: Configure diginsight telemetry to the remote tools.

• HowTo: customize entities rendering on diginsight log streams.

• HowTo: Customize metrics sent to the remote tools.

• HowTo: Customize diginsight log streams row content.

• HowTo: maximize application performance and minimize telemetry cost with diginsight.

diginsight v3 is now available
the following article describes improvements of diginsight v3 over the previous version. Introduction to Diginsight v3.
Features such as old frameworks support (eg. .Net Framework 4.5+) or observability for startup and static sections may still have limited support on Diginsight v3.
In these cases the developer may decide to keep the old model.
Diginsight v2 will not be discontinued until feature parity is reached.
Documentation about v2 packages is still available here Diginsight v2 documentation.

APPLICATION OBSERVABILITY CONCEPTS

Application observability is about aggregating, correlating and analyzing the following key elements:

• Logs with application execution details and data.
• The requests and operations structure (sometimes also referred as Activity, Traces or Spans) with the structure of application calls, related to an event or exception condition.
• Metrics: numeric values (such as latencies, payload sizes, frequencies) that can be aggregated and correlated with the operations structure and the logs.

The image below shows examples about the 3 observability elements on Azure Monitor Performance Management (APM) Tools:

Diginsight makes observability easy as:

• it integrates the 3 observability elements (Log, Traces, Metrics) into high performance text-based streams.
  In particular, traditional File log, Console log or Azure Streaming Console log can be integrated with the full application execution flow.
  
• it publishes the 3 observability elements to OpenTelemetry and allowing application analysis by means of remote APM tools such as Azure Monitor and Grafana.
  

Example analysis on Diginsight telemetry

The following image shows a diginsight application flow on a text based stream for DataAnalyticsReportsController.GetDevices method:

Starting from its traceid (42488cedb33da51726293a70c3463c71), the same flow can be found as an Azure Monitor Application transaction:

Latencies for the same function can be analyzed with the span_duration metric, filtered on DataAnalyticsReportsController.GetDevices method.

In facts, the span_duration metric allows analyzing latencies of any method within code.
Also, we'll see that the developer can easily add other metrics and metric properties to split and compare values in different conditions (eg. by site properties, user properties etc).

GETTING STARTED

With version 3 diginsight streamlines OpenTelemetry integration embracing standard notation for activity tracing with dotnet System Diagnostic API.

Using dotnet System Diagnostic API the following notation can be used to instrument a code span:

Using diginsight v3 the same section can be instrumented with the following notation:

where, StartMethodActivity:

• gathers automatically the method name,
• renders automatically the method payload
• writes the Span START and END to the logger variable

and SetOutput stores the method result for rendering within method END line.

Steps to use Diginsight v3

You can ottain a console log or file log with diginsight by means of the following steps.
The code snippets below are available as working samples within the telemetry_samples repository.

Article HOWTO - Use Diginsight Samples: explores how we can use diginsight samples to test and understand integration of Diginsight telemetry in our own projects.

STEP 01 - Add a package reference to the package Diginsight.Diagnostics or Diginsight.Diagnostics.Log4Net

In the first step you can just add a diginsight reference to your code:

STEP 02 - Configure logging within the Startup sequence

in the second step you can configure the startup sequence to enable diginsight log:

in this case:

• AddDiginsightConsole() is used to enabled log to the application Console.
• AddDiginsightLog4Net() is used to enabled file log by means of log4net.

a separate - log4net.config can be used to specify the usual log4net configuration:

also, the Diginsight:Activities section can be used to specify enabled ActivitySources and whether Activity logging is enabled.

STEP 03 - Add telemetry to code with StartMethodActivity() and ILogger Statements

we are now ready to add instrumentation to the code and make the application flow observable:

STEP 04 - Enable OpenTelemetry and send data to the remote tools

With few changes to the startup sequence, telemetry can be sent to the remote tools. Telemetry to the local tools is less expensive, more efficient, well protected and often it is not even persisted. So, telemetry to the local tools can include verbose data with the maximum level of information.

Telemetry to the remote tools is more expensive (in cost and performance) so it will normally include only critical and warning non verbose information.

In our samples we enable openteemetry by means of the AddObservability() extension method that essentially:

• Configures Opentelemetry options
• Registers Opentelemetry logging provider
• Configures tracing to the remote tools
• Configures metrics to the remote tools

details about opentelemetry configuration is available here: HowTo: Configure diginsight telemetry to the remote tools.

Previous versions of diginsight

diginsight v3 is now available
the following article describes improvements of diginsight v3 over the previous version. Introduction to Diginsight v3.
Features such as old frameworks support (eg. .Net Framework 4.5+) or observability for startup and static sections may still have limited support on Diginsight v3.
In these cases the developer may decide to keep the old model.
Diginsight v2 will not be discontinued until feature parity is reached.
Documentation about v2 packages is still available here Diginsight v2 documentation.

SUMMARY

diginsight telemetry is a set .Net packages that that provides automatic observability for dotnet applications.
In particular, the full application flow is made available to local text based streams such as traditional file logs, the Console log or the Azure Streaming log.

Enabling Opentelemetry, the same information can be made available to remote tools for troubleshooting or performance analysis such as Azure Monitor or Grafana.

SAMPLES

You can start using diginsight telemetry by running the samples on the telemetry_samples repository.

Article HOWTO - Use Diginsight Samples: explores how we can use diginsight samples to test and understand integration of Diginsight telemetry in our own projects.

Build and Test

You can easily test Diginsight integration with OpenTelemetry by means of the EasySampleBlazorv2 project:

• Clone diginsight repository
• Open and build solution Diginsight.sln.
• build the solution

Contribute

Contribute to the repository with your pull requests.

• ASP.NET Core
• Visual Studio Code

License

See the LICENSE file for license rights and limitations (MIT).