Diagnose issues using Telemetry
Diagnose and monitor the backend services and Data-Accelerator's Data Processing component using ApplicationInsights
If for some reason, one of the "Deploy" or "GetSchema" (etc.) buttons are giving an error in the Data Accelerator portal, for diagnosing such an issue, it is extremely useful to look at the telemetry logs via ApplicationInsights. This can prevent having to setup the debugging environment. Just by looking at the logs, one can figure out the issue that may be happening.
Currently, the backend services and DataAccelerator's DataProcessing component send ApplicationInsights telemetry.
To access the logs, you can go to the portal.azure.com and open up the Applicationinsights instance--> click on Analytics
In the Data Accelerator portal if you are trying to GetSchema, you can look at the ApplicationInsights logs and get details for this call (especially in case you run into an error). For looking at the logs you need to go to the Azure portal --> ApplicationInsights resource and click on Analytics. In this case, you are in the Input tab on Data Accelerator portal. Let's say if you entered the wrong connection string, the backend api will return an error message. Based on this error message, you can double check the entered connectionString on the DataAccelerator portal and modify / update it.
- Look at the Error logs:
exceptions | order by timestamp desc | where operation_Name contains "POST SchemaInference/GetInputSchema"
Example output:
- Look at the traces logs:
traces | order by timestamp desc | where operation_ParentId contains "datax-service" | where operation_Name contains "POST SchemaInference/GetInputSchema"
Please refer to the tutorial on Live Query Once you click on an existing flow, the Jupyter kernel is initialized. If there was an error / exception while the kernel was getting initialized, you will see an error on the query tab. For more details, you can go to ApplicationInsights on the Azure portal and issue the below queries to look at the telemetry data. This will help you debug the issue. The recommendation is that if the kernel has not initialized correctly. Steps to follow:
- Click on GetSchema in the Input tab. If there is an error, check Application Insights Telemetry to debug why GetSchema is failing
- Once GetSchema succeeds, this will automatically trigger kernel initialization. If there is an error, go to the Azure portal --> ApplicationInsights resource and click on Analytics.
- Look at the Error logs:
exceptions | order by timestamp desc | where operation_Name contains "POST InteractiveQuery/CreateAndInitializeKernel"
Example output:
- Look at the traces logs:
traces | order by timestamp desc | where operation_ParentId contains "datax-service" | where operation_Name contains "POST InteractiveQuery/CreateAndInitializeKernel"
In the Data Accelerator portal if you are trying to Deploy your flow, you can look at the ApplicationInsights logs and get details for this call (especially in case you run into an error). For looking at the logs you need to go to the Azure portal --> ApplicationInsights resource and click on Analytics. In this case, you are in the Input tab on Data Accelerator portal. Let's say if you entered the wrong connection string, the backend api will return an error message.
- Look at the Error logs:
exceptions | order by timestamp desc | where operation_ParentId contains "datax-service" | where operation_Name contains "FlowManagement"
We are using the ILogger implementation and the ILogger instance (_logger) is available in each of the classes in the backend services. Example:
_logger.LogInformation($"Event count = {events.Count}");
Whereever you feel that the logs should be added, a similar call can be made.
For each of the backend services, for example, DataX.Flow service you can filter additional telemetry in StartUpUtil.cs.
services.AddLogging(logging =>
{
try
{
// In order to log ILogger logs
logging.AddApplicationInsights();
// Optional: Apply filters to configure LogLevel Information or above is sent to
// ApplicationInsights for all categories.
logging.AddFilter<ApplicationInsightsLoggerProvider>("", LogLevel.Information);
// Additional filtering For category starting in "Microsoft",
// only Warning or above will be sent to Application Insights.
logging.AddFilter<ApplicationInsightsLoggerProvider>("Microsoft", LogLevel.Warning);
}
Please note: This will require a redeployment of the services.