Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should we change the category filters for AppInsights and disable Sampling by default? #18

Closed
fume opened this issue Jun 9, 2022 · 1 comment · Fixed by #28
Closed
Labels
enhancement New feature or request question Further information is requested

Comments

@fume
Copy link
Collaborator

fume commented Jun 9, 2022

TL;DR;
I'm willing to reduce the logs sent to AppInsights changing the category filters. At the same time i would like to disable the AdaptiveSampling by default to avoid loosing important logs which are mandatory for AgID.

Details
With the actual default configuration, we send Information logs to AppInsights for every single category:

{
	"Logging": {
		"ApplicationInsights": {
			"LogLevel": {
				"Default": "Information"
			}
		}
       }
}

With this configuration we are also sending framework logs such as the following, which are not really useful and could produce a lot of data-ingestion
image

Should we change the category filters for AppInsights to send Information logs and above for the Microsoft.SPID cateogory and sending only Warning and above logs for the Default category? Something like this:

{
	"Logging": {
		"ApplicationInsights": {
			"LogLevel": {
				"Default": "Warning",
                                "Microsoft.SPID": "Information"
			}
		}
       }
}

We could increase the verbosity at any-time by overriding the setting via Configuration.
I would like to reduce the telemetry sent to AppInsights because AgID requires to store logs for 24 months. We can change the retention of the Application Insights instance to achieve the objective (via the linked LAW or on the AppInsights instance directly), but it has an extra-cost per-GB/per-Month.

Moreover, we have Adaptive Sampling enabled by default. Whene there are a lot of concurrent requets towards the SPIDProxy, sampling is applied hence we loose log details but still have correct numbers (thanks to the itemCount). I would like to disable Sampling by default since we could sample a log which is required by AgID and which we should retain for 24 months.
We can do so by adding the following in the appsettings.json, as also shown here https://docs.microsoft.com/en-us/azure/azure-monitor/app/asp-net-core#configuration-recommendation-for-microsoftapplicationinsightsaspnetcore-sdk-2150-and-later:

{
    "ApplicationInsights": {
        "EnableAdaptiveSampling": false
    }
}

What do you think @tommasodotNET, @MarcoZama, @PaoloCastAway ?

@fume fume added enhancement New feature or request question Further information is requested labels Jun 9, 2022
@fume fume pinned this issue Jun 22, 2022
@MarcoZama
Copy link
Contributor

@fume Looks good for me!

@fume fume linked a pull request Jun 28, 2022 that will close this issue
@fume fume closed this as completed in #28 Jun 28, 2022
@fume fume unpinned this issue Jun 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants