Count aggregation in Azure not measuring count

[cameron-stevenson-seequent]

Describe your environment.

python 3.9.0
opencensus-ext-azure==1.1.0
Azure Monitor

Steps to reproduce.
Export a metric once to Azure using the sample program https://docs.microsoft.com/en-us/azure/azure-monitor/app/opencensus-python.

from datetime import datetime
from opencensus.ext.azure import metrics_exporter
from opencensus.stats import aggregation as aggregation_module
from opencensus.stats import measure as measure_module
from opencensus.stats import stats as stats_module
from opencensus.stats import view as view_module
from opencensus.tags import tag_map as tag_map_module

stats = stats_module.stats
view_manager = stats.view_manager
stats_recorder = stats.stats_recorder

prompt_measure = measure_module.MeasureInt("prompts",
                                           "number of prompts",
                                           "prompts")
prompt_view = view_module.View("prompt view",
                               "number of prompts",
                               [],
                               prompt_measure,
                               aggregation_module.CountAggregation())
view_manager.register_view(prompt_view)
mmap = stats_recorder.new_measurement_map()
tmap = tag_map_module.TagMap()

exporter = metrics_exporter.new_metrics_exporter(
    connection_string='InstrumentationKey=00000000-0000-0000-0000-000000000000')

view_manager.register_exporter(exporter)

def prompt():
    input("Press enter.")
    mmap.measure_int_put(prompt_measure, 1)
    mmap.record(tmap)
    metrics = list(mmap.measure_to_view_map.get_metrics(datetime.utcnow()))
    print(metrics[0].time_series[0].points[0])

def main():
    while True:
        prompt()

if __name__ == "__main__":
    main()

Wait a few minutes for aggregation to happen.
In Azure Monitor Metrics, plot the metric with count aggregation.

What is the expected behavior?
A temporary spike in the metric, returning to zero after the aggregation period in azure passes. For example this standard metric for azure web apps:

What is the actual behavior?
The metric rises the first time it is exported, and never returns to zero. Subsequently incrementing the same metric does not increase the aggregated count reported by Azure.

Additional context.
From playing around with the export interval, it seems the aggregated count in Azure is measuring how many times exporters emit their repeated updates within the aggregation period, rather than the count being recorded by them. Hence incrementing the same metric does not change the aggregation reported by Azure as the existing exporter for the metric continues to update with the same interval, and the thread stays alive so count never drops to zero. Hoping to find out if this graphed aggregation behaviour is the same on other services and if it is intended?

[PlagueHO]

Has anyone else experienced this issue or similar?

[cameron-stevenson-seequent]

Some more things I have found:

• This occurs in Google Cloud Monitoring as well
• With a telemetry processor I checked the envelopes going through the azure exporter, and "count" was set to None. I modified this and whatever number I put in there was correctly aggregated in Azure Monitor, except for 0 which appears to default to 1
• A workaround is to manually keep a count since last export for each set of tag map values, then set the envelope count to this each export. This does not get around the 0 defaulting to 1 problem though. Could try something similar with last value aggregation if it doesn't have this behaviour.

So I think there is an issue somewhere where this envelope "count" should be set to the count aggregated since last export, and the value 0 should be accepted rather than defaulting to 1.

[lzchen]

You can modify count through the telemetry processor as proposed above. Count is not automatically tracked by the SDK. As for the 0 defaulting to 1, this is behavior done by the application insights backend. I believe count is used in calculations for sampling (total/count) so count cannot be 0.

[cameron-stevenson-seequent]

Thanks for clarifying @lzchen. The workaround I ended up using was to track count since last export for every name + tag map values combination, then use a telemetry processor to overwrite "value" in the envelopes being exported (this allows for 0 values). In the Azure side of things, for time series I would plot with "Sum" aggregation to get a rate graph. With queries I would sum "valueSum" across records within a time period to get total count within that interval.