[VictoriaLogs] Support count unique values

[lujiajing1126]

Describe Your Changes

In the contemporary logging UI, it is very important to understand the count/distribution of unique values (cardinality) of a given field within the query range.

The screenshot shows the SLS product on AliCloud.

For example,

1. if developers want to check error logs, normally stats for level fields is necessary,
2. If developers want to filter logs based on logger (Java classes) names, then stats for logger is necessary,

Checklist

The following checks are mandatory:

• My change adheres VictoriaMetrics contributing guidelines.

[lujiajing1126]

Currently, if multiple fields are used, all unique values will be counted in a single map

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 158 lines in your changes are missing coverage. Please review.

Project coverage is 55.18%. Comparing base (8aaa828) to head (f52caf1).
Report is 697 commits behind head on master.

Files	Patch %	Lines
lib/logstorage/stats_uniq_values_count.go	0.00%	153 Missing ⚠️
lib/logstorage/pipe_stats.go	0.00%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6287      +/-   ##
==========================================
- Coverage   60.37%   55.18%   -5.20%     
==========================================
  Files         411      587     +176     
  Lines       76609    78127    +1518     
==========================================
- Hits        46253    43114    -3139     
- Misses      27794    32030    +4236     
- Partials     2562     2983     +421     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[valyala]

@lujiajing1126 , thanks for the pull request! As I understood it correctly, stats uniq_values_count(field_name) works in the way similar to stats by (field_name) count() hits | sort by (hits desc) .

Note also that VictoriaLogs gained support for hits API, which can return the distribution of matching logs over the selected time range, with optional grouping by arbitrary log fields - see these docs.

Do you still think that it is necessary to add uniq_values_count() stats function? If yes, then please rebase the pull request on the latest commit in master branch.

[lujiajing1126]

@lujiajing1126 , thanks for the pull request! As I understood it correctly, stats uniq_values_count(field_name) works in the way similar to stats by (field_name) count() hits | sort by (hits desc) .

Note also that VictoriaLogs gained support for hits API, which can return the distribution of matching logs over the selected time range, with optional grouping by arbitrary log fields - see these docs.

Do you still think that it is necessary to add uniq_values_count() stats function? If yes, then please rebase the pull request on the latest commit in master branch.

Thanks for your reply! I think stats by (field_name) count() hits | sort by (hits desc) works well for us. I will close this PR.