Promtool duplicates error message if multiple alertnames are evaluated at same time

[mwalters-workmarket]

Minimal example:

rules.yaml:

groups:
  - name: MyRules
    rules:
      - alert: AnAlert
        expr: abc{} > 2
        for: 1m
      - alert: AnotherAlert
        expr: def{} > 2

test.yaml:

rule_files:
  - rules.yaml
evaluation_interval: 1m
tests:
  - interval: 1m
    input_series:
      - series: 'abc{}'
        values: '1'
      - series: 'def{}'
        values: '3'
    alert_rule_test:
      - alertname: AnAlert
        eval_time: 1m
      - alertname: AnotherAlert
        eval_time: 1m

when I execute promtool test rules test.yaml
I expect to see output:

Unit Testing:  test.yaml
  FAILED:
    alertname:AnotherAlert, time:1m0s, 
        exp:"[]", 
        got:"[Labels:{alertname=\"AnotherAlert\"} Annotations:{}]"

But the actual output is:

Unit Testing:  test.yaml
  FAILED:
    alertname:AnotherAlert, time:1m0s, 
        exp:"[]", 
        got:"[Labels:{alertname=\"AnotherAlert\"} Annotations:{}]"
    alertname:AnotherAlert, time:1m0s, 
        exp:"[]", 
        got:"[Labels:{alertname=\"AnotherAlert\"} Annotations:{}]"

It looks like the issue is occurring because in the test file, the eval_time for both rules is the same (1m)

[brian-brazil]

@codesome

[codesome]

Thanks @mwalters-workmarket for reporting this. Will look at it.