question about your paper's results

[ResearcherLifeng]

Hi! Dear authors,

I have a question about your reported results.

I have tested your OmniAnomaly model on the dataset MSL and I can get 89% Pot-F1 which is much closer to the result reported in your paper.
But, when I set the model's anomaly score as random numbers from [0, 1], the pot-F1 can reach above 89.8833%. it is confusing since these random "anomaly scores" are not obtained from the model.

I think this issue should be caused by the point-adjust approach mentioned in your paper.
Actually, I also try to evaluate a simple RNN with your codes and settings (same data and same evaluation), its best F1 can also be above 90%.

Can you help to explain my question? Many thanks.

[sashastrelnikoff]

Hi! Dear authors,

I have a question about your reported results.

I have tested your OmniAnomaly model on the dataset MSL and I can get 89% Pot-F1 which is much closer to the result reported in your paper. But, when I set the model's anomaly score as random numbers from [0, 1], the pot-F1 can reach above 89.8833%. it is confusing since these random "anomaly scores" are not obtained from the model.

I think this issue should be caused by the point-adjust approach mentioned in your paper. Actually, I also try to evaluate a simple RNN with your codes and settings (same data and same evaluation), its best F1 can also be above 90%.

Can you help to explain my question? Many thanks.

Hi there, I was just wondering why this question was marked as close without receiving an answer? Did you happen to figure out whether there was an issue with the POT scores or if this is the expected behaviour?

[mirmuss]

+?

Hi! Dear authors,
I have a question about your reported results.
I have tested your OmniAnomaly model on the dataset MSL and I can get 89% Pot-F1 which is much closer to the result reported in your paper. But, when I set the model's anomaly score as random numbers from [0, 1], the pot-F1 can reach above 89.8833%. it is confusing since these random "anomaly scores" are not obtained from the model.
I think this issue should be caused by the point-adjust approach mentioned in your paper. Actually, I also try to evaluate a simple RNN with your codes and settings (same data and same evaluation), its best F1 can also be above 90%.
Can you help to explain my question? Many thanks.

Hi there, I was just wondering why this question was marked as close without receiving an answer? Did you happen to figure out whether there was an issue with the POT scores or if this is the expected behaviour?