Report model change annotations via result stream

[przemekwitek]

This PR makes model change annotations reported via result stream.
It is achieved the following way:

• model classes from maths module report model changes via TModelChangeCallback functional type
• concrete instantiation of TModelChangeCallback creates a CAnnotation object and puts it into in-memory collection m_CurrentBucketStats.s_Annotations
• annotations from models' s_Annotations are collected by CAnomalyJob and written out to the stream using CAnnotationJsonWriter class

Relates elastic/elasticsearch#55781

[droberts195]

The memory of m_CurrentBucketStats.s_Annotations needs to be accounted for. So everywhere where we have this:

bash-3.2$ grep 'mem += core::CMemory::dynamicSize(m_CurrentBucketStats.s_InterimCorrections);' *
CEventRateModel.cc:    mem += core::CMemory::dynamicSize(m_CurrentBucketStats.s_InterimCorrections);
CEventRatePopulationModel.cc:    mem += core::CMemory::dynamicSize(m_CurrentBucketStats.s_InterimCorrections);
CMetricModel.cc:    mem += core::CMemory::dynamicSize(m_CurrentBucketStats.s_InterimCorrections);
CMetricPopulationModel.cc:    mem += core::CMemory::dynamicSize(m_CurrentBucketStats.s_InterimCorrections);

and this:

bash-3.2$ grep 'core::CMemoryDebug::dynamicSize("m_CurrentBucketStats.s_InterimCorrections",' *
CEventRateModel.cc:    core::CMemoryDebug::dynamicSize("m_CurrentBucketStats.s_InterimCorrections",
CEventRatePopulationModel.cc:    core::CMemoryDebug::dynamicSize("m_CurrentBucketStats.s_InterimCorrections",
CMetricModel.cc:    core::CMemoryDebug::dynamicSize("m_CurrentBucketStats.s_InterimCorrections",
CMetricPopulationModel.cc:    core::CMemoryDebug::dynamicSize("m_CurrentBucketStats.s_InterimCorrections",

we need equivalent lines for m_CurrentBucketStats.s_Annotations.

TODO1: when to clear s_Annotations vector?

Am I correct that at the moment it's never cleared?

One option might be to have a method that allows the annotations to be taken instead of got as a const reference. Then after outputting what's been built up they are all gone - no danger of outputting duplicates and no memory build up as long as the output method is called reasonably frequently.

For example:

CEventRateModel::TAnnotationVec CEventRateModel::takeAnnotations() {
    CEventRateModel::TAnnotationVec result;
    result.swap(m_CurrentBucketStats.s_Annotations);
    return result;
 }

Maybe Tom has a better idea though.

[tveasey]

I've done a first pass through. I've made a few suggestions. Aside from defaulting the add annotation callback better, my biggest suggested change is to rejig where the annotations for new decomposition components are written. Also, this should be capturing when a trend is detected and when calendar components are detected. It is probably easiest to discuss this offline.

[tveasey]

It would be good to test, in at least one test, that the annotations are actually created.

[przemekwitek]

How should I go about the unit testing in this case? Should I call addPoint method with a model annotation callback and verify it got called?
Could you point me at the test case which I can take as a base?

[przemekwitek]

This PR needs a changelog entry to make the changelog PR check succeed.

Done.

[przemekwitek]

The memory of m_CurrentBucketStats.s_Annotations needs to be accounted for.

Done.

Am I correct that at the moment it's never cleared?

I've added clearing in the beginning of the sample method in all the models.
By Tom's comment: "I think we should clear the m_CurrentBucketStats.s_Annotations at the start of this function. It'll get called exactly once per bucket and we'll write them out between calls."

IIUC this solution is equivalent to the one you outlined (using swap).

[tveasey]

Thanks @przemekwitek! This looks really close. I have some suggestions on naming: the current choices I think are easy to confuse with change point detection. Other than that I think this looks ready.

[tveasey]

Good stuff @przemekwitek. Thanks for iterating on this! LGTM.