Add backlogSize in topicStats

[k2la]

Fix storageSize and add backlogSize in topicStats

Modifications

• Add backlogSize to topicStats
• same as Fix storage size always 0 without subscription. #5108

[k2la]

retest this please

[codelipenghui]

retest this please

[massakam]

rerun integration tests
rerun java8 tests

[massakam]

rerun integration tests

[k2la]

rerun integration tests

[codelipenghui]

rerun integration tests

[sijie]

run integration tests

[k2la]

rerun integration tests

[codelipenghui]

rerun integration tests

[rdhabalia]

this is not a good practice to break the metrics compatibility. After upgrading the release, it broke our users monitoring and it's not acceptable. we can always add new metrics if needed but it's better to avoid changing existing behavior as there are monitoring and alerts setup of existing metrics.
I would like to revert this change. Please let me know if anyone has any concerns?

[k2la]

@rdhabalia
In my modification, I modified "storageSize" and "backlogSize" based on #5108.
I think if you revert mine, you need to fix #5108 as well.

Also, mine is included in 2.5.1 or later.
the compatibility will be broken by reverting it, is that ok?

[rdhabalia]

the compatibility will be broken by reverting it, is that ok?

Compatibility was broken in 2.5 and it can be fixed in later version.

[codelipenghui]

@rdhabalia If revert this change, we will break the behavior for version 2.5.x - 2.7.x?

[eolivelli]

@rdhabalia probably it is too late.
We should keep now compatibility with at least 2.6.x and 2.7.x.

[rdhabalia]

it has already broken for all the systems which have been using Pulsar for last 7 years. if any system was using the metrics before 2.5, definitely needs this compatibility back and the one which is not using it, for it this doesn't matter. Introducing incompatibility and not handling is not a good practice. so, it's never late to make things right.

[codelipenghui]

@rdhabalia Could you please start an email thread for discussion? As far as I understand, most of the users are using 2.5.x 2.6.x 2.7.x. If there are 80% users working on 2.5.x - 2.7.x, I don't think revert this change is a good choice, it will break again.

[eolivelli]

So the problem is for people upgrading from 2.4.

I agree that we must not change the meaning of metrics and APIs.

Probably it is better to not do now the mistake we did in the past with this PR.

Users coming from 2.4 will have to update their monitoring systems.
I don't know how many users are impacted by this change nowadays

[rdhabalia]

Users coming from 2.4 will have to update their monitoring systems. I don't know how many users are impacted by this change nowadays

I don't agree and it's not a responsible statement to say that it's fine to let old users suffer from the change.