[Bug] [BanyanDB] Metrics breaks periodically

[wu-sheng]

Search before asking

• I had searched in the issues and found no similar issues.

Apache SkyWalking Component

BanyanDB (apache/skywalking-banyandb)

What happened

Since we switched demo database to BanyanDB, most things work well, but I noticed this kind of metric breaking.

Meanwhile, this only happens on minute level,

All services/pods of demo env seem normal

But we noticed this error happens periodically too in the OAP pod logs.

I also checked the BanyanDB client codes, although we don't expose the configuration to BanyanDB provider(@lujiajing1126 we should expose), but the default value is 50m.
But this error log says, ~5m. So, @hanahmily @lujiajing1126 @ButterBright Is the server missing the configurations? And could you check whether there is other reason to make metrics breaking.

What you expected to happen

Make metrics continuously flush into the storage.

How to reproduce

I didn't reproduce this, but observe this on demo env.

Anything else

No response

Are you willing to submit a pull request to fix on your own?

• Yes I am willing to submit a pull request on my own!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Superskyyy]

Seems like a client side problem, the 50MB didn't take effect.

[wu-sheng]

I didn't check that deeply. But it seems existing in the default constructor. Doesn't it?

[hanahmily]

I checked the client, and it seems that the error cannot be from the BanyanDB client.

The error is related to the 'getCompressedBody' function, but the client should always use the 'getUncompressedBody' instead. BTW, maxInboundMessageSize is 50MB even though the client doesn't use it here.

[wu-sheng]

@wankai123 Let us bump up otel collectoe first. This seems irrelevant with DB.