After upgrading td-agent from version 3.8.0 to 4.1.0, cpu utilization of td-agent got increased

[VarunSrinivasa]

Describe the bug

Td-agent of version 3.8.0 and elasticsearch of version 7.8.0 is deployed in kubernetes environment. After upgrading td-agent to verison 4.1.0, we observe that there cpu utilization is very high

1. In the fluentd version 3.8.0 , average cpu utilization is 0.4 core
2. In the fluentd version 4.1.0, average cpu utilization is 1 core

To Reproduce

Deploy fluentd of version 3.8.0 in kubernetes environment with cpu configuration of 1 core and 1gb ram with below configuration

After upgrading the fluentd version 4.1.0 with the same configuration cpu consumption of fluentd got increased to 1 core.

Expected behavior

Cpu consumption of fluentd should not increase after migrating from the version 3.8.0 to 4.1.0

Your Environment

- Fluentd version:
- TD Agent version:4.1.0
- Operating system: centos 7.9
- Kernel version:

Your Configuration

Your Error Log

There are no error messages

Additional context

No response

[ashie]

Could you check #3382?
Probably your issue is same with it.

[Dredd13]

@ashie it seems that there is a bug that it gets fixed with 4.2.0 version. Based on the configuration file given by Varun do you see any possible workaround, configuration wise?

[VarunSrinivasa]

Hi @ashie ,

Can you please confirm any this issue can be fixed by changing any configuration in td-agent or we have to upgrade to fluentd 4.2.0 ?

Regards,
Varun S

[ashie]

Sorry, it seems a different issue.
td-agent 4.1.1 (Ruby 2.7.3 & excon 0.80.1) is affected by #3382 but td-agent 4.1.0 (Ruby 2.7.2 & excon 0.78.1) isn't.

[VarunSrinivasa]

Hi @ashie ,

td-agent 4.1.0 (Ruby 2.7.2 & excon 0.78.1) --->Yes

Sorry td-agent version used here is td-agent-4.1.0-1

Regards,
Varun S

[VarunSrinivasa]

Hi @ashie,

With the above configuration we are not facing any issues so closing this issue, But we are facing issue when http plugin is used. So different issue is raised for the same. Please find the bug id:- #3487 .