Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

metricbeat - metricbeat stops publishing events to Redis #5404

Closed
alextxm opened this issue Oct 18, 2017 · 3 comments

Comments

@alextxm
Copy link

commented Oct 18, 2017

Metricbeat 6.0.0-rc1 stops publishing events to Redis (using the redis output module) after some time, depending on the events volume, but usually in the 4000-5000 events range.

Enabling debugging does not give any futher info; after reaching 4000-5000 events metricbeat continue running but no events are published to redis while the log is filled with INFO level messages like this one:
INFO Non-zero metrics in the last 30s: beat.memstats.memory_alloc=7872 beat.memstats.memory_total=7872

It should be noted that metricbeat successfully works when configured to use Logstash publishing instead (using the logstash output module).
The Redis output module configuration uses default values except for the following customized values: hosts, port, password, key. i've also tried raising the bulk_max_size value but nothing changed.

Tests were performed enablding only the system module, only the vsphere module and finally both of them; all the configurations triggered the problem; please also note the same configurations worked fine in 6.0.0-alpha2.

Tested with:

  • metricbeat 6.0.0-rc1 on Windows Server 2008R2
  • redis 3.0.2 on Ubuntu 16.04
  • redis 4.0.2 on Ubuntu 16.04
  • logstash 5.4.0 on Ubuntu 16.04

@ruflin ruflin added the Metricbeat label Oct 20, 2017

@alextxm

This comment has been minimized.

Copy link
Author

commented Oct 23, 2017

I've done some more testing and it happens on both 6.0-beta1/beta2/rc1 but not alpha2 (alpha2 works fine). Publishing stops just after 4096 items as you can see per this screenshots taken from redis-cli.
mb-redis-60b1
mb-redis-60b2

P.S.: tests were performed with: only events from the vsphere module, only events with the system module, events from both modules; the screenshots are from tests with the vsphere modules.

Could please someone check this issue?
Thanks in advance

@urso

This comment has been minimized.

Copy link
Collaborator

commented Oct 25, 2017

Thanks for testing. Can you share your redis config? Are you using a list or a channel?

@urso

This comment has been minimized.

Copy link
Collaborator

commented Oct 25, 2017

I was able to reproduce the problem. It should be resolved with #5441. Thanks for testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.