Skip to content

Insufficient consumption performance leads to a large number of blocking situations in Redis. #1897

@wangyx89007

Description

@wangyx89007

The current status of my ES cluster:

12c16G x 3 version:5.6.9

Image

Image

Redis:

4c4g version:7.2.1
Image
Among these four jobs, different Redis databases were used. For instance, the first job utilized db1, the second used db2, and so on. The figure shows the queue situation of one of the databases. In fact, this is the case for all four databases.
Queue situation
Image

Exceptionless:

A total of 4 jobs are running on a 4-core 32GB Windows server.

Image

Image

This problem has been bothering me for a week. Even though Redis was upgraded from 2c4g to 4c4g, it still didn't work. Similarly, Elasticsearch was upgraded from 4c8G to its current state, but it also failed. When attempting to run 7 instances of the job, there was still a significant write delay, with the delay being around 1 hour. Not all logs had the delay, but some log queues were blocked. This was because in Exceptionless, querying logs always returned the latest <= 1 second, but it was incomplete because some were still blocked.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions