Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[broker] Output resource usage rate to log on broker #6152

Merged
merged 2 commits into from
Feb 8, 2020

Conversation

massakam
Copy link
Contributor

Motivation

When a broker is under heavy load, the following log may be output and some topics may be unloaded.

Attempting to shed load on broker101.pulsar.xxx.yahoo.co.jp:4080, which has max resource usage above threshold 0.8708186149597168% > 0.85% -- Offloading at least 0.36224863451117845 MByte/s of traffic

This log means that the usage rate of CPU, memory, direct memory, input bandwidth, or output bandwidth has exceeded the threshold, but we don't know which resource usage is high.

Modifications

Output these resource usages along with the above log.

Attempting to shed load on broker101.pulsar.xxx.yahoo.co.jp:4080, which has resource usage 87.08% above threshold 85.0% -- Offloading at least 0.36224863451117845 MByte/s of traffic (cpu: 87.08%, memory: 12.71%, directMemory: 17.19%, bandwidthIn: 11.28%, bandwidthOut: 0.00%)

@massakam massakam added type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages area/broker release/2.5.1 labels Jan 28, 2020
@massakam massakam added this to the 2.6.0 milestone Jan 28, 2020
@massakam massakam self-assigned this Jan 28, 2020
@massakam
Copy link
Contributor Author

rerun integration tests

@massakam
Copy link
Contributor Author

rerun integration tests

1 similar comment
@massakam
Copy link
Contributor Author

rerun integration tests

@massakam
Copy link
Contributor Author

rerun integration tests

1 similar comment
@massakam
Copy link
Contributor Author

rerun integration tests

@massakam massakam changed the title Output resource usage rate to log on broker [broker] Output resource usage rate to log on broker Feb 5, 2020
@sijie sijie merged commit 9b296d8 into apache:master Feb 8, 2020
@massakam massakam deleted the print-resource-usage branch February 8, 2020 16:53
tuteng pushed a commit to AmateurEvents/pulsar that referenced this pull request Feb 23, 2020
### Motivation

When a broker is under heavy load, the following log may be output and some topics may be unloaded.

> Attempting to shed load on broker101.pulsar.xxx.yahoo.co.jp:4080, which has max resource usage above threshold 0.8708186149597168% > 0.85% -- Offloading at least 0.36224863451117845 MByte/s of traffic

This log means that the usage rate of CPU, memory, direct memory, input bandwidth, or output bandwidth has exceeded the threshold, but we don't know which resource usage is high.

### Modifications

Output these resource usages along with the above log.

> Attempting to shed load on broker101.pulsar.xxx.yahoo.co.jp:4080, which has resource usage 87.08% above threshold 85.0% -- Offloading at least 0.36224863451117845 MByte/s of traffic (cpu: 87.08%, memory: 12.71%, directMemory: 17.19%, bandwidthIn: 11.28%, bandwidthOut: 0.00%)
tuteng pushed a commit to AmateurEvents/pulsar that referenced this pull request Mar 21, 2020
### Motivation

When a broker is under heavy load, the following log may be output and some topics may be unloaded.

> Attempting to shed load on broker101.pulsar.xxx.yahoo.co.jp:4080, which has max resource usage above threshold 0.8708186149597168% > 0.85% -- Offloading at least 0.36224863451117845 MByte/s of traffic

This log means that the usage rate of CPU, memory, direct memory, input bandwidth, or output bandwidth has exceeded the threshold, but we don't know which resource usage is high.

### Modifications

Output these resource usages along with the above log.

> Attempting to shed load on broker101.pulsar.xxx.yahoo.co.jp:4080, which has resource usage 87.08% above threshold 85.0% -- Offloading at least 0.36224863451117845 MByte/s of traffic (cpu: 87.08%, memory: 12.71%, directMemory: 17.19%, bandwidthIn: 11.28%, bandwidthOut: 0.00%)

(cherry picked from commit 9b296d8)
tuteng pushed a commit that referenced this pull request Apr 13, 2020
### Motivation

When a broker is under heavy load, the following log may be output and some topics may be unloaded.

> Attempting to shed load on broker101.pulsar.xxx.yahoo.co.jp:4080, which has max resource usage above threshold 0.8708186149597168% > 0.85% -- Offloading at least 0.36224863451117845 MByte/s of traffic

This log means that the usage rate of CPU, memory, direct memory, input bandwidth, or output bandwidth has exceeded the threshold, but we don't know which resource usage is high.

### Modifications

Output these resource usages along with the above log.

> Attempting to shed load on broker101.pulsar.xxx.yahoo.co.jp:4080, which has resource usage 87.08% above threshold 85.0% -- Offloading at least 0.36224863451117845 MByte/s of traffic (cpu: 87.08%, memory: 12.71%, directMemory: 17.19%, bandwidthIn: 11.28%, bandwidthOut: 0.00%)

(cherry picked from commit 9b296d8)
jiazhai pushed a commit to jiazhai/pulsar that referenced this pull request May 18, 2020
### Motivation

When a broker is under heavy load, the following log may be output and some topics may be unloaded.

> Attempting to shed load on broker101.pulsar.xxx.yahoo.co.jp:4080, which has max resource usage above threshold 0.8708186149597168% > 0.85% -- Offloading at least 0.36224863451117845 MByte/s of traffic

This log means that the usage rate of CPU, memory, direct memory, input bandwidth, or output bandwidth has exceeded the threshold, but we don't know which resource usage is high.

### Modifications

Output these resource usages along with the above log.

> Attempting to shed load on broker101.pulsar.xxx.yahoo.co.jp:4080, which has resource usage 87.08% above threshold 85.0% -- Offloading at least 0.36224863451117845 MByte/s of traffic (cpu: 87.08%, memory: 12.71%, directMemory: 17.19%, bandwidthIn: 11.28%, bandwidthOut: 0.00%)
(cherry picked from commit 9b296d8)
huangdx0726 pushed a commit to huangdx0726/pulsar that referenced this pull request Aug 24, 2020
### Motivation

When a broker is under heavy load, the following log may be output and some topics may be unloaded.

> Attempting to shed load on broker101.pulsar.xxx.yahoo.co.jp:4080, which has max resource usage above threshold 0.8708186149597168% > 0.85% -- Offloading at least 0.36224863451117845 MByte/s of traffic

This log means that the usage rate of CPU, memory, direct memory, input bandwidth, or output bandwidth has exceeded the threshold, but we don't know which resource usage is high.

### Modifications

Output these resource usages along with the above log.

> Attempting to shed load on broker101.pulsar.xxx.yahoo.co.jp:4080, which has resource usage 87.08% above threshold 85.0% -- Offloading at least 0.36224863451117845 MByte/s of traffic (cpu: 87.08%, memory: 12.71%, directMemory: 17.19%, bandwidthIn: 11.28%, bandwidthOut: 0.00%)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/broker release/2.5.1 type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants