Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Metricbeat returns incorrect, or misleading, disk usage percentage on linux filesystems #5494

Closed
hogarthj opened this issue Nov 1, 2017 · 3 comments
Closed

Metricbeat returns incorrect, or misleading, disk usage percentage on linux filesystems #5494

hogarthj opened this issue Nov 1, 2017 · 3 comments
Labels
bug Metricbeat Metricbeat module

Comments

@hogarthj
Copy link

hogarthj commented Nov 1, 2017

Details posted to discuss before filing the bug as required:

https://discuss.elastic.co/t/bug-incorrect-disk-usage-statistics/103207

For confirmed bugs, please report:

  • Version: 5.6.0
  • Operating System: RHEL 6.8.z
  • Steps to Reproduce:
    • Check the values returned on a system in system.filesystem.used.pct
    • Compare to df on a target system
    • Observe the 5% difference (by default on ext4... may be different on other filesystems dependent on implementation and configuration of reserved space)
@troelsh
Copy link

troelsh commented Jan 29, 2018 •
edited
Loading

I can confirm this:

  • Metricbeat version 6.1.2 (amd64), libbeat 6.1.2
  • OS: Ubuntu Server 16.04.2 LTS @ Kernel 4.4.0-31-generic

df -h shows 74% usage on root /
Metricbeat (system.filesystem.used.pct) shows 69% used*

@kvch kvch added bug Metricbeat Metricbeat labels Jan 29, 2018
@ruflin ruflin added the module label Jan 30, 2018
@andrewkroh andrewkroh mentioned this issue Jan 30, 2018
Merged
@andrewkroh
Copy link
Member

I opened #6231 to address this issue. Once this is merged it would be great if you all could test the snapshot build and verify the system.filesystem.used.pct values against df.

andrewkroh added a commit to andrewkroh/beats that referenced this issue Jan 31, 2018
@andrewkroh
Fix filesystem used percentage
51c7249 
The used percentage reported by Metricbeat does not match what is reported by `df`. Metricbeat's computation of the `system.filesystem.used.pct` value was based on the total number of bytes, but df uses the total number of bytes available to the unprivileged user.

| FS       | Blocks      | Used        | Available  | Use % | Mount | Metricbeat Used % After This Change | Before This Change |
|----------|-------------|-------------|------------|-------|-------|-------------------------------------|--------------------|
| /dev/vda | 21137846272 | 16992858112 | 3071246336 | 85    | /     | 0.846928312                         | 0.8039067885       |

Fixes elastic#5494
ruflin pushed a commit that referenced this issue Feb 3, 2018
@andrewkroh @ruflin
Fix filesystem used percentage (#6231)
e605495 
The used percentage reported by Metricbeat does not match what is reported by `df`. Metricbeat's computation of the `system.filesystem.used.pct` value was based on the total number of bytes, but df uses the total number of bytes available to the unprivileged user.

| FS       | Blocks      | Used        | Available  | Use % | Mount | Metricbeat Used % After This Change | Before This Change |
|----------|-------------|-------------|------------|-------|-------|-------------------------------------|--------------------|
| /dev/vda | 21137846272 | 16992858112 | 3071246336 | 85    | /     | 0.846928312                         | 0.8039067885       |

Fixes #5494
@troelsh
Copy link

troelsh commented Feb 8, 2018

@andrewkroh i will try and get some time to testing this. As i normally use the packaged versions i will have to setup a test env for this so might take some time before i can test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Metricbeat Metricbeat module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@ruflin @troelsh @hogarthj @kvch @andrewkroh