Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated Low Disk script (fix Open Deleted Files issue) #25

Merged
merged 2 commits into from
Jul 29, 2020
Merged

Updated Low Disk script (fix Open Deleted Files issue) #25

merged 2 commits into from
Jul 29, 2020

Conversation

LukeShirnia
Copy link
Collaborator

@LukeShirnia LukeShirnia commented Jul 27, 2020 •
edited
Loading 

subrepo:
  subdir:   "low_disk"
  merged:   "9bb783a"
upstream:
  origin:   "https://github.com/LukeShirnia/Low_Disk.git"
  branch:   "master"
  commit:   "9bb783a"
git-subrepo:
  version:  "0.4.1"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "a04d8c2"

[lukeshirnia@rax useful-scripts]$ ./audit.sh --update low_disk/

------------------------------------------
Verifying recorded commit...

Commit on file: 0b987573391aae086baa8e716084e60a9ff46053
https://raw.githubusercontent.com/rackerlabs/useful-scripts/master/low_disk/.gitrepo

Remote Repo Info (commit)
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/0b987573391aae086baa8e716084e60a9ff46053/disk_usage_check.sh
sha1sum: 90369db9ff2000a369fdc16acbcd14c180897dee

SubRepo (remote) Info (master)
https://raw.githubusercontent.com/rackerlabs/useful-scripts/master/low_disk/disk_usage_check.sh
sha1sum: 90369db9ff2000a369fdc16acbcd14c180897dee

Verifying commit: sha1sums match!
master (remote branch) is inline with specified commit hash
------------------------------------------

------------------------------------------
Checking if updates available... (script/commit vs script/master)

Remote 'Commit' URL
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/0b987573391aae086baa8e716084e60a9ff46053/disk_usage_check.sh
sha1sum: 90369db9ff2000a369fdc16acbcd14c180897dee

Remote 'Master' URL
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/master/disk_usage_check.sh
sha1sum: f05fee4dd51ce7967d9bf49574a145c157642661

Master and commit hash do NOT match
Updates: Possible updates available

Compare URL:
https://github.com/LukeShirnia/Low_Disk/compare/0b987573391aae086baa8e716084e60a9ff46053...master

------------------------------------------


Error: You appear to be working on the master branch!
Updating/adding should be added to a sub branch.

Would you like to autocreate a new branch (low_disk-update-2020-07-28)? (y|n) y
Branch Updated.....
Verifying Update.....
Cleaning up....
error: branch 'subrepo/low_disk/' not found.

------------------------------------------
Verifying recorded commit...

Commit on file: a0dd94ea13482ff169c39e19737d56652c9b1a97
/home/lukeshirnia/development/github/git_rackspace/useful-scripts/low_disk/.gitrepo

Remote Repo Info (commit)
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/a0dd94ea13482ff169c39e19737d56652c9b1a97/disk_usage_check.sh
sha1sum: f05fee4dd51ce7967d9bf49574a145c157642661

SubRepo (local) Info (low_disk-update-2020-07-28)
/home/lukeshirnia/development/github/git_rackspace/useful-scripts/low_disk/disk_usage_check.sh
sha1sum: f05fee4dd51ce7967d9bf49574a145c157642661

Verifying commit: sha1sums match!
low_disk-update-2020-07-28 (local branch) is inline with specified commit hash
------------------------------------------

[luke@rax useful-scripts]$ git checkout -b low_disk-update-2020-07-27
Switched to a new branch 'low_disk-update-2020-07-27'

[luke@rax useful-scripts]$ git push origin low_disk-update-2020-07-27 -f
Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 4 threads
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 1.06 KiB | 544.00 KiB/s, done.
Total 5 (delta 3), reused 0 (delta 0)
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To https://github.com/rackerlabs/useful-scripts.git
 + d71efc6...627adfb low_disk-update-2020-07-27 -> low_disk-update-2020-07-27 (forced update)

@LukeShirnia LukeShirnia force-pushed the low_disk-update-2020-07-27 branch 2 times, most recently from d71efc6 to 627adfb Compare July 28, 2020 12:10
@LukeHandle
Copy link
Contributor

This is moving from "Open Deleted Files Over 1GB" to now just being any generic files. This results in noise.

eg.

============================================================
 	 == Open DELETED Files ==
============================================================

0 MB  /var/lib/mysqltmp/ibVsfleA
0 MB  /var/lib/mysqltmp/ibNW0z2B
0 MB  /var/lib/mysqltmp/ibk0UwCw
0 MB  /var/lib/mysqltmp/ib6kbfSD
0 MB  /var/lib/mysqltmp/ib3w1qqy

or

============================================================
 	 == Open DELETED Files ==
============================================================

0.606308 MB  /usr/lib/systemd/systemd-logind;5ec0cc38
0.470688 MB  /usr/lib/systemd/systemd-machined;5ec0cc38
0.330147 MB  /usr/lib/systemd/systemd-journald;5ec0cc38
0.212975 MB  /usr/bin/dbus-daemon;5f100b4b
0.00390625 MB  /var/tmp/#263297

There needs to be some cutoff for "this isn't worth displaying to the user".

@LukeShirnia
Copy link
Collaborator Author

LukeShirnia commented Jul 29, 2020 •
edited
Loading

Updated as requested: LukeShirnia/Low_Disk@04d5716

    subrepo:
      subdir:   "low_disk"
      merged:   "04d5716"
    upstream:
      origin:   "https://github.com/LukeShirnia/Low_Disk.git"
      branch:   "master"
      commit:   "04d5716"
    git-subrepo:
      version:  "0.4.1"
      origin:   "https://github.com/ingydotnet/git-subrepo"
      commit:   "a04d8c2"

[lukeshirnia@rax useful-scripts]$ git branch -D low_disk-update-2020-07-27
Deleted branch low_disk-update-2020-07-27 (was 0ce3ec7).
[lukeshirnia@rax useful-scripts]$ git checkout -b low_disk-update-2020-07-27
Switched to a new branch 'low_disk-update-2020-07-27'

[lukeshirnia@rax useful-scripts]$ ./audit.sh --update low_disk/

------------------------------------------
Verifying recorded commit...

Commit on file: 0b987573391aae086baa8e716084e60a9ff46053
https://raw.githubusercontent.com/rackerlabs/useful-scripts/master/low_disk/.gitrepo

Remote Repo Info (commit)
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/0b987573391aae086baa8e716084e60a9ff46053/disk_usage_check.sh
sha1sum: 90369db9ff2000a369fdc16acbcd14c180897dee

SubRepo (remote) Info (master)
https://raw.githubusercontent.com/rackerlabs/useful-scripts/master/low_disk/disk_usage_check.sh
sha1sum: 90369db9ff2000a369fdc16acbcd14c180897dee

Verifying commit: sha1sums match!
master (remote branch) is inline with specified commit hash
------------------------------------------

------------------------------------------
Checking if updates available... (script/commit vs script/master)

Remote 'Commit' URL
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/0b987573391aae086baa8e716084e60a9ff46053/disk_usage_check.sh
sha1sum: 90369db9ff2000a369fdc16acbcd14c180897dee

Remote 'Master' URL
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/master/disk_usage_check.sh
sha1sum: 6f2d57ef82614ac949fe6b212f50283c6614678a

Master and commit hash do NOT match
Updates: Possible updates available

Compare URL:
https://github.com/LukeShirnia/Low_Disk/compare/0b987573391aae086baa8e716084e60a9ff46053...master

------------------------------------------

Branch Updated.....
Verifying Update.....
Cleaning up....
error: branch 'subrepo/low_disk/' not found.

------------------------------------------
Verifying recorded commit...

Commit on file: 04d5716a7dd087848be0fdbd821e820cdb389f79
/home/lukeshirnia/development/github/git_rackspace/useful-scripts/low_disk/.gitrepo

Remote Repo Info (commit)
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/04d5716a7dd087848be0fdbd821e820cdb389f79/disk_usage_check.sh
sha1sum: 6f2d57ef82614ac949fe6b212f50283c6614678a

SubRepo (local) Info (low_disk-update-2020-07-27)
/home/lukeshirnia/development/github/git_rackspace/useful-scripts/low_disk/disk_usage_check.sh
sha1sum: 6f2d57ef82614ac949fe6b212f50283c6614678a

Verifying commit: sha1sums match!
low_disk-update-2020-07-27 (local branch) is inline with specified commit hash
------------------------------------------

@LukeShirnia
git subrepo pull low_disk
2814f11 
subrepo:
  subdir:   "low_disk"
  merged:   "04d5716"
upstream:
  origin:   "https://github.com/LukeShirnia/Low_Disk.git"
  branch:   "master"
  commit:   "04d5716"
git-subrepo:
  version:  "0.4.1"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "a04d8c2"
@LukeShirnia LukeShirnia force-pushed the low_disk-update-2020-07-27 branch from 0ce3ec7 to 2814f11 Compare July 29, 2020 06:31
@LukeShirnia
git subrepo pull low_disk
ca624ad 
subrepo:
  subdir:   "low_disk"
  merged:   "e082e2d"
upstream:
  origin:   "https://github.com/LukeShirnia/Low_Disk.git"
  branch:   "master"
  commit:   "e082e2d"
git-subrepo:
  version:  "0.4.1"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "a04d8c2"
@LukeShirnia
Copy link
Collaborator Author 

    subrepo:
      subdir:   "low_disk"
      merged:   "e082e2d"
    upstream:
      origin:   "https://github.com/LukeShirnia/Low_Disk.git"
      branch:   "master"
      commit:   "e082e2d"
    git-subrepo:
      version:  "0.4.1"
      origin:   "https://github.com/ingydotnet/git-subrepo"
      commit:   "a04d8c2"

[lukeshirnia@rax useful-scripts]$ git checkout low_disk-update-2020-07-27
Switched to branch 'low_disk-update-2020-07-27'
[lukeshirnia@rax useful-scripts]$ ./audit.sh --update low_disk

------------------------------------------
Verifying recorded commit...

Commit on file: 0b987573391aae086baa8e716084e60a9ff46053
https://raw.githubusercontent.com/rackerlabs/useful-scripts/master/low_disk/.gitrepo

Remote Repo Info (commit)
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/0b987573391aae086baa8e716084e60a9ff46053/disk_usage_check.sh
sha1sum: 90369db9ff2000a369fdc16acbcd14c180897dee

SubRepo (remote) Info (master)
https://raw.githubusercontent.com/rackerlabs/useful-scripts/master/low_disk/disk_usage_check.sh
sha1sum: 90369db9ff2000a369fdc16acbcd14c180897dee

Verifying commit: sha1sums match!
master (remote branch) is inline with specified commit hash
------------------------------------------

------------------------------------------
Checking if updates available... (script/commit vs script/master)

Remote 'Commit' URL
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/0b987573391aae086baa8e716084e60a9ff46053/disk_usage_check.sh
sha1sum: 90369db9ff2000a369fdc16acbcd14c180897dee

Remote 'Master' URL
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/master/disk_usage_check.sh
sha1sum: 654c8ad2a144d7490f1d97b9401259d8cf1d7efe

Master and commit hash do NOT match
Updates: Possible updates available

Compare URL:
https://github.com/LukeShirnia/Low_Disk/compare/0b987573391aae086baa8e716084e60a9ff46053...master

------------------------------------------

Branch Updated.....
Verifying Update.....
Cleaning up....
Deleted branch subrepo/low_disk (was e082e2d).

------------------------------------------
Verifying recorded commit...

Commit on file: e082e2d7b46a1c6ef4828f9d54524a387ec1c5c5
/home/lukeshirnia/development/github/git_rackspace/useful-scripts/low_disk/.gitrepo

Remote Repo Info (commit)
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/e082e2d7b46a1c6ef4828f9d54524a387ec1c5c5/disk_usage_check.sh
sha1sum: 654c8ad2a144d7490f1d97b9401259d8cf1d7efe

SubRepo (local) Info (low_disk-update-2020-07-27)
/home/lukeshirnia/development/github/git_rackspace/useful-scripts/low_disk/disk_usage_check.sh
sha1sum: 654c8ad2a144d7490f1d97b9401259d8cf1d7efe

Verifying commit: sha1sums match!
low_disk-update-2020-07-27 (local branch) is inline with specified commit hash
------------------------------------------

@LukeHandle
Copy link
Contributor

Functions as expected:

============================================================
 	 == Top 5 Open DELETED Files over 500MB ==
============================================================

600 MB  /var/lib/mysql/ib_logfile0
[...]

# lsof -p 33155 -a -d 7
COMMAND   PID  USER   FD   TYPE DEVICE  SIZE/OFF   NODE NAME
mysqld  33155 mysql    7uW  REG  253,2 629145600 264201 /var/lib/mysql/ib_logfile0 (deleted)

(with /var/lib/mysql/ib_logfile1 not being listed)

All verifies:

$ curl -s https://raw.githubusercontent.com/LukeShirnia/Low_Disk/master/disk_usage_check.sh | shasum -a 1
654c8ad2a144d7490f1d97b9401259d8cf1d7efe  -

$ ./audit.sh --verify low_disk local

------------------------------------------
Verifying recorded commit...

Commit on file: e082e2d7b46a1c6ef4828f9d54524a387ec1c5c5
/Users/luke8639/Projects/rackerlabs-useful-scripts/low_disk/.gitrepo

\e[4mRemote Repo Info (commit)\e[24m
https://raw.githubusercontent.com/LukeShirnia/Low_Disk/e082e2d7b46a1c6ef4828f9d54524a387ec1c5c5/disk_usage_check.sh
sha1sum: 654c8ad2a144d7490f1d97b9401259d8cf1d7efe

\e[4mSubRepo (local) Info (low_disk-update-2020-07-27)\e[24m
/Users/luke8639/Projects/rackerlabs-useful-scripts/low_disk/disk_usage_check.sh
sha1sum: 654c8ad2a144d7490f1d97b9401259d8cf1d7efe

Verifying commit: sha1sums match!
low_disk-update-2020-07-27 (local branch) is inline with specified commit hash
------------------------------------------

@LukeHandle LukeHandle merged commit ecf46d8 into master Jul 29, 2020
@LukeHandle LukeHandle deleted the low_disk-update-2020-07-27 branch July 29, 2020 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LukeHandle LukeHandle

@dsgnr dsgnr

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@LukeShirnia @LukeHandle