Permalink
Fetching contributors…
Cannot retrieve contributors at this time
218 lines (148 sloc) 7.7 KB

QDirStat for Servers

Executive Summary

QDirStat can be used for headless (no X server, no X libs) servers: It comes with a Perl script qdirstat-cache-writer that can collect data on the server. You just have to copy the data file from the server to your desktop machine where you can view the data with the normal QDirStat application.

Server-Side System Requirements

  • Perl
  • Some command to copy files to your desktop machine: scp, ftp or whatever

One-time Server Setup

Copy the qdirstat-cache-writer script to the server.

You can find that script in scripts/ in the QDirStat source directory or in /usr/bin when you installed QDirStat as a binary package. Alternatively, you can fetch it directly from GitHub:

ssh root@myserver
cd /usr/local/bin
wget https://github.com/shundhammer/qdirstat/raw/master/scripts/qdirstat-cache-writer

By all means, have a look inside to convince yourself that there is no malicious code. It's a very simple script.

Collecting Data on the Server Side

Like QDirStat itself, qdirstat-cache-writer limits its operation to one filesystem. It does not by default descend into mounted filesystems. It scans Btrfs subvolumes, though.

For each filesystem you wish to collect data from, create a qdirstat cache file:

sudo qdirstat-cache-writer /    myserver-root.cache.gz
sudo qdirstat-cache-writer /var myserver-var.cache.gz
sudo qdirstat-cache-writer /srv myserver-srv.cache.gz
...

You should invoke the script with root permissions (thus sudo) to make sure you can read all the directories. The first parameter is the starting point of the directory scan, typically that filesystem's mount point. The last parameter is the name of the output file.

The default is .qdirstat.cache.gz which is useful for desktop machines, but not for servers, so it is recommended to explicitly specify a name here.

You might consider collecting those data in a nightly cron job.

Transfer Data to Your Desktop Machine

scp "root@myserver:~/tmp/*.cache.gz" ~/tmp

View Data on Your Desktop Machine

qdirstat --cache ~/tmp/myserver-root.cache.gz

or

qdirstat -c ~/tmp/myserver-root.cache.gz

or start qdirstat and use "Read Cache File..." from the "File" menu.

Limitations

You cannot use QDirStat's built-in cleanup operations, of course; they'd still run on your desktop machine instead of your server. There is also no indication that your are seeing the contents of a cache file rather than data collected live from your local system, so be careful what you are doing.

Security Concerns

Don't give a cache file to somebody you wouldn't trust to read all directories on that server with root permissions. Don't even make it easily available to such persons. If the directory contents include sensitive information, treat the cache file with the same degree of confidentiality as you would the original directory on the server.

Why Not Use QDirStat over Remote X?

You can do that as well, of course. But that means you'll need at least the X11 libs and the Qt libs (Qt 5 by default) on the server. And you need to install QDirStat on the server, too.

Also notice that with the advent of Qt 5, remote X has become very slow with Qt applications: Qt 5 no longer uses Xlib / X protocol primitives for painting (XDrawRectangle etc.), but renders into a pixel buffer and transfers that pixel buffer. While this is a considerable speedup for local X, it is pretty slow for remote X.

To ease that pain a little, QDirStat has a --slow-update (or -s) command line option which is intended for remote X:

ssh -X myserver
qdirstat --slow-update

This makes QDirStat update its display only every 3 seconds rather than the default 333 milliseconds during directory reading.

This interval can be configured in ~/.config/QDirStat/QDirStat.conf :

[DirectoryTree]
SlowUpdateMillisec = 3000

Looking Into a Cache File

A cache file is a gzipped text file, so it can be viewed with zless:

[qdirstat 1.0 cache file]
# Generated by qdirstat-cache-writer
# Do not edit!
#
# Type  path            size    mtime           <optional fields>

D /var  4096    0x53cef170
# Device: /dev/sda6

L       run     4       0x54bbf0e3
L       lock    9       0x54bbf0e3
D /var/cache    4096    0x58125237
D /var/cache/dictionaries-common        4096    0x53ceef0a
F       hunspell.db     188     0x53ceef0a
F       ispell-dicts-list.txt   0       0x53ceef0a
F       wordlist.db     267     0x53cef022
F       ispell.db       188     0x53ceef0a
F       jed-ispell-dicts.sl     881     0x53cef024
F       aspell.db       741     0x53cef024
F       sqspell.php     366     0x53cef024
F       emacsen-ispell-default.el       173     0x53ceef0a
F       emacsen-ispell-dicts.el 897     0x53cef024
D /var/cache/cracklib   4096    0x53ceef8b
F       cracklib_dict.pwi       22972   0x53ceef8b
F       cracklib_dict.hwm       1024    0x53ceef8b
F       src-dicts       104     0x53ceef8b
F       cracklib_dict.pwd       412618  0x53ceef8b
D /var/cache/cups       4096    0x58a842ad
F       job.cache       992     0x58a842ad
F       ppd-updates     271     0x567b004a

The file format is described in detail in cache-file-format.txt.

In short, each line contains one entry for a file, directory, or symlink.

  • The first field is the type: 'D' for directory, 'F' for file, 'L' for symlink.

  • The second field is the name; for directories, that's always the full path, for files, the path can be omitted.

  • The third field is the size of the object (not including any child objects).

  • The fourth field is the mtime in hex (seconds since 1970-01-01 00:00:00).

Long file format

qdirstat-cache-writer -l /var /tmp/var.cache.gz
zless /tmp/var.cache.gz

[qdirstat 1.0 cache file]
# Generated by qdirstat-cache-writer
# Do not edit!
#
# Type  path            size    mtime           <optional fields>

D /var  4096    0x53cef170
# Device: /dev/sda6
            
L /var/run      4       0x54bbf0e3
L /var/lock     9       0x54bbf0e3
D /var/cache    4096    0x58125237
D /var/cache/dictionaries-common        4096    0x53ceef0a
F /var/cache/dictionaries-common/hunspell.db    188     0x53ceef0a
F /var/cache/dictionaries-common/ispell-dicts-list.txt  0       0x53ceef0a
F /var/cache/dictionaries-common/wordlist.db    267     0x53cef022
F /var/cache/dictionaries-common/ispell.db      188     0x53ceef0a
F /var/cache/dictionaries-common/jed-ispell-dicts.sl    881     0x53cef024
F /var/cache/dictionaries-common/aspell.db      741     0x53cef024
F /var/cache/dictionaries-common/sqspell.php    366     0x53cef024
F /var/cache/dictionaries-common/emacsen-ispell-default.el      173     0x53ceef0a
F /var/cache/dictionaries-common/emacsen-ispell-dicts.el        897     0x53cef024
D /var/cache/cracklib   4096    0x53ceef8b
F /var/cache/cracklib/cracklib_dict.pwi 22972   0x53ceef8b
F /var/cache/cracklib/cracklib_dict.hwm 1024    0x53ceef8b
F /var/cache/cracklib/src-dicts 104     0x53ceef8b
F /var/cache/cracklib/cracklib_dict.pwd 412618  0x53ceef8b
D /var/cache/cups       4096    0x58a842ad
F /var/cache/cups/job.cache     992     0x58a842ad
F /var/cache/cups/ppd-updates   271     0x567b004a

The only difference is that all entries are always specified with their full path. That makes the file a bit larger, but now you can use it as a substitute for the locate command:

zgrep cracklib /tmp/var.cache.gz

Cache Files on Desktop Machines

See scripts/README.md