Skip to content

@shundhammer shundhammer released this Jul 26, 2020 · 18 commits to master since this release

Overview

The QDirStat project proudly presents its latest release: QDirStat 1.7.

This is a stable release. It brings a number new features and some bug fixes:

  • Closing the gap between sizes reported by QDirstat and sizes reported by the du command: Now also taking the allocated size into account.

  • Now also displaying the allocated size where it makes sense.

  • New "Mounted Filesystems" window showing output similar to the df command (but without the cruft).

  • New directory selection dialog showing all (real) filesystems.

  • New checkbox to cross filesystems (temporarily) in the directory selection dialog.

  • Workaround for NTFS oddities: Ignoring hard links on NTFS.

  • Added config option to ignore hard links in general.

  • Added a document about Btrfs free size and how different tools tend to show different values:
    Btrfs-Free-Size.md

  • Bug fixes

Details

Closing the Gap Between QDirstat and the "du" Command

QDirStat now also takes the allocated size (the number of allocated disk blocks) into account.

There were repeated threads on various social media where users wondered why QDirStat displayed different sizes than the du or the df commands, sometimes slightly different, sometimes off by quite a lot.

In short, this is mostly due to the difference between a file's byte size and its allocated blocks: On most filesystem types, a file of 49 bytes still consumes a minimum of 4 KB (4096 bytes); disk space is allocated in clusters (typically 4 KB), not byte by byte. On a typical Linux root filesystem with many thousands of tiny files, this difference can add up to a lot.

Formerly, QDirStat only added up the byte sizes, disregarding the allocated, but really unused part at the end of tiny files. The rationale was that some filesystem types handle that in more intelligent ways, yet there is no documented way to get information from a filesystem if it actually does that. It turned out that in reality, most of them don't bother; they simply let most of that last cluster go to waste.

Now QDirStat displays both sizes where useful:

  • For tiny files, the tree view now shows both sizes: "49 B (4k)"

  • For directories, the tree view now shows the total of the allocated sizes. This may sound a bit inconsistent, but it feels very natural and intuitive: You are typically interested in how much disk space the subtree consumes, no matter if some part of that is really wasted.

  • The details view shows both sizes when there is a difference (in the displayed numeric precision): "Size: 42.1 MB" vs. "Allocated: 42.2 MB", but not "Size: 42.0 MB" vs. "Allocated: 42.0 MB".

  • Treemap tiles now correspond to a file's allocated size. This makes a real difference for tiny files.

More details at GitHub Issue #134: Size Difference between QDirStat Reports and the "du" and "df" Commands.

New "Mounted Filesystems" Window

Menu "View" -> "Show Mounted Filesystems" or Ctrl-M:

This shows information about "normal" mounted filesystems (excluding system mounts like /dev, /proc, /sys, bind mounts, Btrfs subvolumes) with usage data as reported by the filesystem itself. The "Reserved" column shows the disk space reserved for the root user on that filesystem, "Free" is the available space for non-privileged users.

Sometimes the "Used" size reported here may be different from what QDirStat reports after reading that complete filesystem; this can be due to metadata, journals or snapshots. Notice that in those cases, the du command will also display a different value than the df command.

New Directory Selection Dialog

QDirStat now has its own custom directory selection dialog; it no longer uses one of the simplistic Qt standard file dialogs:

The "Places" bar on the left shows your home directory and all "real" filesystems (the same as in the new "Mounted Filesystems" window). For each filesystem, it shows

  • The mount point
  • The total size of the filesystem
  • The filesystem type ("ext4", "btrfs", "ntfs", ...)
  • The device name ("/dev/sda2") (in a tooltip)

In the "Path" combo box you can simply edit the path or copy and paste it from another window. Use the "Up" button to move one directory level up.

Crossing Filesystems Temporarily

The "Cross Filesystems" checkbox lets you temporarily override the global configuration option of the same name: QDirStat will then no longer stop when a mount point is found during reading a subtree, it will descend into that mounted filesystem and read it, too; but again, only for "real" filesystems, not for system mounts such as /dev, /sys, /proc, not for bind mounts, not for filesystems mounted multiple times, and not for network mounts (NFS, Samba / CIFS).

More details at GitHub Issue #129.

NTFS File Sizes and Hard Links

Added a workaround for wrong size sums on NTFS: Now disregarding hard links on NTFS.

The (current?) ntfs-3g implementation using fuseblk seems to disagree with Windows tools which files really have hard links.

It appears that ntfs-3g regards even the MS-DOS compatible short filename (PROGRA~2 vs. Program Files) as a hard link which is of course utter nonsense; that means that almost all files on an NTFS partition are reported as having multiple hard links, so QDirStat displayed them as having only half their real size.

Notice that this is a bug in ntfs-3g, not in QDirStat.

See also GitHub Issue #88.

Completely Ignore Hard Links

Added a config option to ignore hard links.

This is useful for a very small number of use cases. Hard links are not very common anymore in today's Linux / BSD / Unix-like systems, so most users won't have to bother with this at all.

By default, QDirStat sums up the disk space for a file with multiple hard links for each hard link's share of the overall size: If a file with 1 MB has 4 hard links, for each of those 4 links QDirStat adds 1/4 of the size (i.e., 256 kB) to the parent directory. If all those 4 links are in the same directory, that's very simple: They add up to 4 * 256 kB = 1 MB, so the sum is correct.

If those hard links are all in different directories, each directory only gets part of that disk space allocated, because in fact they share the disk space among each other; the total disk space sum taking all those directories into account is still correct, of course.

The trouble starts when you want to make a backup of only one of those directories: Even though the disk space is still shared with other directories, on the backup medium, you still need the disk space for that complete file, i.e. the full 1 MB, not only that directory's share (256 kB). With a lot of hard-linked files, that can add up to a lot of difference between what QDirStat displays and what disk space you actually need for the backup.

There was a user who makes heavy use of that, and for that kind of use case there is now the option to ignore hard links: In that case, QDirStat sums up the complete size (the full 1 MB) for each hard link of the file.

While that is useful for this special case, and you can now see the total size that you will need for your backup medium for that one directory, the total size higher up in the directory tree where more than one of those directories that share hard linked files with each other is off: That file now appears 4 times with 1 MB each, so it will add up to 4 MB.

There is now a new config option in ~/.config/QDirStat/QDirStat.conf:

[DirectoryTree]
...
IgnoreHardLinks=false

This is intentionally not available in the GUI config dialog to avoid confusion; use a text editor (while QDirStat is not running) to change this setting.

Please use that config option only when you are aware of the consequences; this is a specialized option for rare, specialized use cases. It basically makes sense only if the other hard links are all outside the subtree that QDirStat displays.

More details at GitHub Issue #124.

New Document about Btrfs Free Size

There is now a new document about Btrfs free size and how different tools tend to show different values: Btrfs-Free-Size.md

Whenever users run out of disk space on Btrfs (which happens a lot, mostly because of snapshots) and they try to figure out where all their disk space went, they are confused about different tools reporting totally different and inconsistent sizes; traditional Linux / Unix command line tools like du and df just like GUI tools like QDirStat. Hopefully, this document will shed some light on that.

Bug Fixes

  • Fixed the internal cache writer (the one called from the File menu, not the qdirstat-cache-writer Perl script): For files with multiple hard links, it wrote the wrong size to the cache file: The result of size / links, not size.

    This was part of GitHub Issue #124.

  • Fixed crash when terminating with Ctrl-Q while reading directories
    (GitHub Issue #122).

  • Now automatically reopening the old current branch in the tree view when using actions like "refresh selected" and "continue reading at mount point
    (GitHub Issue #135).

  • Not a bug, but a minor new feature: Now automatically expanding the first directory level of a mount point after "continue reading at mount point".

  • Now correctly labelling a mount point as mount point in the "Details" panel.

Donations

QDirStat is Free Open Source Software. You are not required to pay anything.
Donations are most welcome, of course.

If you like, you can donate via PayPal (freely select the amount to donate):

paypal

Assets 2

@shundhammer shundhammer released this Feb 13, 2020 · 129 commits to master since this release

Overview

The QDirStat project proudly presents its latest release: QDirStat 1.6.1.

This is a stable release. It brings some new (minor) features and one bug fix:

  • Much better handling for "permission denied" errors while reading directories.

  • Now showing the exact byte size (134 495 994 Bytes instead of 128.3 MB) upon mouse click in the tree (right click) and in the details panel (left or right click).

  • New optional tree column "Oldest File" (not enabled by default).

  • Bug fix: Support for dark widget themes in File Size Histogram window.

Details

Improved Handling for "Permission Denied" Errors while Reading Directories

If you start QDirStat with insufficient permissions, you could easily overlook large subtrees in the filesystem that consume disk space, but were not visible to you. They did get a special icon, but you would have to open the parent directory in the tree view to see that.

Now, QDirStat notifies you in several ways:

  • All parent directories that contain a subtree that could not be read are now displayed in dark red in the tree view.

  • The Size field of those directories as well as other accumulated values (Items, Files, Subdirs) are now preceded with a "greater than" sign to indicate that there is most likely more, but that information could not be retrieved: ">7.2 MB" indicating that it's at least 7.2 MB and most likely more than that, but we don't know because one or more subdirectories could not be read.

  • A message panel in the main window between the tree view and the treemap with a message that some directories could not be read.

    You can close the message with the [x] close button on its right side, but you can also simply leave it open. This is a lot less obtrusive than a pop-up dialog, yet less temporary than a message in the bottom status line that will disappear in a few seconds or when anything else is reported.

  • Clicking on the "Details..." link in that message opens a separate window to report all directories that could not be read (typically because of insufficient permissions).

    This window is non-modal, i.e. you can still interact with the main window when it is open. Click on any directory that it lists to locate it in the main window: The tree view will open that branch and scroll to make it visible.

  • In addition to the "locked folder" icon, unreadable directories are shown in bright red in the tree view.

  • When an unreadable directory is selected in the tree view, the details panel now shows a large padlock icon and a message "[Permission Denied]", and the permissions are highlighted in red.

Exact Byte Size on Demand

You can now see the exact size in bytes both in the tree view and in the details panel: 134 495 994 Bytes instead of 128.3 MB. The field is still (somewhat) human readable with thousands separators (using blanks to avoid confusion with different decimal / thousands separators in different languages).

This can make it easier to compare sizes with other tools that report them in bytes or that insist in using 1000-based units (QDirStat uses 1024-based size units: 1 kB = 1024 Bytes; 1 MB = 1024 kB; 1 GB = 1024 MB; etc.).

Not using tool tips that appear automatically was a conscious decision: This level of detailed information is not needed that often, and tool tips get in the way whenever the mouse cursor lingers too long at an active spot. More often than not a tool tip obscurs other content that the user might want to read at that very moment. This is why in QDirStat in the rare cases that you are interested in those exact numbers, you have to click:

  • In the tree view, right-click a size field (a left click is used for selecting an item in tree views, so the context menu is pressed into service for that purpose (only for the size column)).

  • In the details panel, use left or right click. To indicate what fields can be clicked, they are now underlined when the mouse hovers over them.

    Hovering over fields that can be clicked shows them underlined, very much like a hyperlink.

    Clicking (left or right mouse button) shows more details: In this case, the exact byte size.

New Tree Column "Oldest File"

There is now an optional new column "Oldest File" that shows the timestamp (the mtime) of the oldest file in a subtree. This is strictly about files; directories, symlinks and special files (block or character devices, FIFOs etc.) are ignored for this.

This may be useful to spot some old cruft, for example leftover dot files in your home directory. Many programs generate such files when you start them for the first time, and they are rarely cleaned up when they fall out of use.

Notice that this column is not enabled by default. If you would like to use it, switch to layout L2 or L3, right-click the tree header to open the columns context menu, select Hidden Columns, then Show Column "Oldest File".

Of course you can also sort by this column to see the oldest files first (or last).

Bug Fix

Fixed text color in histogram in dark widget themes (GitHub Issue #117).

Assets 2

@shundhammer shundhammer released this Jul 22, 2019 · 211 commits to master since this release

Overview

The QDirStat project proudly presents its latest release: QDirStat 1.6.

This is a stable release. It brings performance improvements, new features and some bug fixes:

  • Performance improvement while reading directories

  • Vast performance improvement for huge directories (100.000+ entries in a single directory)

  • New packages view

  • New unpackaged files view

  • New standard cleanup: Check File Type

  • Support for excluding directories containing a file with a specific name or pattern.

  • Greatly improved man page

  • Some minor bug fixes.

Details

Performance Improvements

New Views

  • New packages view:
    QDirStat can now visualize the file lists of installed packages:

    I.e. files are now grouped by the package they belong to, and in each subtree only the files that belong to the package are displayed: In this example, in /usr/bin only the chromium-browser binary is displayed, not all the other files in /usr/bin. This is intentional.

    You can display all installed packages with their file lists (but that takes a while), or you can select just a subset. Use Menu File -> Show Installed Packages or start QDirStat with a pkg:/ command line argument.

    As with the other package manager related features, this is supported for all Linux distributions that use one of dpkg, rpm or pacman (or any higher-level package manager based on any of those like apt, zypper etc.).

    More details at Pkg-View.md.

  • New unpackaged files view:

    QDirStat can now visualize the files in system directories that are not packaged, i.e. that are not part of any file list of any installed software package.

    This can be useful to track down problems after package upgrades or after manually installing software with sudo make install.

    This is supported for all Linux distributions that use dpkg or rpm (or any higher-level package manager based on any of those like apt, zypper etc.).

    More details at Unpkg-View.md.

Misc

  • New standard cleanup: Check File Type. This uses the file command to find out more detailed information what exactly a file is and displays it in the cleanup output window.

    More details, screenshots and how to get it if you already have an existing QDirStat cleanup configuration file (i.e. if you used QDirStat before): GitHub Issue #102.

  • Implemented GitHub Issue #90: Support excluding directories containing a file with a specific name or pattern.

    Similar to some backup tools, you can now specify an exclude rule that lets you exclude a directory that contains a file like .nobackup or .qdirstatexclude.

  • Greatly improved the man page; see man 1 qdirstat.

  • Some minor bug fixes.

Assets 2
Jul 1, 2019
Version bump for latest fix

@shundhammer shundhammer released this Nov 7, 2018 · 444 commits to master since this release

Overview

The QDirStat project proudly presents its latest release: QDirStat 1.5.

This is a stable release following the recent 1.4.97-Beta release.
It brings a number of new features as well as some bug fixes:

  • (Optional) breadcrumbs navigation

  • (Optional) a new "details" panel for the currently selected item

  • (Optional) new tree columns:
    owner, group, permissions both in "rwxrwxrwx" and in octal format

  • Package manager support for the major Linux package managers (dpkg / rpm / pacman) to see what package a system file belongs to.

  • Quick-switchable different layouts for the main window

  • A new "General" page in the configuration dialog

  • Show in the window title if the program runs with root permissions (sudo etc.)

  • (Optional) show the URL in the window title

  • Some bug fixes (see below)

There were very few changes compared to the 1.4.97-Beta release. No bugs were reported against that Beta.

Details

Main Window

Breadcrumbs Navigation (Optional)

Above the directory tree there is now a breadcrumbs navigation widget.

This shows the current path with each component a clickable link so you can easily go up the tree - up to the top of the tree that you started QDirStat with.

You can switch this on and off with View -> Show Current Path.

The New "Details" Panel (Optional)

On the right side of the tree there is now a "Details" panel showing information about the currently selected item, or, if multiple items are selected, about the sum of them all.

You can switch this on and off with View -> Show Details Panel.

What information is dispayed depends on the type of the selected item:

  • File:

    • File name (without path)
    • Type (file / symlink / block device / character device / FIFO)
    • MIME Category (Documents / Images / Videos / ...) as defined for the treemap colors (see MIME Categories in the configuration dialog).
    • Size
    • A notification if this is very likely a system file, i.e. a file owned by a user ID < 500 and in a known system directory like /bin, /dev, /etc, /lib, /opt; /usr, but not /usr/local; etc.
    • For system files: The software package that it belongs to, if known (see below)
    • User (file owner)
    • Group
    • Permissions in both "rwxrwxrwx" and octal ("0777") format
    • Date and time of last modification (mtime)
  • Directory:

    • Subtree information:
      Total size, items, files, subdirectories; date and time of the last modification anywhere in that subtree
    • Information about the directory itself:
      Size of the directory itself without the subtree, user, group, permissions; date and time of the last modification of the directory (which might be less recent than the last modification of any file anywhere in the subtree).
  • Files pseudo directory:

    Similar to directory, but only the subtree information since there is no counterpart to this object on disk; it just serves to collect the direct file children of the directory.

  • Multi-selection:

    • Total size of all selected items including any subtrees
    • Number of directly selected files (without subtrees)
    • Number of directly selected directories (without subtrees)
    • Total number of files in all selected subtrees

Package Manager Support

If a file is identified as a system file and a supported package manager is detected on the system, QDirStat now queries the package manager what software package the file belongs to and displays that information in the new "Details" panel.

This gives a surprising amount of insight where all the disk space on the system directories goes to, most importantly some of the big blobs in the tremap.

Currently, QDirStat supports the most common Linux package managers: dpkg, rpm, pacman.

Please notice that higher-level package managers such as apt, zypper, yum or front-ends like synaptic, PackageKit all use one of those lower-level package managers internally, so even if you normally only use one of those higher-level package managers, it still works without any restriction.

That means that QDirStat currently supports this feature for:

  • Debian
  • Ubuntu / Kubuntu / Xubuntu / Lubuntu
  • SUSE (openSUSE Tumbleweed or Leap, SLES)
  • Red Hat (Fedora, RHEL)
  • Arch Linux
  • Manjaro

... and dozens more; basically all that are based on any of the above.

This feature works by running dpkg -S, rpm -qf or pacman -Qo as external commands, so this is a somewhat expensive operation. To keep the user interface responsive, QDirStat now has an "adaptive timer" for updating that information: Normally, the result is shown instantly, but if you click around wildly, there is a timer that is increased or decreased (thus "adaptive") for a delayed update after that timeout (0 / 333 / 1000 / 2500 millisec right now).

It can even handle rpm installed as a foreign package manager on a dpkg based system (and the other way round); it tries the primary package manager first, then any others that are also installed.

Support for BSD systems (FreeBSD, OpenBSD) was also planned, but it turned out that currently the package managers of those systems only know what package owns files of ports / 3rd party, not for the base system, so this feature might not be very useful on those systems.

Unified Size Columns

In the tree there used to be separate colums for total size and own size; this is now unified to only one size column. Own size was really mostly useful for directories where the directory itself has a size, and then there are all its children in the subtree. But that information (which is only rarely useful anyway) is available in the "Details" panel, so it no longer makes much sense to consume so much screen space with that separate column.

New Tree Columns (Optional)

This was the result of GitHub issue #80: Show owner and group in the tree.

The directory tree view can now display those additional columns:

  • User (file owner)
  • Group
  • Permissions in "rwxrwxrwx" format
  • Permissions in octal ("0777") as used for the chmod command.

Remember that you can hide or show tree columns in the tree by using the context menu in the tree header: Right-click on any header in the tree. You can also rearrange columns by dragging them where you want them - with the exception of the Name column which always remains at the left edge.

Switching Layouts

Since the new columns consume so much screen space and they are only useful in certain situations, but in others the details panel is so much more useful, it is now possible to quickly switch between three different layouts for the upper half of the main window.



  • Layout 1 (short): Display only a bare minimum of columns in the tree view, but show the new details panel on the right side. This is to minimize clutter in the tree view, yet keep all the information available.

  • Layout 2 (classic): Display the same columns as always in the tree view and additionally the new details panel on the right side.

  • Layout 3 (full): Display all available columns in the tree view, including the new file owner, group, permissions in both "rwxrwxrwx" and octal. But don't display the new display side panel so there is enough screen space to show all those columns.

Switching between the layouts is as easy as pressing one of the three new buttons in the tool bar: L1, L2, L3. Or use the corresponding entries in the "View" menu. Or use the Alt-1, Alt-2, Alt-3 key combinations.

For each view, you can individually configure what columns to display and whether or not to display the new details side panel and / or the new breadcrumbs navigator.

The column widths are still configured globally (and this will remain like that); after some experiments, it turned out that it is very awkward to have to set all of them individually for each layout.

New "General" Configuration Dialog Page

This is a bit unorganized (as might be expected for misc settings), but those misc settings had accumulated over time, so it was about time to finally add such a page.

The settings were always accessible, but only by editing the config file (~/.config/QDirStat/QDirStat.conf) manually. For some settings you might still have to do that: Some are too exotic for general use, some others would confuse the average user much more than they would help.

Information in the Window Title

  • QDirStat now shows [root] in the window title if it is running with root privileges.

  • If enabled in the new General configuration page, QDirStat can now show the URL in the window title, i.e. the path of the toplevel directory that it was started with. (GitHub issue #77).

New Application Icon

Noah Davis (noahdvs) contributed a new application icon for QDirStat:

The old icon goes back to early 2000 for the first KDirStat 0.86 release. It was never a masterpiece, but the new application just needed an icon. Finally somebody with more artistic talent than myself made one that looks a lot more professional. Thank you, Noah!

Bug Fixes

  • Fixed GitHub issue #84: Crash if picking up a cache file in the starting directory (subtree root)

  • Fixed GitHub issue #74: Crash deleting folder while scan is in progress

  • Fixed GitHub issue #72: Allow to specify install prefix (e.g. /usr/local instead of /usr)

  • Fixed duplicate keyboard shortcut for actions "Stop Reading" (now: Ctrl-S) and "Copy URL to Clipboard" (still Ctrl-C).

  • Fixed exclude rules not matching correctly against direct root directory children: A rule that should match "/var" now works correctly. You'd need to specify "//var" in the rule which of course was wrong.

  • Now only adding the default exclude rule ".snapshot" once. If you remove that one, it should no longer keep reappearing if there are no other exclude rules.

  • Fixed some minor issues in the exclude rule configuration dialog with some widgets not being correctly disabled if they made no sense to use.

  • Now restoring the file owner of the QDirStat configuration files (~/.config/QDirStat/*.conf) after running with sudo on systems where the sudo command still uses the user's home directory (all Ubuntu versions): They used to be owned by root after sudo qdirstat, so the user could not write to them anymore, so all configuration changes (including window size and positions) were silently discarded.

Donations

QDirStat is Free Open Source Software. You are not required to pay anything.
Donations are most welcome, of course.

If you like, you can donate via PayPal (freely select the amount to donate):

paypal

Assets 2
Pre-release

@shundhammer shundhammer released this Oct 26, 2018 · 452 commits to master since this release

Removed the change log to avoid duplication; see the 1.5 release.

Assets 2

@shundhammer shundhammer released this Jun 4, 2017 · 639 commits to master since this release

Overview

The QDirStat project proudly presents its latest release: QDirStat 1.4.

This release brings a number of new features as well as some bug fixes:

  • File size statistics window with histogram

  • Filling the gaps in the treemap (directories with lots of small files)

  • Checked code with Coverity and fixed all issues it pointed out

  • Improved support for file and directory names with UTF-8 special characters (Qt 4 only issue)

  • Improved support for directories that have read, but not execute permissions

Details

New Features

File Size Statistics Window with Histogram

File Size Histogram Screenshot

This is a whole new kind of statistics in QDirStat showing how file sizes are distributed. You can start that for any selected directory (menu View -> File Size Statistics or F2) or from the File Type Statistics window if you select any filename suffix (extension) there and then File Type -> Size Statistics (or F2). In the latter case, you can see how large all your photos (.jpg), your videos (.mp4) or whatever are.

This new statistics window deals with a lot of things you might have come to hate at school or at university, and which your math teacher or your statistics professor never explained in a way that mere mortals can understand, so there are explanations of those things as a bonus. There is a landing page for that in that new window:

File Size Help

Or you might use this as a starting point.

Everybody thinking "I have no clue what this is all about", please have a look at the Median, Quartiles and Percentiles Explained document to convince yourself that really everybody can easily understand this.

There is also a GitHub issue to discuss this; comments are welcome. Feel free to reopen this for any discussions about that topic.

Filling the Gaps in the Treemap

GitHub issue #58 shows that users feel under-informed when there are grey areas in the treemap. The explanation is simple: Treemap tiles are only displayed when they have at least a certain minimum size (by default 3 pixels). Otherwise the treemap just gets cluttered with tiny things that don't show any information whatsoever.

The remaining space is taken by its parent directory's tile. They were rendered just flat grey which makes their boundaries against each other invisible, thus giving the impression that there is nothing.

So there were some experiments with visible borders, but that completely destroyed the visual impression of the treemap because those borders were everywhere. Fill patterns also didn't help: They were just ugly, and there was no way to tell where one directory tile ends and where the next one starts.

The next approach was using gradients. The first impression was good, but then it turned out that it was hard to tell which item was a (now over-emphasized) directory and which one a large file. Locating large files deep inside the directory hierarchy is the major strong point of the treemap visualization, so giving that up did not seem like a smart idea. After playing a bit with the gradient parameters (toning it down and giving it just a little blueish tint) this was the result:

Using Gradients

This appears to be a good compromise.

Of course this is configurable: Edit ~/.config/QDirStat/QDirStat.conf:

[Treemaps]
...
DirGradientEnd=#707080
DirGradientStart=#606070
...
UseDirGradient=true

Beware that QSettings sorts the entries alphabetically, so the start is after the end (how philosophical...).

Bugfixes

  • Fixed problem with directories that have read, but not execute permissions thanks to slodki:

    In that case, you'd get a warning in the log for every entry in such a directory, and it would get the wrong icon (a locked folder) and null values for all fields. Now checking for execute and read permission of the directory in advance and not even trying to read any contents (because the values would be bogus anyway).

  • Fixed GitHub Issue #61: Files and directories with UTF-8 special characters in the name not read correctly when built with Qt 4.x

    This happened only when QDirStat was built against Qt 4.x, but that is the default for NHellFire's PPA, so this affects all Ubuntu users who installed QDirStat from that PPA.

    Thanks to slodki who pointed this problem out!

Other Improvements

  • Checked code with Coverity

    Coverity offers free static code analysis for Open Source projects. This is really an outstanding tool, and it does a really thorough analysis.

    You might be glad to hear that while it complained about some minor things, there was not a single issue that would have been user relevant (let alone any security problems - there were none). Still, I did my best to fix the small complaints it had, and now we are down to zero outstanding defects reported by Coverity in QDirStat's 130,000 lines of code.

  • More consistency between file type and size statistics

    Like the new file size statistics window, the older file type statistics window now uses the currently selected directory (in the tree view), not always the tree's toplevel directory. If nothing is selected, it still uses the toplevel directory.

    That means that F3 no longer toggles that window, but re-populates it with the currently selected directory instead. This is consistent with the F2 view.

    Of course, the "Locate Files by Type" window now is also restricted to that subtree which actually gives it better functionality if you know that you want to locate files only there.

    This means that you can now select a subdirectory in the tree, open the file type statistics for it (F3 key), then select any filename extension (suffix) in that window and then open the file size statistics (F2 key) for that file type in that subtree.

    Previously, you would have to start QDirStat to show only that directory, then open the file type statistics window (F3), then the file size statistics window (F2) from there.

Assets 2

@shundhammer shundhammer released this Mar 5, 2017 · 763 commits to master since this release

Overview

The QDirStat project proudly presents its latest release: QDirStat 1.3.

This release brings a number of new features as well as some bug fixes:

  • File type statistics window

  • Locate files per directory by file type

  • Display path and size of the current item in the status line while hovering over a treemap tile

  • Optional alternate layout of the main window: Tree view and treemap side by side

  • Improved logging: More secure and rotating logs

  • New document: QDirStat for Headless Servers for server admins

  • New document: The GitHub Workflow for developers

Details

New Features

File Type Statistics Window

People who know WinDirStat inevitably want its File Type view in QDirStat, too. I was really reluctant to do that because I didn't quite see the point; in WinDirStat, it serves mostly as a legend to the treemap colors since they are constantly changing in WinDirStat: The file type that consumes most disk space always gets color #1, the next-most color #2 etc., so it depends which directory you scan what color each file type gets. In QDirStat, the colors are stable; they are predefined and configurable in the MIME Type Categories configuration dialog.

And as most of you probably know, filename extensions have a much stricter meaning in Windows than on Linux/Unix systems; Linux people get very creative when it comes to using dots in filenames. Sometimes those dots delimit a filename's extension (suffix) from its base name, sometimes they are used for entirely different purposes.

But there was one user who was insistent enough to make me reconsider and convinced me with valid use cases for this, and after some experimenting I found an approach that works for QDirStat -- both technically and in terms of usability:

File Type Statistics Window Screenshot

So it's not just a flat list of filename extensions (suffixes); rather, they are sorted into their respective MIME categories (yes, the ones you might know from the Treemap Colors config dialog) where appropriate and Other for everything that doesn't fit into a known category.

See the whole discussion with more screenshots at GitHub issue #45.

Limitations

Since filename extensions (suffixes) don't have as much semantics in Linux/Unix systems as they do in Windows, many files are categorized as "Other". This is a known limitation, but it's a limitation of the whole concept of using suffixes to categorize files by type. And no, checking file headers for magic byte sequences like the "file" command does is not an option here; QDirStat would have to do that for (at least) all the 30,000+ files typically listed under the "Other" category.

So we'll have to live with that limitation. More exactly, everybody will have to live with this; I will not spend hours upon hours anymore explaining that same thing to the umpteenth user demanding this to be more sophisticated.

This will also not become part of the main window (as was suggested before) because it just takes up too much space, and it would clutter the main window too much; See also GibHub issue #52 and GitHub issue #51.

Locating Files by Type

Locating FilesWindow

You can now locate files with a specific filename extension directly:

  • You select a file type (a filename extension) in the "File Type Statistics" window.

  • You click "Locate" or you double-click the item.

  • The "Locate Files" window opens.

  • You click a directory there.

  • In the main window, the branch for that directory opens, and all matching files are selected in the tree view and in the treemap.

  • You can now directly start cleanup actions for those files.

See also GitHub issue #48.

The File Type Statistics and the Locate window are both non-modal, i.e. you can still work in the QDirStat main window while they are open, and they both remember their size and position so you can arrange them on the screen to have everything visible at the same time that you need to see.

Improved Logging

Logging is now more secure, and the logs are automatically rotated, so there is no longer one large log file that keeps growing and growing.

QDirStat now uses its own log directory /tmp/qdirstat-$USER (where $USER is your user name; the numeric user ID is now only used if the user name cannot be obtained). It no longer keeps one single log file growing, but starts a new one each time it is started. 3 old logs are kept; any older ones are deleted.

The permissions for that directory are set up in a pretty restrictive way (0700, i.e. rwx------) when it is created. If it already exists, QDirStat checks the owner and creates a new one with a random name if it is owned by anyone else than the user who started QDirStat.

[sh @ balrog] ~ 68 % ls -ld /tmp/qdirstat-sh
drwx------ 2 sh sh 4096 Feb 24 18:29 /tmp/qdirstat-sh
[sh @ balrog] ~ 69 % ls -l /tmp/qdirstat-sh
total 16
-rw-rw-r-- 1 sh sh 2067 Feb 24 18:29 qdirstat-00.old
-rw-rw-r-- 1 sh sh 2067 Feb 24 18:07 qdirstat-01.old
-rw-rw-r-- 1 sh sh 2067 Feb 24 18:07 qdirstat-02.old
-rw-rw-r-- 1 sh sh 2067 Feb 24 18:29 qdirstat.log

For anybody regularly watching the log file this means they will now have to use tail -F qdirstat.log rather than tail -f since the latter does not realize when the file it watches is renamed and a new one is created under the same name.

Other New Features

  • Implemented GitHub issue #30:

    When hovering over a treemap tile, display the full path and the total size of that element in the status bar. When the hover ends (when the mouse cursor leaves the treemap tile), display the current selection again in the status bar.

  • Optional alternate layout for the main window: You can now switch from the traditional layout of tree view above and treemap below to a side-by-side layout with the tree view on the left and the treemap on the right (Menu Treemap -> Treemap as Side Panel).

    This was contributed by @flurbius.

New Documents

  • QDirStat for Headless Servers

    This document explains step by step how to use the qdirstat-cache-writer Perl script to collect data on your headless (without X server, without X libs) server somewhere in a rack in a data center and display the data on your local workstation machine with QDirStat.

    This information has been out there for at least 10 years, but still people keep asking that on sites like stackexchange.com and superuser.com; I lost count how many times I answered it. And still people come up with the most crazy and arcane command lines to do a shitty job for the same problem when they could also have the good solution.

  • The GitHub Workflow

    This document is a step-by-step instruction guide how to collaborate with projects such as QDirStat (but also other Open Source projects) with GitHub and git with upstream repositories and your personal fork and local working copies; how to set it up, how to fetch and rebase, how to work with pull requests and getting your merged contributions from upstream back to your fork.

    I wrote that when I caught myself explaining that in great detail in a pull request comment; I thought that might be useful for a lot more people out there. So here it is.

Bugfixes

  • Fixed GitHub issue #24:

    During directory reading, subdirectories would get out of sync when opening a tree branch.

    It looks like QDirStat's tree display was a bit too dynamic for the concepts of the underlying Qt classes (QTreeView / QAbstractItemModel): During reading, QDirStat would sort the tree by the number of pending read jobs. That number is constantly changing, so the sort order would also constantly need to change. This is very hard to do properly with the limitations those underlying classes impose; basically it would require a reset of all the data the QTreeView keeps, thus making it forget things like its current scrollbar position or which tree branches were expanded or collapsed. That would make the user interface pretty much unusable.

    So the fix for this is to not sort by read jobs, but by directory names instead since they don't change all the time. The user can still sort by any other column, but that sort is a momentary thing that might become invalid moments later as data (accumulated sizes, number of child items) are updated. Everybody please notice that this is a known limitation and any complaints about that will flatly be rejected. The alternative would be to not allow the user to sort at all during directory reading, and that is certainly a lot less desirable.

  • Fixed GitHub issue #54:

    Crash with cleanup action with "Refresh Policy: Refresh this item" when more than one file item was selected.

  • Fixed crash upon refresh when deleting the toplevel directory via cleanup.

Acknowledgements

This release was supported by my employer SUSE Linux GmbH with the SUSE Hack Week where R&D people are free to spend one whole working week on their favourite Open Source projects. SUSE does this roughly every 6-8 months to support the Open Source movement.

About one third of my time working on this release was covered by the Hack Week.

Assets 2

@shundhammer shundhammer released this Jan 3, 2017 · 838 commits to master since this release

Release Announcement

The QDirStat project proudly presents its latest release: QDirStat 1.2.

This release is focused on Btrfs improvements, so upgrading to this release is highly recommended for Btrfs users:

If you used QDirStat to scan a Btrfs partition, any subvolumes of that partition were not scanned (see GitHub issue #39).

Btrfs subvolumes were treated just like ordinary mount points (which, to all intents and purposes, they are). So you might have wondered why the df command shows your 40 GB root filesystem as 97% full, yet QDirStat shows only about 7 GB. The rest might be hidden in subvolumes.

QDirStat stops reading at mount points - which only makes sense because normally you want to know what eats up the disk space on that one partition that is filling up, not on any others like /home that are mounted there. Unfortunately, a Btrfs subvolume is also just another mount point, and QDirStat would stop reading there, too - at /var/log, at /var/spool, at /var/lib/libvirt etc.; a typical Btrfs root filesystem has about a dozen subvolumes, and all files in them were disregarded by QDirStat.

This is now fixed: Despite Btrfs doing its best to make this difficult (using one single privileged system call for all its functionality, including simple info calls), QDirStat now detects if a mount point is a Btrfs subvolume and continues reading if it is. QDirStat uses /proc/mounts (or, if this is not available, /etc/mtab) to find this out.

This is fixed in the qdirstat-cache-writer script, too.

Other Changes

  • Bernhard Walle contributed some patches for MacOS X support.
    Thanks, Bernhard!

  • Fixed GitHub issue #40:
    Crash without useful error message when no display available

    When ssh'ing without -X to a remote machine and starting QDirStat there, it would just dump core and not issue any meaningful message. The fatal error message was only in the log file:

    <ERROR> :0 (): QXcbConnection: Could not connect to display

    Now this message is also repeated on stderr, and in this particular case ("Could not connect to display"), it does not dump core any more, but just exits with error code 1.

About QDirStat

QDirStat is a graphical application to show where your disk space has gone and to help you to clean it up.

This is a Qt-only port of the old Qt3/KDE3-based KDirStat, now based on the latest Qt 5. It does not need any KDE libs or infrastructure. It runs on every X11-based desktop on Linux, BSD and other Unix-like systems.

QDirStat has a number of new features compared to KDirStat. To name a few:

  • Multi-selection in both the tree and the treemap.
  • Unlimited number of user-defined cleanup actions.
  • Properly show errors of cleanup actions (and their output, if desired).
  • File categories (MIME types) and their treemap color are now configurable.
  • Exclude rules for directories are easily configurable.
  • Desktop-agnostic; no longer relies on KDE or any other specific desktop.

Target Platforms

  • Linux
  • BSD
  • Unix-like systems

Target Audience

  • System administrators
  • Advanced users
  • Interested users who are not afraid of learning new things

Screenshots

Main Window Screenshot

Main window screenshot - notice the multi-selection in the tree and the treemap

Cleanup Action Output Screenshot

Screenshot of output during cleanup actions. Of course this window is purely optional.

Cleanup Action Configuration Screenshot

Screenshot of cleanup configuration.

MIME Categories Configuration Screenshot

Screenshot of MIME category configuration where you can set the treemap colors for different file types (MIME types), complete with a real treemap widget as preview.

Exclude Rules Configuration Screenshot

Screenshot of the exclude rules configuration where you can define rules which directories to exclude when reading directory trees.

More Information

https://github.com/shundhammer/qdirstat

Assets 2

@shundhammer shundhammer released this Oct 31, 2016 · 865 commits to master since this release

Release Announcement

The QDirStat project proudly presents its latest release: QDirStat 1.1-Pumpkin.

This is mostly a bug fix release with some small features thrown in.

It was about time for another official release to get the accumulated fixes and
small changes out into the world. Since the release day was Halloween, this release is named Pumpkin .

The last stable release, V1.0, was in mid-May (2016-05-16). Since then, there
were 5 bug fixes and one small feature (the config file split up into
independent parts so admins can provide presets to their users without
overwriting the complete configuration), all described in greater detail
below.

Changes

  • Fixed GitHub issue #32:
    %p does not escape single quotes properly

    If you have a file name like Don't do this.txt (with a quote character in
    the name), the shell used when executing a cleanup action with this would
    complain about unmatched single quotes.

    QDirStat had always escaped such single quotes, but not the way common
    shells (Bash, Zsh) expect it: They don't want a backslash in front of that
    embedded single quote. Rather, you need to terminate the string with a
    single quote, escape the embedded quote with a backslash (or put it into
    double quotes), and then re-open the old string with another single quote.

    Thus, 'Don't do this' becomes 'Don'\''t do this'.

    This is certainly not what most people expect. I just wonder how much other
    software is out there that does it the intuitive (yet wrong) way: Just
    escape the single quote with a backslash ('Don\'t do this').

    Of course, such file names should be avoided entirely, but you can't help
    some slightly broken MP3 ripper program doing it, so it needs to be handled
    correctly.

  • Fixed GitHub issue #31:
    Segfault with cleanup action while reading directories

    Now disabling cleanups that have a refresh policy other than "No Refresh"
    while directory reading is in progress; otherwise the re-read when the
    cleanup action has finished clashes with the directory read already in
    progress.

    This is not an optimal solution, but a very pragmatic one; the optimal
    solution might queue updates and execute them after the main read is done.

  • Fixed GitHub issue #33:
    Added command line option --slow-update (or -s) for slow remote X connections.

  • Fixed GitHub issue #23:

    The internal cache writer would sometimes generate incorrect cache files
    because of buggy URL escaping resulting in an empty file name and thus
    invalid cache file syntax. This affected file names with colons (which is
    weird, but legal).

    One of these days I'm going to throw out all that QUrl stuff and replace the
    few things that I need with something that actually works consistently and
    not just under optimum conditions.

  • Fixed GitHub issue #22:

    Cache files containing the root file system would not display correctly or
    segfault under certain conditions. This is now fixed.

  • Added "Refresh All" action to the main window tool bar. I had consciously
    avoided that because it's just too tempting to re-read the complete
    directory tree rather than think about what actually might have changed and
    then refresh just that, but it has become so common to use that action in
    web browsers that I found myself missing that more and more. And re-reading
    is not that expensive on today's mainstream PCs.

  • Fixed GitHub issue #21:

    When started from a desktop menu, i.e. without any command line parameters,
    QDirStat would not prompt for a directory to read, but read the current
    directory (typically the user's home directory) right away.

  • More graceful handling for nonexisting paths specified on the commmand
    line: It now no longer just throws an exception right after starting the
    program (which looks like a crash to the unwary user), but posts an error
    popup instead and then asks for a directory to read.

  • Split up config file into four separate ones below ~/.config/QDirStat:

    • QDirStat.conf
    • QDirStat-cleanup.conf
    • QDirStat-exclude.conf
    • QDirStat-mime.conf

    This should make it much easier for site administrators to provide their
    own site-wide cleanup actions, exclude rules, or MIME categories. I did
    this with this in mind:

    http://moo.nac.uci.edu/~hjm/kdirstat/kdirstat-for-clusters.html

    Here, they describe how users should overwrite their KDirStat config file
    with one provided by the site admin so all users have those carefully
    crafted cleanup actions. But that also means that all other settings get
    lost each time there is a change in any of those commands, and users have
    to update that config file again.

    With the latest change, it is now possible to only replace the cleanup
    action config (QDirStat-cleanup.conf) and leave everything else untouched.

    Notice that this is far from a perfect solution; all cleanup actions the
    user added himself still get lost. But doing this perfectly might pretty
    quickly become an overengineered solution that would be hard to understand
    for everybody.

    As for migration from previous single-file configurations, QDirStat does
    that automatically: It reads the single file and moves the respective parts
    where they belong. No need to bother with any migration scrips or anything
    like that.

Assets 2
You can’t perform that action at this time.