Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Destination filesystem almost full #2181

Closed
frapell opened this issue Jul 4, 2019 · 20 comments

Comments

@frapell
Copy link

commented Jul 4, 2019

Description of the problem

I was encoding a video to an external pendrive directly, and right at the middle of it, the process paused because the filesystem was getting full while it still had 10GB free (Full encode was 1.5GB, so there was plenty of space left)

I see a closed issue about this same issue here gh-853

HandBrake version (e.g., 1.0.0)

1.2.1

Operating system and version (e.g., Ubuntu 18.04 LTS, macOS 10.14 Mojave, Windows 10 1809)

Ubuntu bionic 18.04

Error message text or screenshot

D-o08nrXkAAofOp

HandBrake Activity Log (see https://handbrake.fr/docs/en/latest/help/activity-log.html)

I don't have one since I restarted Handbrake and seems the old log is deleted when a new one is created :(

@sr55

This comment has been minimized.

Copy link
Contributor

commented Jul 4, 2019

I'm not sure what your asking here?

The app is designed to pause things when the storage volume starts getting low on space. 10GB is a reasonable default limit to do so.

The limit is configurable in preferences so you can adjust based on your use case. For example, if it's a boot drive, you'd want a considerable amount of free space to cover for things like hibernation, swap etc. (Assuming your not booting from USB, you could set to a lower limit, or just turn it off)

@frapell

This comment has been minimized.

Copy link
Author

commented Jul 5, 2019

@sr55 Well, I am not a regular user of the application, let me explain you why I created this ticket and why I consider it a bug:

  • I needed to copy a video file to a pendrive
  • Noticed the file was bigger than my free space
  • Downloaded the latest version of handbrake and installed it
  • Started to re-encode my video so it would fit my free space
  • Whole process would take an estimate of 30 minutes
  • Went to do something else, instead of staring at my computer for 30 minutes
  • Came back 45 minutes later, to find the process was stopped at 20% with the error shown
  • Had to wait 20 more minutes for the process to complete

The final result was a 1.5GB file, so issuing a warning at 10GB doesn't sound reasonable... Even less so in a 16GB pendrive.

Again, this is as an end user experience... A warning like this does sound useful, but I think the defaults need some tweaking... Maybe have different defaults depending on the quality of the encoding? or maybe have a lower, more reasonable default (maybe 1 or 2 GB) and let the user increase it if need be?

FWIW, I don't think an application like this should take into consideration swap or hibernation needs, becase it is so different from system to system that you will never get it right, and you end up getting in the way of some users (like my case).

Anyway, if you don't consider this a bug, feel free to close it

Cheers!

@bradleysepos

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

That certainly is frustrating, and not the experience we intend of course. On the other hand, this feature has helped many users avoid system instability due to a full system disk; we know this because reports have become nil and now we only occasionally see a complaint about the protection feature.

I do think we need to re-evaluate the 10 GB limit. Perhaps something tighter like 2 GB would be enough. Additionally, if possible, we might want to look into an option to have the limit only apply to the boot disk.

Anyway, thanks for sharing your experience. For now, adjust the setting in preference to your needs and we'll discuss how we can reduce friction.

@bradleysepos

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

Marking as an enhancement. @jstebbins @galad87 @sr55 can we agree on a tighter constraint, maybe only 2 GB (likely enough to preserve system stability), and is it possible to add a checkbox to apply the constraint only to a boot volume?

@sr55

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

I'd argue against lowering it. By rights it should be no less than system ram +2 GB. So most commonly that would be 10GB, 18GB or 34GB for the vast majority.

10GB is a reasonable compromise as it'll catch the majority use case. 2GB will cause a lot of folks to get caught out again which would defeat the purpose.

As for boot drive, it's not quite that simple. You'd have to go looking at the system to see how/where it's configured to store it's various assets. It's not necessarily the boot drive.

@bradleysepos

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

Does Windows not keep a hibernation file static on disk? macOS does.

@sr55

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

I believe so, but it's not always fully allocated it seems.

@frapell

This comment has been minimized.

Copy link
Author

commented Jul 5, 2019

I am not a regular user of this software, and if you used to have a lot of reports of people running out of space, and this implementation has helped turn all those reports down, I would then agree that it is positive and maybe you shouldn't change it.

Maybe (And I am just thinking out loud) an enhancement would be to display a big flashy warning at a high threshold (say 10GB), but do not stop the process until a more critical value is met (say 2GB)

Also, adding information in the warning on how to tweak or even disable this feature would be useful for non-regular users (I knew about it being a configurable feature after @sr55 first reply to this thread)

@bradleysepos

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

Ah. Perhaps an interface like this would make sense, at least on Windows:


Disk space warning level:

(o) 10 GB (System RAM + 2 GB, Default)
(  ) Custom level:
      [            ] GB

[x] Apply disk space warning level only to system disk

100 MB minimum warning level applies to all disks


This would at least make it more clear what is happening and why, while allowing users encoding to external/non-system media to avoid the entire problem.

@bradleysepos

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

For reference, since the macOS VM/hibernation file is always present, we don't have to worry about this at all. The preference looks like this and 2 GB seems sane:

Screen Shot 2019-07-05 at 3 08 18 PM

The wording "Pause queue..." seems more indicative of what is going to happen as well.

@sr55

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

I suspect the Mac GUI, like the Windows GUI (probably) isn't doing active monitoring. I honestly don't recall. I think it only checks on start. It would be better if it was active in my opinion but it was quicker to implement as-is

@sr55

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

Confirmed, Windows UI only paused at the start of a job.

@bradleysepos

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

A high/low threshold UI could be something like this:


Disk space warning levels:

[x] Warn when disk space is below:
    (o) 10 GB (System RAM + 2 GB)
    (  ) Custom level:
          [            ] GB

[x] Pause queue when disk space is below:
    (o) 2 GB
    (  ) Custom level:
          [            ] GB

[x] Levels apply to system disk only

100 MB minimum warning level applies to all disks


@bradleysepos

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

Here's another idea, in the layout of the current Windows UI:


[x] Pause queue if system/hibernation disk space is lower than (GB):     [              ]
      System RAM (8 GB) + 2 GB recommended

[x] Pause queue if non-system disk space is lower than (GB):                  [              ]
      2 GB recommended


Currently, the two relevant lines in the preferences are disconnected by an unrelated preference (libdvdnav) between them.

This idea combines the checkbox (enable) and textbox (amount) into a single line, adds recommended levels (ideally in a smaller font size just below the label), and separates the behavior for system and non-system disks.

@bradleysepos

This comment has been minimized.

Copy link
Contributor

commented Jul 5, 2019

We were also talking on IRC about how to warn the user. I'd prefer a status message to a modal dialog. Instead of "Queue paused" or similar when manually paused, perhaps something like "Queue paused due to low disk space" when automatically paused due to exceeding one of the thresholds.

A system notification would seem to be in order as well.

sr55 added a commit that referenced this issue Jul 5, 2019

WinGui: Build out code for active monitoring of storage and battery p…
…ower on the system. (Similar to what the LinUI does)

- Automatic pause on "Low" or "Critical" battery alarms.  The % level is set in Windows power settings. Automatic Resume when AC returns, if it was paused by an alarm.
- Automatic encode pause when destination drive drops below 2GB.  (May make this a preference set later)
- Behaviour of pause queue on low disk space with a user defined level in preferences is unchanged.

#2109 #2181
@jstebbins

This comment has been minimized.

Copy link
Contributor

commented Jul 11, 2019

Just for reference, I started a queue full of stuff a week ago then went on a trip. When I got back (a week later), my queue was half done, paused, and this dialog was waiting for me. I moved the finished encodes to their final destinations to free up space and resumed the queue. It saved me from having diagnose why a bunch of encodes failed.

In this case, the disk being written to was not by system partition, so the system would have remained stable. But it would have filled my user partition which could have resulted in some mysterious behaviour that I would not have initially understood till diagnosing the encode failures. I.e. I may not have been able to open the activity log since editors usually create temp file backups of the edited file.

I agree that the default limit could be substantially lower. On linux 1GB is more than enough. When I set it initially, I was thinking in terms of encoding to HDD where sizes are massive and 10GB is 0.0025% of the total space. Percentage-wise, it seemed like a good number ;)

@jstebbins

This comment has been minimized.

Copy link
Contributor

commented Jul 11, 2019

FYI, preferences for this feature on linux looks like:
hb-monitor-disk

@sr55

This comment has been minimized.

Copy link
Contributor

commented Jul 14, 2019

I don't really see value in spending time adding complexity to deal with drive types. Folks have all kinds of wonderful setups and thinking about all the scenarios that could come up just isn't a good use of time.

Inst read, I'd rather just keep it simple.

I've matched the Mac UI's 2GB default (and I've added Active monitoring of the destination drive like the Linux GUI so that it's far less likely to run the drive to 0)

Since the Linux UI already has active monitoring, I'd say set it to 2 and call it a day.

It'll still trip folks up from time to time, but that's exactly why the preference is there.

@jstebbins

This comment has been minimized.

Copy link
Contributor

commented Jul 15, 2019

I've reduced the default to 2GB on linux to match the other UIs 1e6078b

@sr55

This comment has been minimized.

Copy link
Contributor

commented Jul 15, 2019

Fair enough. Think we call it a day then. Mac is already lower :)

@sr55 sr55 closed this Jul 15, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.