Syntax error on Debian Stretch, no output

[jph76]

Running btrfs-du stops at the first subvolume with the following error message:

root@proliant:~# btrfs-du /
Subvolume                                                         Total  Exclusive  ID
─────────────────────────────────────────────────────────────────────────────────────────
home/lxc/.local/share/lxc/lms                                                       963
/usr/local/sbin/btrfs-du: line 118: EXCL_TOTAL +  : syntax error: operand expected (error token is "+  ")
─────────────────────────────────────────────────────────────────────────────────────────
Total exclusive data                                                             0Bytes

Debian Stretch amd64, btrfs-progs 4.7.3-1, linux-image-4.9.0.5-amd64 4.9.65-3+deb9u2

[nachoparker]

Hi,

Yes, I realized that on Raspbian 9 it doesn't work.

I run it on btrfs-progs v4.14, and @jpluimers on v4.13 and v3.12

Workarounds would need to be implemented to account for this case

[jpluimers]

Can you ping me in 3 weeks time? I'm about to leave for a 2 week holiday then need a few days with appointments around the care of my mentally retarded brother.

[nachoparker]

@jpluimers enjoy your holidays ;)

[ArneBachmann]

Hi, same problem here on OpenSuse.

ash@mega:~/Desktop/svn> sudo sh /usr/local/sbin/btrfs-du 
Subvolume                                                         Total  Exclusive  ID        
─────────────────────────────────────────────────────────────────────────────────────────
@                                                                                   257       
/usr/local/sbin/btrfs-du: line 118: EXCL_TOTAL +  : syntax error: operand expected (error token is "+  ")

[mikkokaar]

Got it working by:

1. Change line 118 in script to:

EXC_TMP=${EXC_[$i]}
EXCL_TOTAL=$(( EXCL_TOTAL + EXC_TMP ))

2. btrfs quota enable /
3. btrfs quota rescan /

btrfs-tools 4.4-1ubuntu1
GNU bash, version 4.3.48

[ArneBachmann]

Yeah that did the trick! I create automatic snapshots on my home partition every 5 minutes:

Subvolume                                                         Total  Exclusive  ID        
─────────────────────────────────────────────────────────────────────────────────────────
@                                                              59.73GiB   12.91MiB  257       
.snapshots                                                     96.00KiB   96.00KiB  335       
.snapshots/1/snapshot                                          58.67GiB  418.23MiB  336       
.snapshots/358/snapshot                                        58.68GiB  414.96MiB  693       
.snapshots/1259/snapshot                                       59.05GiB  374.88MiB  1594      
.snapshots/2003/snapshot                                       59.09GiB  383.19MiB  2338      
.snapshots/3161/snapshot                                       59.32GiB  269.11MiB  3496      
.snapshots/3209/snapshot                                       59.40GiB   81.25MiB  3544      
.snapshots/3227/snapshot                                       59.43GiB  127.40MiB  3562      
.snapshots/3449/snapshot                                       59.45GiB  163.69MiB  3784      
.snapshots/3455/snapshot                                       59.38GiB   91.08MiB  3790      
.snapshots/3521/snapshot                                       59.58GiB  280.88MiB  3856      
.snapshots/3538/snapshot                                       59.56GiB   36.32MiB  3873      
.snapshots/3540/snapshot                                       59.56GiB   31.93MiB  3875      
.snapshots/3547/snapshot                                       59.55GiB   12.26MiB  3882      
.snapshots/3552/snapshot                                       59.55GiB   12.88MiB  3887      
.snapshots/3555/snapshot                                       59.55GiB   28.49MiB  3890      
.snapshots/3559/snapshot                                       59.55GiB   16.65MiB  3894      
.snapshots/3565/snapshot                                       59.55GiB    7.54MiB  3900      
.snapshots/3566/snapshot                                       59.55GiB    7.71MiB  3901      
.snapshots/3573/snapshot                                       59.56GiB  122.76MiB  3908      
.snapshots/3585/snapshot                                       59.67GiB  236.78MiB  3920      
.snapshots/3591/snapshot                                       59.67GiB   27.96MiB  3926      
.snapshots/3592/snapshot                                       59.70GiB    5.28MiB  3927      
.snapshots/3596/snapshot                                       59.71GiB   12.53MiB  3931      
.snapshots/3599/snapshot                                       59.72GiB   76.78MiB  3934      
.snapshots/3608/snapshot                                       59.71GiB  564.00KiB  3943      
.snapshots/3609/snapshot                                       59.71GiB    1.69MiB  3944      
.snapshots/3614/snapshot                                       59.72GiB    2.12MiB  3949      
.snapshots/3615/snapshot                                       59.72GiB    1.24MiB  3950      
.snapshots/3616/snapshot                                       59.72GiB    1.21MiB  3951      
.snapshots/3617/snapshot                                       59.72GiB    1.31MiB  3952      
.snapshots/3618/snapshot                                       59.72GiB    3.09MiB  3953      
.snapshots/3619/snapshot                                       59.72GiB  476.00KiB  3954      
.snapshots/3620/snapshot                                       59.72GiB  396.00KiB  3955      
.snapshots/3621/snapshot                                       59.72GiB  244.00KiB  3956      
.snapshots/3622/snapshot                                       59.72GiB  208.00KiB  3957      
.snapshots/3623/snapshot                                       59.72GiB  284.00KiB  3958      
.snapshots/3624/snapshot                                       59.72GiB  176.00KiB  3959      
.snapshots/3625/snapshot                                       59.72GiB  240.00KiB  3960      
.snapshots/3626/snapshot                                       59.72GiB    3.25MiB  3961      
.snapshots/3627/snapshot                                       59.72GiB    1.80MiB  3962      
.snapshots/3628/snapshot                                       59.72GiB    3.93MiB  3963      
.snapshots/3629/snapshot                                       59.73GiB    4.31MiB  3964      
.snapshots/3630/snapshot                                       59.73GiB    3.89MiB  3965      
.snapshots/3631/snapshot                                       59.73GiB    3.07MiB  3966      
.snapshots/3632/snapshot                                       59.73GiB    2.96MiB  3967      
.snapshots/3633/snapshot                                       59.73GiB    3.02MiB  3968      
.snapshots/3634/snapshot                                       59.73GiB    3.43MiB  3969      
.snapshots/3635/snapshot                                       59.73GiB  268.00KiB  3970      
.snapshots/3636/snapshot                                       59.73GiB  192.00KiB  3971      
.snapshots/3637/snapshot                                       59.73GiB  208.00KiB  3972      
.snapshots/3638/snapshot                                       59.73GiB  208.00KiB  3973      
.snapshots/3639/snapshot                                       59.73GiB  176.00KiB  3974      
.snapshots/3640/snapshot                                       59.73GiB  176.00KiB  3975      
.snapshots/3641/snapshot                                       59.73GiB  176.00KiB  3976      
.snapshots/3642/snapshot                                       59.73GiB  176.00KiB  3977      
.snapshots/3643/snapshot                                       59.73GiB  256.00KiB  3978      
.snapshots/3644/snapshot                                       59.73GiB  468.00KiB  3979      
.snapshots/3645/snapshot                                       59.73GiB    5.04MiB  3980      
.snapshots/3646/snapshot                                       59.73GiB    3.71MiB  3981      
.snapshots/3647/snapshot                                       59.73GiB    4.21MiB  3982      
.snapshots/3648/snapshot                                       59.73GiB    7.41MiB  3983      
.snapshots/3649/snapshot                                       59.73GiB    3.80MiB  3984      
.snapshots/3650/snapshot                                       59.73GiB    3.55MiB  3985      
─────────────────────────────────────────────────────────────────────────────────────────
Total exclusive data                                                            3.24GiB

[nachoparker]

@mikkokaar @ArneBachmann if you find a way of auto-enabling quota (disabling it afterwards) if it is not enabled on your systems you can send a PR and we'll support your use case ;)

[ArneBachmann]

It's not that easy (an decision), because enabling quotas 1) can have an impact on the system speed, and 2) takes several minutes to compute actual storage occupation.
An automatic enable/du/disable could probably take up to an hour of time. On my 70GiB SSD it was 10 minutes.

[nachoparker]

@ArneBachmann I know. In my HDD it takes around a minute... it's not an easy decision indeed that's why I decided to disable it after du calculation. This is what current code does: if quota is not enabled, enable it, wait for scan, calculate, disable it.

If the user decides to leave quotas enabled, then there would be no need to wait, but that's their decision. Thoughts?

[ArneBachmann]

Why not mount one snapshot after the other and do a normal du? Could be slow but works always (unless snapshot was already removed before mounting).

[nachoparker]

Because BTRFS is a copy-on-write filesystem, so commands such as du and df are not accurate. The point of snapshots is that most data is de-duplicated so takes zero extra space

[backupandforkacc]

I still have the issue in debian stretch. Any solution or workaround?

[ruliane]

When quotas are disabled on the volume, btrfs qgroup show fails with the following :

ERROR: can't perform the search - No such file or directory
ERROR: can't list qgroups: No such file or directory

Scripts fails to parse this output.

[ruliane]

PR pushed, reviews are welcome before commit.

[nachoparker]

thanks!