Skip to content

Conversation

@Torstein-Eide
Copy link
Contributor

Hi Dave
I have made a page with examples for balance of BTRFS, so it easier to understand how to use it.

boryas and others added 13 commits June 7, 2022 13:59
Adapt the existing send/receive tests by passing '-o compress-force' to
the mount commands in a new test. After writing a few files in the
various compression formats, send/receive them with and without
--force-decompress to test both the encoded_write path and the fallback
to decode+write.

Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
Fix the following warnings:

    [SPHINX] man
../CHANGES:37: ERROR: Unexpected indentation.
../CHANGES:1183: WARNING: Block quote ends without a blank line; unexpected unindent.
./Documentation/DocConventions.rst:31: ERROR: Unexpected indentation.
./Documentation/Source-repositories.rst:2: WARNING: Duplicate explicit target name: "web access".
./Documentation/DocConventions.rst: WARNING: document isn't included in any toctree

The free format of CHANGES sometimes does not align with RST so fix it
so it's visually similar in both formats. Remove RST references to 'web
access', URLs are parsed and rendered clickable and we don't have full
reference list yet. Doc conventions have been updated to RST but not
finalized so put it to TODO section.

Signed-off-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Rename the section to contain btrfs-progs as there will be more
sections, eg. kernel or by feature.

Signed-off-by: David Sterba <dsterba@suse.com>
Copy contents from https://btrfs.wiki.kernel.org/index.php/Changelog#By_feature

The formatting is done by a definition and list, instead of a table.
Unfortunatelly RST does not wrap long text in table cells so the width
exceeds visible area.

Signed-off-by: David Sterba <dsterba@suse.com>
Copy contents from wiki page Problem_FAQ.

Signed-off-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There are build-time options for LZO and ZSTD support, the stream v2+
supports compression. The help text lists what has been compiled in,
similar to what 'restore' does, with a similar limitation that a stream
with compressed data cannot be processed if any of the extents is
compressed.

Signed-off-by: David Sterba <dsterba@suse.com>
Now that LZO and ZSTD are optional for not just restore, rename the
build variables to a more generic name and update configure summary.

Signed-off-by: David Sterba <dsterba@suse.com>
Add constant for initial value to avoid unexpected clashes with user
defined getopt values and shift the common size getopt values.

Signed-off-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The initial proposal for file attributes was built on simply doing
SETFLAGS but this builds on an old and non-extensible interface that has
no direct mapping for all inode flags. There's a unified interface
fileattr that covers file attributes and xflags, it should be possible
to add new bits.

On the protocol level the value is copied as-is in the original inode
but this does not provide enough information how to apply the bits on
the receiving side. Eg. IMMUTABLE flag prevents any changes to the file
and has to be handled manually.

The receiving side does not apply the bits yet, only parses it from the
stream.

Signed-off-by: David Sterba <dsterba@suse.com>

.. code-block:: bash

btrfs balance start -v -mconvert=raid1 -dconvert=raid1 $BTRFS_MOUNT
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could use soft here too. With soft, if the balance is interrupted, it can be restarted with the same command line.

There is an example down below with soft but I'm not sure why there's a separate example just for soft.

Converting without soft is useful for custom userspace balance tools, but for simple profile conversions a user almost never wants to run convert without it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sound to me that soft should be the default for most user.

Do you have a example when not soft should be used?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you are doing something that requires moving the chunk and also specifying the target profile. e.g. if you need to make an unallocated hole so that you can add a new disk to a raid1c3, you might temporarily downgrade a few chunks to raid1:

# Make space on each existing device by lowering the redundancy.
# If the data is already in raid1 profile then we still want it moved.
btrfs balance start -ddevid=1,limit=10,convert=raid1 $MOUNT_PT
btrfs balance start -ddevid=2,limit=10,convert=raid1 $MOUNT_PT
btrfs balance start -ddevid=3,limit=10,convert=raid1 $MOUNT_PT
# Add device, 4 disks now
btrfs dev add $NEW_DEV $MOUNT_PT
# Rebalance entire filesystem across 4 disks instead of 3
btrfs balance start -dconvert=raid1c3 $MOUNT_PT

Here almost all of the data is already in raid1c3 profile, but in this case we want to redistribute all of the data across 4 drives, so we don't want soft here. We want the 30 chunks that happen to be raid1 to become raid1c3 as we go.

We could do something like this:

# Convert 30 raid1 chunks back to raid1c3
btrfs balance start -dconvert=raid1c3,soft $MOUNT_PT
# Redistribute everything across 4 devices
btrfs balance start -d $MOUNT_PT

but this will process the 30 temporary raid1 block groups twice (once to convert them back to raid1c3, and again to redistribute the filesystem across the 4 devices).

Comment on lines 96 to 100
Only targets non `raid1` chunks.

.. code-block:: bash

btrfs balance start -mconvert=raid1,soft $BTRFS_MOUNT
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The command only targets metadata chunks. Data chunks will not be balanced if only -m is specified.

Generally if a user is in this sort of situation, they will want to run both conversions to ensure their chosen profiles are selected and no others. This can be done in a single command:

btrfs balance start -mconvert=raid1,soft -dconvert=raid1,soft $BTRFS_MOUNT

or multiple commands if it's necessary to force the order (e.g. due to space constraints):

btrfs balance start -dconvert=raid1,soft $BTRFS_MOUNT
btrfs balance start -mconvert=raid1,soft $BTRFS_MOUNT

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and system.

Is the point about due to space constraints, valid for the example?
Or more fit to be in the ch-balance-filters.rst.

Before changing profiles, make sure there is enough unallocated space on
existing drives to create new metadata block groups (for filesystems
over 50GB, this is 1GB * (number_of_devices + 2)).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The space constraints always have to be observed. A filesystem with unequal disk sizes and raid1 profile can fill up with ordinary data writes if there isn't enough unallocated space.

The thing that makes balance different is that a new profile can be introduced, which changes the requirements for future allocations abruptly.

@Torstein-Eide
Copy link
Contributor Author

I have squashed the commits, and add changes to ch-balance-filters.rst, to reflect the input from @Zygo .

ERROR: 'soft' option can be used only when converting profiles
@kdave kdave force-pushed the devel branch 2 times, most recently from e3c0fe2 to 588429c Compare August 16, 2022 13:23
@kdave kdave force-pushed the devel branch 8 times, most recently from 54326e2 to ad2a1bf Compare September 21, 2022 13:42
@kdave kdave force-pushed the devel branch 5 times, most recently from 7bfedf8 to cda1f14 Compare February 22, 2023 02:06
@kdave kdave force-pushed the devel branch 4 times, most recently from 63a7417 to 4fc291a Compare March 1, 2023 14:20
@kdave kdave force-pushed the devel branch 4 times, most recently from e53f194 to 405f4c7 Compare April 26, 2023 22:51
@kdave kdave force-pushed the devel branch 3 times, most recently from af681f4 to 1628b2f Compare May 10, 2023 19:47
@kdave kdave force-pushed the devel branch 2 times, most recently from a8c6f08 to 34c5de3 Compare May 22, 2023 12:32
@kdave kdave force-pushed the devel branch 5 times, most recently from 5e02de0 to adfc8a9 Compare May 26, 2023 17:53
@kdave
Copy link
Owner

kdave commented Jun 9, 2023

Thanks for the PR and comments, merged to devel. I've updated formatting and wording.

@kdave kdave closed this Jun 9, 2023
@kdave kdave added this to the v6.3.2 milestone Jun 9, 2023
kdave pushed a commit that referenced this pull request Jun 9, 2023
Add more examples and explanations how the filters can be used.

Pull-request: #486
Author: Eideen
Signed-off-by: David Sterba <dsterba@suse.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants