Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 23 additions & 6 deletions docs/guides/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,19 +136,36 @@ The simplest way to have the correct layout is to copy to a directory with the c
... so it is a good choice when reading a file sequentially or in large chunks, but if one reads shorter chunks in random order it might be better to reduce the size, the performance will be smaller, but the performance of your application might actually increase.
See the [Lustre documentation](https://doc.lustre.org/lustre_manual.xhtml#managingstripingfreespace) for more information.


!!! example "Settings for large files"
*Remember:* Settings only apply to files added to the directory after this command.
```console
lfs setstripe --stripe-count -1 --stripe-size 4M <big_files_dir>`
```bash
lfs setstripe --stripe-count -1 --stripe-size 4M <big_files_dir>
```

*Remember:* Settings applied with `lfs setstripe` only apply to files added to the directory after this command.
[Use `lfs migrate` to update the settings for existing files][ref-guides-storage-examples-lfs-migrate].

Lustre also supports composite layouts, switching from one layout to another at a given size `--component-end` (`-E`).
With it it is possible to create a Progressive file layout switching `--stripe-count` (`-c`), `--stripe-size` (`-S`), so that fewer locks are required for smaller files, but load is distributed for larger files.

!!! example "Good default settings"
```console
lfs setstripe -E 4M -c 1 -E 64M -c 4 -E -1 -c -1 -S 4M <base_dir>
```bash
lfs setstripe --component-end 4M --stripe-count 1 --component-end 64M --stripe-count 4 --component-end -1 --stripe-count -1 --stripe-size 4M <base_dir>
```

[](){#ref-guides-storage-examples-lfs-migrate}
!!! example "Updating settings for existing files"
While `lfs setstripe` applies to newly created files, `lfs migrate` can be used to re-layout existing files.
For example, to set the recommended settings above on an existing file:
```bash
lfs migrate --component-end 4M --stripe-count 1 --component-end 64M --stripe-count 4 --component-end -1 --stripe-count -1 --stripe-size 4M <file>
```

Alternatively, to migrate all files recursively in a directory:
```bash
lfs find --type file <base_dir> | xargs lfs migrate --verbose --component-end 4M --stripe-count 1 --component-end 64M --stripe-count 4 --component-end -1 --stripe-count -1 --stripe-size 4M
```
The `--verbose` flag makes `lfs migrate` print the path of each file after the file has been migrated.
Also note the use of `lfs find` instead of regular `find` as `lfs` can more efficiently retrieve the list of files recursively.

### Iopsstor vs Capstor

Expand Down