Skip to content

2.31.0.0-b242

2.31.0.0-b242: [#32229,#32352] DocDB: Add support for labeled disks in fs_data_dirs
@samsonshaji samsonshaji tagged this 08 Jul 21:40 
Summary:
Adds tiered-storage groundwork: labeled `--fs_data_dirs` with a fixed storage-class set `{ssd, hdd}`, default `ssd` for unlabeled paths, validation of explicit labels at tserver startup, and propagation of `storage_tier` on heartbeat `path_metrics` to the master.

- `--fs_data_dirs`: tokens `path[:tier`] into path and tier; invalid explicit tiers fail init with a clear error; unlabeled paths are defaulted to `ssd`.
- `FsManager`: `ValidStorageTiers()`, `IsValidStorageTier()`, tier maps and accessors; WAL defaulting still strips `:tier` from data dirs when building `--fs_wal_dirs`.
- Master path: optional `path_metrics.storage_tier` in master proto; tserver heartbeat sets tier per data root; master stores/exposes it (including JSON on tablet-servers API).

**Upgrade/Rollback safety:**

Describe how this change handles upgrade and rollback of YugabyteDB.
  - New format: `storage_tier` is an optional field on `PathMetrics`. Old masters ignore unknown fields; old tservers do not send it.
  - New tservers on old masters: heartbeats remain valid; tier may not appear until master is upgraded.
  - New master on old tservers: path metrics without `storage_tier` are treated as empty on the master side (existing JSON/API checks skip empty tier).
  - Config / behavior: Clusters that never use `:tier` behave as before; all data dirs are classified as `ssd`.
  - Rollback to a binary without tier parsing: use plain paths only (no `:tier` suffixes).

What Test/Preview/AutoFlag is used to guard the feature?
Feature guard: No AutoFlag; behavior is backward compatible via optional proto + default tier for unlabeled disks. Explicit invalid tiers are rejected only on new binaries that run tier validation.

Test Plan:
Ran unit tests using following command:

```
./yb_build.sh release --cxx-test fs_manager-test \
  --gtest_filter 'StorageTierTest.*' \
  --sj --skip-pg-parquet --no-odyssey --no-ybc
```

Integration test command:

```
./yb_build.sh release --cxx-test tserver_metrics_heartbeat_data_provider-itest \
  --gtest_filter 'TServerMetricsHeartbeatDataProviderITest.PathMetricsCarryStorageTier' \
  --sj --skip-pg-parquet --no-odyssey --no-ybc
```

Manual testing:
Ran the following commands on terminals.
Teminal 1 (master):

```
./build/latest/bin/yb-master \
  --master_addresses=127.0.0.1:7100 \
  --rpc_bind_addresses=127.0.0.1:7100 \
  --fs_data_dirs=/tmp/yb_master_data \
  --replication_factor=1
```

Terminal 2 (tserver):

```
./build/latest/bin/yb-tserver \
  --tserver_master_addrs=127.0.0.1:7100 \
  --rpc_bind_addresses=127.0.0.1:9100 \
  --fs_data_dirs=/tmp/yb_data_ssd:ssd,/tmp/yb_data_hdd:hdd,/tmp/yb_data:samson_tier \
  --fs_wal_dirs=/tmp/yb_data_nvme
```

Terminal 3 (curl master to see output):
```
curl -s http://127.0.0.1:7000/api/v1/tablet-servers | python3 -m json.tool
```

Screenshot of output:
- Right terminal shows error message for having a tier label that is NOT in `{ssd, hdd}`.
- Left terminal shows output of the curl, which includes the newly added `storage_tier` field.
{F513874}

Reviewers: mhaddad, anijhawan

Reviewed By: mhaddad

Subscribers: svc_phabricator, ybase

Differential Revision: https://phorge.dev.yugabyte.com/D54935
Assets 2
Loading