Skip to content

Commit

Permalink
Refactor chunkserver to generate layers of snapshots
Browse files Browse the repository at this point in the history
A chunk might have multiple snapshots when we remove the assumption
that snapshots are always exported and deleted afterwards.

Changes are:
1. Introduce CSSnapshots to manage snapshots;
2. `correctedSn' no longer used for CoW logic.
3. Two new fields are introduced to `ChunkRequest':
  - snapSn, the snapshot sequence number;
  - snaps, all snapshot sequence numbers.

TODO:
1. managing of `snaps' field in client side.
2. CRUD of `internal' snapshots.
3. storage consumption and statistics - take care of CoW files.
4. Recover & clone of `internal' snapshots.
5. caching of meta objects, and take care of memory footprint.
6. optimize deletion - no need to merge if intended to delete both file and snapshots.
7. Remove `correctedSn' from `ChunkFileMetaPage' and 'CSChunkInfo'.

UT:
1. `test/chunkserver/clone/clone_test' still fails.
2. new unit tests is still under development.

Signed-off-by: David Lee <live4thee@gmail.com>
  • Loading branch information
live4thee committed Apr 12, 2023
1 parent 72cb758 commit 04462d8
Show file tree
Hide file tree
Showing 31 changed files with 708 additions and 389 deletions.
3 changes: 2 additions & 1 deletion proto/chunk.proto
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,13 @@ message ChunkRequest {
optional uint32 size = 7; // for read/write/clone 读取数据大小/写入数据大小/创建快照请求中表示请求创建的chunk大小
optional QosRequestParas deltaRho = 8; // for read/write
optional uint64 sn = 9; // for write/read snapshot 写请求中表示文件当前版本号,读快照请求中表示请求的chunk的版本号
optional uint64 correctedSn = 10; // for CreateCloneChunk/DeleteChunkSnapshotOrCorrectedSn 用于修改chunk的correctedSn
optional uint64 snapSn = 10; // for CreateCloneChunk/DeleteChunkSnapshot
optional string location = 11; // for CreateCloneChunk
optional string cloneFileSource = 12; // for write/read
optional uint64 cloneFileOffset = 13; // for write/read
optional uint64 fileId = 18; // for io fence
optional uint64 epoch = 19; // for io fence
repeated uint64 snaps = 20 [packed = true];
};

enum CHUNK_OP_STATUS {
Expand Down
3 changes: 3 additions & 0 deletions proto/nameserver2.proto
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,9 @@ message FileInfo {
optional uint64 stripeCount = 16;

optional uint64 epoch = 18;

// sequence numbers of snapshots
repeated uint64 snaps = 19;
}

// status code
Expand Down
4 changes: 2 additions & 2 deletions src/chunkserver/chunk_service.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -388,9 +388,9 @@ void ChunkServiceImpl::DeleteChunkSnapshotOrCorrectSn(
return;
}

if (false == request->has_correctedsn()) {
if (false == request->has_snapsn()) {
response->set_status(CHUNK_OP_STATUS::CHUNK_OP_STATUS_INVALID_REQUEST);
LOG(ERROR) << "delete chunk snapshot failed, no corrected sn:"
LOG(ERROR) << "delete chunk snapshot failed, no snapshot sn:"
<< request->logicpoolid() << "," << request->copysetid();
return;
}
Expand Down

0 comments on commit 04462d8

Please sign in to comment.