Skip to content

Support GetLiveFiles on secondary DB instances#14475

Closed
archang19 wants to merge 1 commit into
facebook:mainfrom
archang19:export-D97563143
Closed

Support GetLiveFiles on secondary DB instances#14475
archang19 wants to merge 1 commit into
facebook:mainfrom
archang19:export-D97563143

Conversation

@archang19
Copy link
Copy Markdown
Contributor

Summary:
GetLiveFiles was previously blocked on secondary instances with
Status::NotSupported, even though the operation is safe to perform.

The only reason GetLiveFiles was originally blocked is that it defaults
to flushing the memtable (flush_memtable=true), which is a write
operation. However, the actual implementation in DBImpl::GetLiveFiles
(db_filesnapshot.cc) does two things:

  1. Optionally flush the memtable — which we skip by passing
    flush_memtable=false. The secondary already overrides
    FlushForGetLiveFiles() as a no-op, so even if true were passed
    it would not actually flush.

  2. Read live file state from versions_ under the mutex — this is
    purely read-only. It iterates the ColumnFamilySet to collect live
    table and blob file numbers, builds relative file paths for SST
    files, blob files, CURRENT, MANIFEST, and OPTIONS, and reads the
    manifest file size.

The secondary maintains its own VersionSet which it keeps up to date
via MANIFEST replay in TryCatchUpWithPrimary(). So all of this state
is valid and accurate — it reflects exactly which files the secondary
considers live at its current replay point.

This is the same approach used by DBImplReadOnly and CompactedDBImpl,
which both delegate to DBImpl::GetLiveFiles with flush_memtable=false.

Differential Revision: D97563143

Summary:
GetLiveFiles was previously blocked on secondary instances with
Status::NotSupported, even though the operation is safe to perform.

The only reason GetLiveFiles was originally blocked is that it defaults
to flushing the memtable (flush_memtable=true), which is a write
operation. However, the actual implementation in DBImpl::GetLiveFiles
(db_filesnapshot.cc) does two things:

1. Optionally flush the memtable — which we skip by passing
   flush_memtable=false. The secondary already overrides
   FlushForGetLiveFiles() as a no-op, so even if true were passed
   it would not actually flush.

2. Read live file state from versions_ under the mutex — this is
   purely read-only. It iterates the ColumnFamilySet to collect live
   table and blob file numbers, builds relative file paths for SST
   files, blob files, CURRENT, MANIFEST, and OPTIONS, and reads the
   manifest file size.

The secondary maintains its own VersionSet which it keeps up to date
via MANIFEST replay in TryCatchUpWithPrimary(). So all of this state
is valid and accurate — it reflects exactly which files the secondary
considers live at its current replay point.

This is the same approach used by DBImplReadOnly and CompactedDBImpl,
which both delegate to DBImpl::GetLiveFiles with flush_memtable=false.

Differential Revision: D97563143
@meta-cla meta-cla Bot added the CLA Signed label Mar 20, 2026
@meta-codesync
Copy link
Copy Markdown

meta-codesync Bot commented Mar 20, 2026

@archang19 has exported this pull request. If you are a Meta employee, you can view the originating Diff in D97563143.

@github-actions
Copy link
Copy Markdown

✅ clang-tidy: No findings on changed lines

Completed in 116.0s.

@meta-codesync
Copy link
Copy Markdown

meta-codesync Bot commented Mar 23, 2026

This pull request has been merged in 89e384c.

doxtop pushed a commit to flyingw/rocksdb that referenced this pull request Apr 7, 2026
Summary:
Pull Request resolved: facebook#14475

GetLiveFiles was previously blocked on secondary instances with
Status::NotSupported, even though the operation is safe to perform.

The only reason GetLiveFiles was originally blocked is that it defaults
to flushing the memtable (flush_memtable=true), which is a write
operation. However, the actual implementation in DBImpl::GetLiveFiles
(db_filesnapshot.cc) does two things:

1. Optionally flush the memtable — which we skip by passing
   flush_memtable=false. The secondary already overrides
   FlushForGetLiveFiles() as a no-op, so even if true were passed
   it would not actually flush.

2. Read live file state from versions_ under the mutex — this is
   purely read-only. It iterates the ColumnFamilySet to collect live
   table and blob file numbers, builds relative file paths for SST
   files, blob files, CURRENT, MANIFEST, and OPTIONS, and reads the
   manifest file size.

The secondary maintains its own VersionSet which it keeps up to date
via MANIFEST replay in TryCatchUpWithPrimary(). So all of this state
is valid and accurate — it reflects exactly which files the secondary
considers live at its current replay point.

This is the same approach used by DBImplReadOnly and CompactedDBImpl,
which both delegate to DBImpl::GetLiveFiles with flush_memtable=false.

Reviewed By: xingbowang

Differential Revision: D97563143

fbshipit-source-id: 8d5b52e26a478ef190eba598819de6527817bcfc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants