Problem
In v0.5.1, drop_index() can fail midway and leave a collection unable to reopen when dropping one indexed field while another indexed field of the same index family remains.
This is reproducible for both:
- multiple FTS fields
- multiple inverted scalar fields
Minimal trigger:
- Create a collection with a vector field and either two FTS fields or two inverted scalar fields.
- Insert documents.
- Run
optimize() so data moves into persisted segments.
- Close and reopen the collection in read-write mode.
- Call
drop_index() on one indexed field.
- Close and reopen again.
Observed result:
drop_index() fails with read-only RocksDB flush errors.
- The next reopen fails while opening FTS or scalar index blocks.
- Vector files may still exist on disk, but collection open is blocked before vector indexes are loaded.
Example FTS errors:
Cannot flush RocksDB[.../fts.1.rocksdb] in read-only mode
FtsIndexer: flush failed during snapshot
open_fts_indexers: open failed at [...]
Example inverted-index errors:
Cannot flush RocksDB[.../scalar.index.1.rocksdb] in read-only mode
Failed to flush InvertedIndexer[...] during creating a snapshot
Failed to open scalar indexer
Root Cause
Persisted segments are opened internally with SegmentOptions.read_only_ = true, even when the collection itself is opened read-write.
During drop_index(), segment DDL tasks reuse those read-only indexers and call create_snapshot().
Both snapshot paths currently flush before creating a RocksDB checkpoint:
FtsIndexer::create_snapshot()
InvertedIndexer::create_snapshot()
For persisted segment indexes, there are no pending writes and the underlying RocksDB is read-only, so this flush fails.
There is a second direct issue for inverted indexes: after snapshotting, drop_scalar_index() opens the snapshot copy with options_.read_only_. The copy is then mutated by remove_column_indexer() and seal(), so it must be opened writable.
Impact
This can leave a partially-created writing segment index on disk while the manifest/schema still expects the old index layout. On the next reopen, the collection may fail before vector indexes are loaded.
Proposed Fix
- Expose
RocksdbContext::read_only().
- Skip snapshot-time
flush() when the source RocksDB context is read-only.
- Keep flushing for writable/mutable indexers.
- Open scalar index snapshot copies as writable when they are about to be modified during
drop_scalar_index().
Verification Plan
Regression coverage should include:
- two FTS fields + vector field + optimize + reopen + drop one FTS index + reopen
- two inverted fields + vector field + optimize + reopen + drop one inverted index + reopen
This sub-issue follows up on #565 and tracks the direct DDL corruption path reported after attempting drop_index() + create_index() as a repair workflow.
Problem
In v0.5.1,
drop_index()can fail midway and leave a collection unable to reopen when dropping one indexed field while another indexed field of the same index family remains.This is reproducible for both:
Minimal trigger:
optimize()so data moves into persisted segments.drop_index()on one indexed field.Observed result:
drop_index()fails with read-only RocksDB flush errors.Example FTS errors:
Example inverted-index errors:
Root Cause
Persisted segments are opened internally with
SegmentOptions.read_only_ = true, even when the collection itself is opened read-write.During
drop_index(), segment DDL tasks reuse those read-only indexers and callcreate_snapshot().Both snapshot paths currently flush before creating a RocksDB checkpoint:
FtsIndexer::create_snapshot()InvertedIndexer::create_snapshot()For persisted segment indexes, there are no pending writes and the underlying RocksDB is read-only, so this flush fails.
There is a second direct issue for inverted indexes: after snapshotting,
drop_scalar_index()opens the snapshot copy withoptions_.read_only_. The copy is then mutated byremove_column_indexer()andseal(), so it must be opened writable.Impact
This can leave a partially-created writing segment index on disk while the manifest/schema still expects the old index layout. On the next reopen, the collection may fail before vector indexes are loaded.
Proposed Fix
RocksdbContext::read_only().flush()when the source RocksDB context is read-only.drop_scalar_index().Verification Plan
Regression coverage should include:
This sub-issue follows up on #565 and tracks the direct DDL corruption path reported after attempting
drop_index()+create_index()as a repair workflow.