Skip to content

Bug: drop_index can corrupt collection reopen after optimized FTS or inverted indexes #568

Description

@egolearner

Problem

In v0.5.1, drop_index() can fail midway and leave a collection unable to reopen when dropping one indexed field while another indexed field of the same index family remains.

This is reproducible for both:

  • multiple FTS fields
  • multiple inverted scalar fields

Minimal trigger:

  1. Create a collection with a vector field and either two FTS fields or two inverted scalar fields.
  2. Insert documents.
  3. Run optimize() so data moves into persisted segments.
  4. Close and reopen the collection in read-write mode.
  5. Call drop_index() on one indexed field.
  6. Close and reopen again.

Observed result:

  • drop_index() fails with read-only RocksDB flush errors.
  • The next reopen fails while opening FTS or scalar index blocks.
  • Vector files may still exist on disk, but collection open is blocked before vector indexes are loaded.

Example FTS errors:

Cannot flush RocksDB[.../fts.1.rocksdb] in read-only mode
FtsIndexer: flush failed during snapshot
open_fts_indexers: open failed at [...]

Example inverted-index errors:

Cannot flush RocksDB[.../scalar.index.1.rocksdb] in read-only mode
Failed to flush InvertedIndexer[...] during creating a snapshot
Failed to open scalar indexer

Root Cause

Persisted segments are opened internally with SegmentOptions.read_only_ = true, even when the collection itself is opened read-write.

During drop_index(), segment DDL tasks reuse those read-only indexers and call create_snapshot().

Both snapshot paths currently flush before creating a RocksDB checkpoint:

  • FtsIndexer::create_snapshot()
  • InvertedIndexer::create_snapshot()

For persisted segment indexes, there are no pending writes and the underlying RocksDB is read-only, so this flush fails.

There is a second direct issue for inverted indexes: after snapshotting, drop_scalar_index() opens the snapshot copy with options_.read_only_. The copy is then mutated by remove_column_indexer() and seal(), so it must be opened writable.

Impact

This can leave a partially-created writing segment index on disk while the manifest/schema still expects the old index layout. On the next reopen, the collection may fail before vector indexes are loaded.

Proposed Fix

  • Expose RocksdbContext::read_only().
  • Skip snapshot-time flush() when the source RocksDB context is read-only.
  • Keep flushing for writable/mutable indexers.
  • Open scalar index snapshot copies as writable when they are about to be modified during drop_scalar_index().

Verification Plan

Regression coverage should include:

  • two FTS fields + vector field + optimize + reopen + drop one FTS index + reopen
  • two inverted fields + vector field + optimize + reopen + drop one inverted index + reopen

This sub-issue follows up on #565 and tracks the direct DDL corruption path reported after attempting drop_index() + create_index() as a repair workflow.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions