fix: vector list wildcards and pagination

[ferhatelmas]

What kind of change does this PR introduce?

Bug fix

What is the current behavior?

Vector index listing can paginate second page onward.
Like operator wildcards aren't escaped.

What is the new behavior?

Fix pagination continuation and wildcards in prefix filters. Increase test coverage.

Additional context

None

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 5a067d12-9cb5-4705-a44d-045ce63eb34f

📥 Commits

Reviewing files that changed from the base of the PR and between d8dba53 and d588158.

📒 Files selected for processing (3)

• src/storage/protocols/vector/knex.ts
• src/storage/protocols/vector/vector-store.ts
• src/test/vectors.test.ts

---

📝 Walkthrough

Summary by CodeRabbit

• New Features

  • Pagination support added for vector index listings with nextToken-based navigation for large result sets.

• Bug Fixes

  • Enhanced input sanitization for search queries with improved special character handling.

Walkthrough

The changes enhance pagination and input handling in the vector storage layer. The code adds input sanitization using escapeLike for LIKE-pattern queries in bucket and index prefix filtering. The pagination mechanism is adjusted to use the name column instead of id for token-based pagination in the indexes listing. The VectorStoreManager.listIndexes method is updated to return a nextToken field alongside indexes. Comprehensive tests are added to validate pagination behavior with nextToken and prefix filtering with special character handling.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}