Skip to content

Conversation

@rootfs
Copy link
Collaborator

@rootfs rootfs commented Oct 20, 2025

What type of PR is this?

What this PR does / why we need it:

The inmemory cache search is not optimized, adding HNSW index will accelerate the search

This is the benchmark result:
image
image

cc @aeft

Which issue(s) this PR fixes:

Fixes #

Release Notes: Yes/No

Signed-off-by: Huamin Chen <hchen@redhat.com>
@netlify
Copy link

netlify bot commented Oct 20, 2025

Deploy Preview for vllm-semantic-router ready!

Name Link
🔨 Latest commit 6ffbef9
🔍 Latest deploy log https://app.netlify.com/projects/vllm-semantic-router/deploys/68f6698db53f4f000895ede7
😎 Deploy Preview https://deploy-preview-491--vllm-semantic-router.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions
Copy link

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 src

Owners: @rootfs, @Xunzhuo, @wangchen615
Files changed:

  • src/semantic-router/pkg/cache/comprehensive_benchmark_test.go
  • src/semantic-router/pkg/cache/cache_factory.go
  • src/semantic-router/pkg/cache/cache_interface.go
  • src/semantic-router/pkg/cache/inmemory_cache.go
  • src/semantic-router/pkg/cache/inmemory_cache_integration_test.go

📁 config

Owners: @rootfs
Files changed:

  • config/config.development.yaml
  • config/config.yaml

📁 website

Owners: @Xunzhuo, @rootfs, @yuluo-yx
Files changed:

  • website/docs/tutorials/semantic-cache/in-memory-cache.md

vLLM

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

@rootfs
Copy link
Collaborator Author

rootfs commented Oct 21, 2025

closing and replacing it with #504

@rootfs rootfs closed this Oct 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants