KVec

A modular semantic search stack.

It can be fully cloud-based with Cloudflare KV or Upstash Redis, OpenAI or Cohere Text Embedding, and Pinecone or Qdrant Cloud Vector Database.

It can also be self-hosted with CouchDB, Qdrant on Docker.

Features

• Support for different item types: KVec supports text, web page, and image items out of the box, and you can easily extend it to support other types.
• Modular architecture: KVec allows you to easily change components like the encoder, object store, and vector store to fit your specific use case.
• Authentication: KVec supports authentication using JWT tokens.
• Dashboard GUI: KVec comes with a simple dashboard that allows you to manage your items and issue authentication tokens.
• RESTful API: KVec provides a simple API for creating, reading, deleting, and searching items.

Setup

Cloudflare Pages

To use Cloudflare Pages to host your KVec.

You should set up the following environment variables:

• APP_SECRET: Secret for signing JWT tokens
• PINECONE_API_KEY: To enable PineconeVecStore (see Structure)
• PINECONE_ENDPOINT: The endpoint of your Pinecone index, nessary for PineconeVecStore
• OPENAI_API_KEY: To enable OpenAIEncoder (see Structure)
• HF_API_TOKEN: To enable BaseTextAdapter's image feature

And bind a KV namespace:

• KV: KV namespace for storing items, nessary for CloudflareKVObjStore (see Structure)

You can simply:

1. Fork this repo.
2. Create a new Cloudflare Pages project, connect to your forked repo, and setup the environment variables.
3. Create a KV namespace and bind it to the project.

Docker

To use KVec with Docker, you'll need to setup environment variables in the .env file.

You may want something like this:

PORT="8080"
ORIGIN="http://localhost:8080"
APP_SECRET="my-kvec-secret"

# Adapter
HF_API_TOKEN="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

# Encoder
COHERE_API_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

# VecStore
QDRANT_SERVER="http://qdrant:6333"
QDRANT_COLLECTION="kvec"

# ObjStore
COUCHDB_URL="http://couchdb:5984"
COUCHDB_DB="kvec"
COUCHDB_USER="admin"
COUCHDB_PASSWORD="password"

Then, you can run the following command to start:

docker compose up -d

To setup the Qdrant collection at the first time, you can run the following command:

curl -X 'PUT' 'http://localhost:6333/collections/kvec' -H 'accept: application/json' -H 'content-type: application/json' -d '
{
  "vectors": {
        "size": 4096,
        "distance": "Cosine"
    }
}'
# The vector size for Cohere's embedding is 4096, if you are using OpenAI's embedding, you should change it to 1536.

To setup the CouchDB database at the first time, you can run the following command:

curl -X PUT http://admin:password@localhost:5984/kvec 

Now, you should be able to access the dashboard from http://localhost:8080/.

GUI

You can use the dashboard GUI to manage items and issue tokens.

You can access the dashboard from https://kvec.yourdomain.com/.

API

Authentication

You will need to get a JWT token to make requests to the item API.

One way to do this is to use the dashboard GUI.

However, you can also issue tokens using the API itself:

curl -X POST \
    -H "Content-Type: application/json" \
    -d '{ "secret": "your-app-secret", "exp": 3600, "perm": { "read": true, "write": false } }' \
    https://kvec.yourdomain.com/api/auth
# Creates a token that expires in 1 hour and only allows read access

{
    "token": "ISSUED_JWT"
}

The token can be passed in the Authorization header of the request or the kvec_token cookie.

Item API

The item API allows you to create, read, delete, and search items.

Create an item

write permission is required.

curl -X POST \
    -H "Content-Type: application/json" \
    -H "Authorization: YOUR_TOKEN" \
    -d '{ "data": { text: "the content of the item" } }' \
    https://kvec.yourdomain.com/api/item

{
    "id": "ITEM_ID"
}

This will create a new item with the text the content of the item.

Examples for other types of items:

Web page. The BaseTextAdapter will automatically fetch and use the page title and description as the page feature.

{
    "data": {
        "page": "https://github.com/JacobLinCool/kvec"
    }
}

Image. HF_API_TOKEN environment variable is required. It uses Hugging Face's inference API to transform the image into text as the image feature. The default model is nlpconnect/vit-gpt2-image-captioning, but you can specify a different model by setting the HF_IMGCAP_MODEL environment variable. http://, https://, and data: are supported.

{
    "data": {
        "img": "https://kvec.pages.dev/icon.png"
    }
}

Read an item

read permission is required.

curl -X GET \
    -H "Content-Type: application/json" \
    -H "Authorization: YOUR_TOKEN" \
    https://kvec.yourdomain.com/api/item/<ITEM_ID>

{
    "item": {
        "id": "ITEM_ID",
        "data": {
            "text": "the content of the item"
        },
        "meta": {
            "type": "text"
        }
    }
}

Delete an item

write permission is required.

curl -X DELETE \
    -H "Content-Type: application/json" \
    -H "Authorization: YOUR_TOKEN" \
    https://kvec.yourdomain.com/api/item/<ITEM_ID>

{
    "deleted": true,
    "item": {
        "id": "ITEM_ID",
        "data": {
            "text": "the content of the item"
        },
        "meta": {
            "type": "text"
        }
    }
}

Search items

It performs a semantic search to find items that are similar to the query.

read permission is required.

curl -X GET \
    -H "Content-Type: application/json" \
    -H "Authorization: YOUR_TOKEN" \
    https://kvec.yourdomain.com/api/item?q=<QUERY>

{
    "items": [
        {
            "id": "ITEM_ID_1",
            "data": {
                "text": "the content of item 1"
            },
            "meta": {
                "type": "text"
            }
        },
        {
            "id": "ITEM_ID_2",
            "data": {
                "text": "the content of item 2",
                "page": "https://example.com/item2"
            },
            "meta": {
                "type": "page"
            }
        },
        {
            "id": "ITEM_ID_3",
            "data": {
                "text": "the content of item 2",
                "img": "https://example.com/item3.png"
            },
            "meta": {
                "type": "img"
            }
        }
    ]
}

Structure

The KVec structure is mainly based on 6 components:

• The API and GUI layer, which allows other services to interact with KVec easily and manage the authorizations.
• The Adapter layer, which is responsible for adapting the data from the API layer to the encoder layer.
• The Encoder layer, which is responsible for encoding the data into vectors (embeddings).
• The ObjStore layer, which is responsible for storing the items itself.
• The VecStore layer, which is responsible for storing the vectors, and performing the search.
• The Cache layer, which is responsible for caching the search results.

The Adapter, Encoder, ObjStore, VecStore, and Cache layers are all pluggable, so you can easily customize them to fit your needs.

Currently, the following implementations are available:

• Adapter
  • BaseTextAdapter: Support text, web page, and image (env HF_API_TOKEN required) items.
• Encoder
  • OpenAIEncoder: Use OpenAI's text-embedding-ada-002 to create embeddings
  • CohereEncoder: Use Cohere as the embedding service
  • JustEncoder: Only for local development
• ObjStore
  • CloudflareKVObjStore: Use Cloudflare KV as the object store backend
  • UpstashRedisObjStore: Use Upstash Redis as the object store backend
  • CouchDBObjStore: Use CouchDBObjStore as the object store backend
  • MemoryObjStore: Only for local development
• VecStore
  • PineconeVecStore: Use Pinecone as the vector store backend
  • QdrantVecStore: Use Qdrant as the vector store backend
  • MemoryVecStore: Only for local development
• Cache
  • CloudflareCache: Use Cloudflare's Cache API
  • MemoryCache: Only for local development, it just "don't cache anything"

The auto module will automatically load the correct implementation based on the environment variables. See src/lib/server/auto/index.ts You can also take a look at the README of each module to see how to configure them.