Skip to content

Model Context Protocol

Jump to bottom
Piergiorgio Lucidi edited this page Jul 20, 2026 · 4 revisions

Model Context Protocol (MCP) Integration & Secure Vector Server

The OpenCrawling® Secure MCP Server (McpVectorServer) is an enterprise-grade component built on Spring AI 2.0 and the Model Context Protocol (MCP) standard (pioneered by Anthropic).

It exposes identity-aware vector search, full document retrieval, and knowledge discovery tools directly to Large Language Models (LLMs) and AI agents (such as Claude Desktop, Cursor, Antigravity, or custom agent frameworks) over standard Server-Sent Events (SSE) and HTTP WebMVC transports.

🛡️ Zero-Trust Architecture & Security Model

In typical RAG (Retrieval-Augmented Generation) architectures, vector databases return top-K semantic matches without evaluating the querying user's security permissions. This leads to critical data leaks when an AI agent accesses confidential files (e.g., payroll spreadsheets or strategic blueprints) on behalf of an unauthorized user.

OpenCrawling solves this with Zero-Trust Context Retrieval, combining SQL-level pre-filtering with Java-level Open Ingestion Standard (OIS) security rule evaluation before any context is delivered to the LLM client.

graph TD
    subgraph LLM / MCP Client
        Agent[AI Agent / LLM Client]
    end

    subgraph OpenCrawling Runtime
        MCP[McpVectorServer]
        Auth[Authority Service]
        VecStore[PgVectorStore / MilvusStore]
    end

    subgraph Database Boundary
        DB[(PostgreSQL + pgvector)]
    end

    Agent -->|1. secureVectorSearch query, userPrincipal, userRoles| MCP
    MCP -->|2. Validate principal & map active groups| Auth
    MCP -->|3. Construct dynamic FilterExpression| VecStore
    VecStore -->|4. SQL Execution WHERE acl IN user_tokens| DB
    DB -->|5. Candidate Vector Chunks| VecStore
    VecStore -->|6. Post-Filter OIS Security Rules| MCP
    MCP -->|7. Return Authorized Context JSON| Agent
Loading

Two-Phase Security Filter Execution

  1. Phase 1 — SQL-Level Pre-Filtering (Database Layer) When secureVectorSearch or getDocumentContent is invoked, McpVectorServer uses Spring AI's FilterExpressionBuilder to restrict candidate rows at the database query level:

    SELECT * FROM vector_store 
WHERE embedding <=> :query_vector < :threshold
  AND (acl = 'public' OR acl = :user_principal OR acl = ANY(:user_roles));

  2. Phase 2 — Java-Level OIS Security Evaluation (isAccessible) For documents containing rich, multi-identity metadata under the Open Ingestion Standard (OIS) security specification, the server evaluates candidate chunks against granular rules:

    • Explicit DENY Overrides: Any rule with access: "deny" matching the user principal or any of their assigned roles immediately invalidates document access, overriding general READ permissions.
    • Enterprise Identity Resolution: Automatically resolves cmis-user/cmis-group identities and bpmn-assignee/bpmn-candidate-group permissions.
    • Legacy ACL Fallback: If structured security metadata is absent, access defaults to evaluating flat acl string fields (public, exact principal match, or active role match).

🛠️ Exposed MCP Tools Reference

The McpVectorServer registers three @McpTool components:

1. secureVectorSearch

Performs similarity searches on vectorized enterprise documents, returning only chunks authorized for the caller's identity.

  • Signature & Parameters:

    Parameter Type Required Description
    query String Yes Natural language search query or keywords.
    userPrincipal String Yes User email or unique principal identity (e.g. john.doe@company.com).
    userRoles String No Comma-separated LDAP/OAuth roles or groups (e.g. finance,engineering).
    maxResults Integer No Maximum number of results to return (default: 5).
    minScore Double No Minimum similarity threshold score from 0.0 to 1.0 (default: 0.0).
    dimensions Integer No Target vector store dimension to query (384, 768, or 1024).

  • Return DTO (DocumentSearchResult):

    [
  {
    "id": "chunk-10492",
    "uri": "file:///data/finance/q3_report.pdf",
    "content": "Q3 Revenue reached $14.2M representing a 18% YoY growth...",
    "metadata": {
      "uri": "file:///data/finance/q3_report.pdf",
      "acl": "finance",
      "mimeType": "application/pdf"
    },
    "acl": "finance",
    "score": 0.892
  }
]

2. getDocumentContent

Retrieves the complete content and metadata of a specific indexed file by URI, validating permissions prior to returning the document body.

  • Signature & Parameters:

    Parameter Type Required Description
    documentUri String Yes Exact URI of the document (e.g. file:///docs/architecture.pdf).
    userPrincipal String Yes Caller principal identity.
    userRoles String No Comma-separated group memberships.
    dimensions Integer No Vector store dimension selector (384, 768, 1024).

  • Error Handling: If the document does not exist or the caller lacks authorization, the tool throws a NoSuchElementException("Document not found or access denied.").

3. listAccessibleSources

Enumerates all indexed knowledge documents accessible to the specified caller identity.

  • Signature & Parameters:

    Parameter Type Required Description
    userPrincipal String Yes Caller principal identity.
    userRoles String No Comma-separated group memberships.
    dimensions Integer No Target vector dimension store.

  • Return Output Example:

    [
  {
    "id": "doc-884",
    "uri": "file:///shared/wiki/getting-started.md",
    "acl": "public",
    "security": null,
    "lastModified": "2026-07-15T14:30:00Z"
  }
]

📐 Multi-Dimension Vector Store Routing

OpenCrawling supports multi-tenant vector models with varying output dimensions. McpVectorServer accepts an optional dimensions parameter and routes queries to the corresponding VectorStore instance:

Dimension Primary Embedding Model Default Target Store
384 all-MiniLM-L6-v2 / Lightweight local models vectorStore384
768 nomic-embed-text / Standard enterprise models vectorStore768
1024 mxbai-embed-large / High-accuracy dense models vectorStore1024
null / Default Configured primary vector store vectorStore

⚙️ Server Configuration & Deployment

1. Spring Boot application.yml Settings

Enable the MCP server module in your oc-runtime application settings:

spring:
  ai:
    vectorstore:
      pgvector:
        url: jdbc:postgresql://localhost:5432/opencrawling
        username: opencrawling
        password: opencrawling_password
        dimensions: 1024
    mcp:
      server:
        name: "opencrawling-secure-vector-mcp"
        version: "1.0.0"
        transport: STREAMABLE_HTTP
        annotation-scanner:
          enabled: true

2. Standalone Docker Container Deployment

Build and run the decoupled MCP microservice container using Dockerfile.mcp-server:

# Build the dedicated MCP image
docker build -f docker/Dockerfile.mcp-server -t opencrawling-mcp-server .

# Run the container exposing port 8080
docker run -d \
  --name opencrawling-mcp \
  -p 8080:8080 \
  -e OPENCRAWLING_MCP_SERVER_ENABLED=true \
  -e SPRING_AI_MCP_SERVER_ANNOTATION_SCANNER_ENABLED=true \
  opencrawling-mcp-server

🔌 Connecting MCP Clients & Integrations

1. Claude Desktop

Add the server definition to claude_desktop_config.json:

{
  "mcpServers": {
    "opencrawling-secure-vector-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/client-cli",
        "sse",
        "http://localhost:8080/mcp/sse"
      ]
    }
  }
}

2. Cursor IDE

Create or edit .cursor/mcp.json in your workspace or user home directory (~/.cursor/mcp.json):

{
  "mcpServers": {
    "opencrawling-secure-vector-mcp": {
      "url": "http://localhost:8080/mcp/sse"
    }
  }
}

3. Windsurf IDE

Edit ~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "opencrawling-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/client-cli",
        "sse",
        "http://localhost:8080/mcp/sse"
      ]
    }
  }
}

4. Continue.dev (VS Code & JetBrains Plugin)

Add the SSE server to ~/.continue/config.json:

{
  "experimental": {
    "modelContextProtocolServers": [
      {
        "transport": {
          "type": "sse",
          "url": "http://localhost:8080/mcp/sse"
        }
      }
    ]
  }
}

5. Antigravity & agy CLI

Register the server via CLI command or ~/.gemini/antigravity-cli/mcp.json:

# Register using agy CLI command
agy mcp add opencrawling-mcp http://localhost:8080/mcp/sse --transport sse

Or JSON configuration file:

{
  "mcpServers": {
    "opencrawling-mcp": {
      "transport": "sse",
      "url": "http://localhost:8080/mcp/sse"
    }
  }
}

6. Python MCP SDK (mcp library)

Programmatically call OpenCrawling tools inside Python agent applications (LangChain, LlamaIndex, or custom scripts):

import asyncio
from mcp import ClientSession
from mcp.client.sse import sse_client

async def main():
    # Connect via SSE transport
    async with sse_client("http://localhost:8080/mcp/sse") as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()
            
            # Perform a security-filtered vector search
            search_response = await session.call_tool(
                "secureVectorSearch",
                arguments={
                    "query": "Kubernetes cluster architecture and deployment steps",
                    "userPrincipal": "devops@company.com",
                    "userRoles": "engineering,infrastructure",
                    "maxResults": 5,
                    "minScore": 0.3
                }
            )
            print("Search Results:", search_response.content)

asyncio.run(main())

7. Java / Spring AI MCP Client

In corporate Java services, consume the MCP server using Spring AI Starter Client (spring-ai-starter-mcp-client-webmvc):

pom.xml dependency:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-mcp-client-webmvc</artifactId>
    <version>2.0.0</version>
</dependency>

application.yml:

spring:
  ai:
    mcp:
      client:
        enabled: true
        name: opencrawling-client
        sse:
          opencrawling-server:
            base-url: http://localhost:8080
            sse-endpoint: /mcp/sse

Java Client Service implementation:

package com.example.service;

import org.springframework.ai.mcp.client.McpClient;
import org.springframework.stereotype.Service;
import java.util.Map;

@Service
public class EnterpriseKnowledgeService {

    private final McpClient mcpClient;

    public EnterpriseKnowledgeService(McpClient mcpClient) {
        this.mcpClient = mcpClient;
    }

    public String queryKnowledgeBase(String userEmail, String userGroups, String userQuery) {
        var response = mcpClient.callTool("secureVectorSearch", Map.of(
            "query", userQuery,
            "userPrincipal", userEmail,
            "userRoles", userGroups,
            "maxResults", 5
        ));
        return response.content().toString();
    }
}

8. Direct HTTP & cURL JSON-RPC Testing

Verify the MCP server endpoints manually using curl:

Step 1: Open SSE Stream Listener

curl -N -H "Accept: text/event-stream" http://localhost:8080/mcp/sse

Response includes session endpoint, e.g. event: endpoint -> /mcp/message?sessionId=abc123xyz

Step 2: Send Tool Execution JSON-RPC Request

curl -X POST "http://localhost:8080/mcp/message?sessionId=abc123xyz" \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/call",
    "params": {
      "name": "secureVectorSearch",
      "arguments": {
        "query": "annual financial results",
        "userPrincipal": "alice@company.com",
        "userRoles": "finance"
      }
    }
  }'

🧪 Verification & Testing

Unit tests for the server implementation are located in McpVectorServerTest.java:

# Execute unit tests for MCP module
mvn test -pl oc-runtime -Dtest=McpVectorServerTest

🗺️ Documentation

🧪 Testing

🤝 Community

Clone this wiki locally