SDK Restructuring: Multi-Module Architecture

[renuka-fernando]

Problem Statement

The current SDK structure bundles all functionality into a single monolithic module, creating several limitations:

Current Structure

sdk/
├── go.mod                           # Single module for everything (heavy deps: milvus, redis)
├── go.sum
├── gateway/
│   ├── policy/v1alpha/              # Core policy interfaces
│   │   ├── interface.go             # Policy, PolicyFactory, ProcessingMode
│   │   ├── context.go               # RequestContext, ResponseContext
│   │   ├── action.go                # RequestAction, ResponseAction
│   │   ├── headers.go               # Headers utilities
│   │   ├── types.go                 # Common types
│   │   ├── definition.go            # Policy definition structures
│   │   └── api_key.go               # API key helpers
│   └── policyengine/v1/             # Policy engine config
│       ├── config.go                # PolicyChain, Configuration
│       └── api_key_xds.go           # API key xDS structures
└── utils/
    ├── jsonpath.go                  # JSON path utilities
    ├── embeddingproviders/          # AI: OpenAI, Azure OpenAI, Mistral
    │   ├── provider.go
    │   ├── openai_embedding_provider.go
    │   ├── azure_openai_embedding_provider.go
    │   └── mistral_embedding_provider.go
    └── vectordbproviders/           # AI: Redis, Milvus vector databases
        ├── provider.go
        ├── redis_vectordb_provider.go
        ├── milvus_vectordb_provider.go
        └── cacheable_response.go

Limitations

1. Dependency Bloat (Current Problem)

   • Policies requiring only core interfaces (e.g., rate limiting, header modification) must pull in heavy dependencies:
     • milvus-io/milvus/client/v2 - Vector database client
     • redis/go-redis/v9 - Redis client
     • 120+ indirect dependencies (etcd, grpc, prometheus, etc.)
   • A simple policy that only needs Policy interface gets ~90KB go.sum

2. Tight Coupling

   • All utilities version together, even if unrelated
   • Can't independently evolve AI features without affecting core interfaces
   • Breaking changes in AI utilities affect all policies

3. Unclear Boundaries

   • No clear separation between "core contracts" and "optional AI helpers"
   • gateway/policy/v1alpha/ (core) and utils/embeddingproviders/ (AI) are siblings under same module
   • Encourages adding more to the monolith

4. Versioning Constraints

   • Core interfaces (stable) and AI utilities (rapidly evolving) forced to same version
   • Can't have sdk/core v1.0.0 (stable) with sdk/ai v0.2.0 (experimental)

Proposed Solution

Restructure SDK into multiple independent Go modules under a nested directory structure:

sdk/                                          # Parent directory (not a module)
├── core/                                     # Core SDK module
│   ├── go.mod                               # github.com/wso2/api-platform/sdk/core
│   ├── policy/                              # Policy interfaces (from gateway/policy/v1alpha)
│   │   ├── interface.go                     # Policy, PolicyFactory, ProcessingMode
│   │   ├── context.go                       # RequestContext, ResponseContext, SharedContext
│   │   ├── action.go                        # RequestAction, ResponseAction
│   │   ├── headers.go                       # Headers manipulation
│   │   ├── types.go                         # Common types
│   │   ├── definition.go                    # Policy definition structures
│   │   └── api_key.go                       # API key helpers
│   ├── policyengine/                        # Policy engine config (from gateway/policyengine/v1)
│   │   ├── config.go                        # PolicyChain, PolicyInstance, Configuration
│   │   └── api_key_xds.go                   # API key xDS structures
│   └── utils/                               # Lightweight universal helpers
│       └── jsonpath.go                      # JSON path utilities
│
└── ai/                                       # AI SDK module (optional)
    ├── go.mod                               # github.com/wso2/api-platform/sdk/ai
    ├── embeddings/                          # Embedding providers (from utils/embeddingproviders)
    │   ├── provider.go                      # EmbeddingProvider interface
    │   ├── openai.go                        # OpenAI implementation
    │   ├── azure_openai.go                  # Azure OpenAI implementation
    │   └── mistral.go                       # Mistral implementation
    └── vectordb/                            # Vector DB providers (from utils/vectordbproviders)
        ├── provider.go                      # VectorDBProvider interface
        ├── redis.go                         # Redis implementation
        ├── milvus.go                        # Milvus implementation
        └── cacheable_response.go            # Response caching structures

Extensibility: Additional independent modules can be added in the future as common patterns emerge (e.g. caching, validation). Each new module would follow the same pattern - independent go.mod, clear boundaries, and opt-in usage.

Benefits

1. Dependency Isolation

• Policies import only what they need
• JWT auth policy: only sdk/core (~20KB)
• Semantic cache policy: sdk/core + sdk/ai (with AI dependencies)
• No unnecessary dependencies in policy binaries

2. Independent Versioning

sdk/core v1.0.0    (stable, rarely changes)
sdk/ai v0.1.0      (evolving, frequent updates)
# Future modules version independently as they're added

3. Clear Architecture

• Core: Required by all policies (interfaces, execution engine)
• Extensions: Optional modules for specialized use cases
• Explicit import paths show what a policy depends on

4. Flexibility

• Can add new SDK modules without affecting existing policies
• Policies can mix-and-match extensions as needed
• Third-party SDKs possible (community contributions)

5. Better Maintenance

• Changes to AI utilities don't require core SDK versioning
• Each module has its own release cadence
• Easier to reason about compatibility

Module Characteristics

Module	Dependencies	Change Frequency	Required By
sdk/core	Minimal (stdlib only: goccy/go-json)	Low	100% of policies
sdk/ai	Heavy (milvus-client, redis, net/http)	High	~5% of policies (semantic cache, AI gateway)

Future Extensions: Additional modules can be added as patterns emerge from real policy development (e.g., sdk/observability, sdk/validation, sdk/caching).

Import Path Examples

Before (Monolithic)

// Every policy imports the same module - pulls ALL dependencies
import (
    "github.com/wso2/api-platform/sdk/gateway/policy/v1alpha"
    "github.com/wso2/api-platform/sdk/utils/embeddingproviders"  // Forces milvus, redis deps
)

After (Multi-Module)

// Basic policy - core only (~minimal dependencies)
import (
    "github.com/wso2/api-platform/sdk/core/policy"
    "github.com/wso2/api-platform/sdk/core/utils"
)

// AI-powered policy - core + ai extension (only when needed)
import (
    "github.com/wso2/api-platform/sdk/core/policy"
    "github.com/wso2/api-platform/sdk/ai/embeddings"
    "github.com/wso2/api-platform/sdk/ai/vectordb"
)

Policy go.mod Examples

Basic Policy (e.g., Rate Limiting, Header Modification)

module github.com/wso2/policies/ratelimit/v1.0.0

go 1.25

require (
    github.com/wso2/api-platform/sdk/core v1.0.0
)

replace github.com/wso2/api-platform/sdk/core => ../../../../../sdk/core

Dependencies: Only stdlib + minimal JSON library (~small binary)

AI-Powered Policy (e.g., Semantic Cache)

module github.com/wso2/policies/semantic-cache/v1.0.0

go 1.25

require (
    github.com/wso2/api-platform/sdk/core v1.0.0
    github.com/wso2/api-platform/sdk/ai v0.1.0
)

replace (
    github.com/wso2/api-platform/sdk/core => ../../../../../sdk/core
    github.com/wso2/api-platform/sdk/ai => ../../../../../sdk/ai
)

Dependencies: Core + milvus-client, redis, net/http (larger binary, only for AI policies)

Breaking Changes

Impact

• All existing policies must update imports:
  • sdk/gateway/policy/v1alpha → sdk/core/policy
  • sdk/gateway/policyengine/v1 → sdk/core/policyengine
  • sdk/utils/embeddingproviders → sdk/ai/embeddings
  • sdk/utils/vectordbproviders → sdk/ai/vectordb
  • sdk/utils (jsonpath) → sdk/core/utils
• Policy go.mod files need updated module paths
• Replace directives need updated paths

Migration Effort

• ~5-10 lines per policy go.mod file
• Can be automated with find/replace or script
• No code logic changes required (just import path updates)

Migration Path

Step 1: Restructure SDK into Core Module

# Create core module structure
mkdir -p sdk/core/policy
mkdir -p sdk/core/policyengine
mkdir -p sdk/core/utils

# Move policy interfaces to core
mv sdk/gateway/policy/v1alpha/* sdk/core/policy/
mv sdk/gateway/policyengine/v1/* sdk/core/policyengine/
mv sdk/utils/jsonpath.go sdk/core/utils/

# Create new go.mod for core (minimal dependencies)
cat > sdk/core/go.mod << 'EOF'
module github.com/wso2/api-platform/sdk/core

go 1.25.1

require (
    github.com/goccy/go-json v0.10.5
    github.com/google/uuid v1.6.0
)
EOF

cd sdk/core && go mod tidy

Step 1.5: Create AI Module (for embedding/vectordb providers)

# Create AI module structure
mkdir -p sdk/ai/embeddings
mkdir -p sdk/ai/vectordb

# Move AI utilities to ai module
mv sdk/utils/embeddingproviders/* sdk/ai/embeddings/
mv sdk/utils/vectordbproviders/* sdk/ai/vectordb/

# Create new go.mod for ai (with heavy dependencies)
cat > sdk/ai/go.mod << 'EOF'
module github.com/wso2/api-platform/sdk/ai

go 1.25.1

require (
    github.com/wso2/api-platform/sdk/core v1.0.0
    github.com/milvus-io/milvus/client/v2 v2.6.1
    github.com/milvus-io/milvus/pkg/v2 v2.6.7
    github.com/redis/go-redis/v9 v9.8.0
)

replace github.com/wso2/api-platform/sdk/core => ../core
EOF

cd sdk/ai && go mod tidy

# Clean up old directories
rm -rf sdk/gateway sdk/utils
rm sdk/go.mod sdk/go.sum

Step 2: Update Policy Engine Runtime

cd gateway/policy-engine

# Update go.mod replace directives
# OLD: replace github.com/wso2/api-platform/sdk => ../../sdk
# NEW: replace github.com/wso2/api-platform/sdk/core => ../../sdk/core
#      replace github.com/wso2/api-platform/sdk/ai => ../../sdk/ai (if using AI)

# Update imports in code
find . -name "*.go" -type f -exec sed -i '' \
  's|github.com/wso2/api-platform/sdk/gateway/policy/v1alpha|github.com/wso2/api-platform/sdk/core/policy|g' {} \;
find . -name "*.go" -type f -exec sed -i '' \
  's|github.com/wso2/api-platform/sdk/gateway/policyengine/v1|github.com/wso2/api-platform/sdk/core/policyengine|g' {} \;

go mod tidy

Step 3: Update All Sample Policies

cd gateway/sample-policies

# Script to update all policies
for policy_dir in */v*/; do
    echo "Updating ${policy_dir}"

    # Update imports in .go files
    find "${policy_dir}" -name "*.go" -type f -exec sed -i '' \
      's|github.com/wso2/api-platform/sdk/gateway/policy/v1alpha|github.com/wso2/api-platform/sdk/core/policy|g' {} \;
    find "${policy_dir}" -name "*.go" -type f -exec sed -i '' \
      's|github.com/wso2/api-platform/sdk/gateway/policyengine/v1|github.com/wso2/api-platform/sdk/core/policyengine|g' {} \;

    # Update go.mod replace directive
    sed -i '' 's|github.com/wso2/api-platform/sdk =>|github.com/wso2/api-platform/sdk/core =>|g' \
      "${policy_dir}go.mod"
    sed -i '' 's|=> \.\./\.\./\.\./sdk$|=> ../../../sdk/core|g' \
      "${policy_dir}go.mod"

    # Tidy module
    cd "${policy_dir}" && go mod tidy && cd ../..
done

Step 4: Update Gateway Builder

cd gateway/gateway-builder

# Update imports in code
find . -name "*.go" -type f -exec sed -i '' \
  's|github.com/wso2/api-platform/sdk/gateway/policy/v1alpha|github.com/wso2/api-platform/sdk/core/policy|g' {} \;
find . -name "*.go" -type f -exec sed -i '' \
  's|github.com/wso2/api-platform/sdk/gateway/policyengine/v1|github.com/wso2/api-platform/sdk/core/policyengine|g' {} \;

# Update go.mod replace directive
sed -i '' 's|github.com/wso2/api-platform/sdk =>|github.com/wso2/api-platform/sdk/core =>|g' go.mod

go mod tidy