LLMTesting

Integrating llms.txt with the Server-Side JS SDK

1. Purpose & Context

1.1 What is llms.txt?

The llms.txt specification is an informal proposal (published September 3, 2024 by Jeremy Howard) that introduces a standard route /llms.txt for websites to provide LLM-friendly context. Think of it as a "robots.txt for AI" — it gives Large Language Models a ready-made, token-efficient summary of your website's content so they can answer questions without parsing full HTML.

Key characteristics:

Inference-focused, not training: Designed for real-time AI responses, not dataset building
Informal standard: Unlike robots.txt, this has no IETF/W3C backing and limited confirmed adoption
Plain text format: Must return Content-Type: text/plain; charset=utf-8
Fixed route: The URL /llms.txt must always resolve (no redirects)

Current reality (as of October 2025):

📊 Adoption: < 1% of top million sites, mostly developer docs platforms (Mintlify, nbdev, Apidog)
🤖 Crawler support: Unconfirmed/mixed — Mintlify claims OpenAI/Anthropic crawl it; independent audits show zero systematic requests
⚖️ Industry consensus: "Not at critical mass" — experimental rather than guaranteed traffic lever

Learn more: llmstxt.org

1.2 Why Test llms.txt Variations?

While adoption is early-stage, forward-thinking companies can experiment with different llms.txt content strategies to understand:

Which content structure drives more referral traffic from AI-powered search (Perplexity, ChatGPT web search, etc.)
Which tone/format (technical vs. marketing) performs better
Whether including product links vs. documentation links impacts conversions

The challenge: AI crawlers don't execute JavaScript. They fetch /llms.txt as a static text file. To run A/B tests, you must make variation decisions server-side before the content is served.

1.3 Why Server-Side SDK Integration Matters

Traditional client-side A/B testing won't work here:

❌ AI crawlers don't execute JavaScript
❌ Browser-based SDKs activate after page load
✅ Server-side decisions happen before response is sent

The Convert JS SDK (Node.js) can run server-side to:

Decide experiment variations on every request
Maintain consistent bucketing across routes (/llms.txt, /page, etc.)
Use cookies or custom storage for persistent visitor assignments
Track which variants drive downstream traffic via UTM parameters

2. Architectural Overview

2.1 Request Flow Diagram

graph TB
    subgraph "Visitor Browser / AI Crawler"
        A[Request /llms.txt]
        B[Request /pricing]
    end
    
    subgraph "Your Server (Node.js / Express)"
        C[Express Middleware:<br/>SDK Initialization]
        D[Create/Retrieve<br/>Visitor ID]
        E[Read Cookie:<br/>convert_sdk_vid]
        F[Context = SDK.createContext<br/>visitorId + attributes]
        G[Run ALL Experiments:<br/>runExperiences]
        H[Store Decisions<br/>in Cookie]
        I{Route?}
        J[Read variant file:<br/>llms-v1.txt or llms-v2.txt]
        K[Render page using<br/>experiment decisions]
    end
    
    subgraph "Persistent Storage"
        L[(Cookie: convert_sdk_vid<br/>bucketing data)]
    end
    
    A -->|1| C
    B -->|1| C
    C -->|2| D
    D -->|3| E
    E -->|4| F
    F -->|5| G
    G -->|6| H
    H -->|7| I
    H -.->|persist| L
    L -.->|read| E
    
    I -->|/llms.txt| J
    I -->|other routes| K
    
    J -->|8| M[Return text/plain<br/>with Cache-Control]
    K -->|8| N[Return HTML]
    
    style G fill:#e1f5ff
    style H fill:#ffe1f5
    style L fill:#fff4e1

2.2 Sequence: Simultaneous /page and /llms.txt Requests

sequenceDiagram
    participant Crawler as AI Crawler
    participant Browser as User Browser
    participant Server as Express Server
    participant SDK as Convert SDK
    participant Cookie as Cookie Store
    participant Files as File System

    Note over Crawler,Files: AI crawler discovers site, user browses independently
    
    par AI Crawler Request
        Crawler->>+Server: GET /llms.txt
        Server->>Server: Extract/Generate visitor_id from<br/>Cookie or User-Agent hash
        Server->>Cookie: Read convert_sdk_vid cookie
        Cookie-->>Server: {bucketing: {exp123: var2}}
        Server->>+SDK: createContext(visitor_id, {})
        SDK-->>-Server: Context
        Server->>+SDK: context.runExperiences()
        Note right of SDK: Returns Array<BucketedVariation>
        SDK-->>-Server: [{experienceKey: 'exp123', key: 'variation-2'}, ...]
        Server->>Cookie: Set convert_sdk_vid cookie (updated)
        Server->>Server: Find: variations.find(v => v.experienceKey === 'exp123')
        Server->>Files: Read llms/variation-2.txt
        Files-->>Server: Content
        Server-->>-Crawler: 200 text/plain<br/>Cache-Control: private, max-age=86400
    end
    
    par User Browser Request
        Browser->>+Server: GET /pricing
        Server->>Server: Extract visitor_id from<br/>convert_sdk_vid cookie
        Server->>Cookie: Read convert_sdk_vid cookie
        Cookie-->>Server: {bucketing: {exp123: var1}}
        Server->>+SDK: createContext(visitor_id, {device: desktop})
        SDK-->>-Server: Context
        Server->>+SDK: context.runExperiences()
        Note right of SDK: Returns Array<BucketedVariation>
        SDK-->>-Server: [{experienceKey: 'exp123', key: 'variation-1'}, ...]
        Server->>Cookie: Set convert_sdk_vid cookie (updated)
        Server->>Server: Render pricing page using var-1
        Server-->>-Browser: 200 text/html<br/>Set-Cookie: convert_sdk_vid=...
    end

    Note over Crawler,Files: Both requests use consistent visitor IDs → consistent variations

2.3 Key Principles

Single SDK Instance: Initialize once at app startup, reuse across all requests
Consistent Visitor ID: Same ID for a user across /llms.txt, /page, /checkout
Cookie-Based Persistence: Implement a custom dataStore with get/set methods for bucketing state
Decide Early: Call SDK in middleware before route handlers
No Redirects: Serve /llms.txt content directly with 200 OK

3. Prerequisites

3.1 SDK Requirements

Package: @convertcom/js-sdk version ≥ 2.0
Node.js: v14+ (v18+ recommended for modern crypto APIs)
Server Framework: Express, Fastify, Next.js, or similar

Installation:

npm install @convertcom/js-sdk cookie-parser
# or
yarn add @convertcom/js-sdk cookie-parser

3.2 Convert Account Setup

Create an experiment in your Convert account:
- Type: A/B/n experience
- Name: llms-txt-content-test
- Variations:
  - variation-1: Technical documentation focus
  - variation-2: Product marketing focus
- Targeting: No specific rules (all traffic)
- Traffic allocation: 50/50 split
Obtain your SDK Key:
- Navigate to Project Settings → SDK Configuration
- Copy your sdkKey (format: 10xxxxxx/10yyyyyy)

Prepare variation files:

/server/llms/
├── variation-1.txt    # Technical docs focus
└── variation-2.txt    # Marketing focus

3.3 Visitor ID Strategy

You need a function to generate/retrieve a consistent visitor ID. Options:

Option A: Cookie-based (Recommended)

function getVisitorId(req, res) {
  let visitorId = req.cookies.convert_sdk_vid;
  if (!visitorId) {
    visitorId = crypto.randomUUID(); // Node 16+
    res.cookie('convert_sdk_vid', visitorId, {
      maxAge: 365 * 24 * 60 * 60 * 1000, // 1 year
      httpOnly: true,
      sameSite: 'lax'
    });
  }
  return visitorId;
}

Option B: User-Agent hash (for crawlers without cookies)

const crypto = require('crypto');

function hashVisitorId(userAgent, ip) {
  return crypto
    .createHash('sha256')
    .update(userAgent + ip)
    .digest('hex')
    .substring(0, 32);
}

function getVisitorId(req, res) {
  // Try cookie first
  let visitorId = req.cookies.convert_sdk_vid;
  if (!visitorId) {
    // Fallback to User-Agent + IP hash for crawlers
    const userAgent = req.headers['user-agent'] || 'unknown';
    const ip = req.ip || req.connection.remoteAddress || 'unknown';
    visitorId = hashVisitorId(userAgent, ip);
    
    // Still set cookie in case it's accepted
    res.cookie('convert_sdk_vid', visitorId, {
      maxAge: 365 * 24 * 60 * 60 * 1000,
      httpOnly: true
    });
  }
  return visitorId;
}

3.4 Domain & Cookie Scope

Ensure cookies are scoped correctly if you have subdomains:

res.cookie('convert_sdk_vid', visitorId, {
  domain: '.yourdomain.com',  // Shares across www., app., etc.
  path: '/',
  maxAge: 365 * 24 * 60 * 60 * 1000,
  httpOnly: true,
  secure: true,  // HTTPS only
  sameSite: 'lax'
});

4. Implementation: Node.js

4.1 Complete Express.js Example

This example shows a production-ready Express app with SDK integration for /llms.txt testing.

File: server.js

const express = require('express');
const cookieParser = require('cookie-parser');
const fs = require('fs').promises;
const path = require('path');
const crypto = require('crypto');
const ConvertSDK = require('@convertcom/js-sdk');

const app = express();
const PORT = process.env.PORT || 3000;

// ========================================
// 1. SDK INITIALIZATION (once at startup)
// ========================================

let convertSDK;

// Create a single dataStore instance (see section 4.3 for implementation)
const CookieDataStore = require('./cookieDataStore');
const dataStore = new CookieDataStore();

async function initializeSDK() {
  console.log('Initializing Convert SDK...');
  
  const config = {
    sdkKey: process.env.CONVERT_SDK_KEY, // e.g., '10123456/10234567'
    
    // Pass the dataStore instance for persistent bucketing
    dataStore: dataStore,
    
    // Optional: Disable automatic tracking if you want manual control
    network: {
      tracking: true
    },
    
    // Optional: Adjust data refresh interval (default 5 min)
    dataRefreshInterval: 300000 // 5 minutes
  };
  
  convertSDK = new ConvertSDK(config);
  
  // Wait for SDK to fetch config
  await convertSDK.onReady();
  console.log('✓ Convert SDK ready');
}

// Initialize SDK before starting server
initializeSDK().then(() => {
  app.listen(PORT, () => {
    console.log(`Server running on http://localhost:${PORT}`);
  });
}).catch(err => {
  console.error('Failed to initialize SDK:', err);
  process.exit(1);
});

// ========================================
// 2. MIDDLEWARE SETUP
// ========================================

app.use(cookieParser());
app.use(express.json());

// ========================================
// 3. VISITOR ID HELPER
// ========================================

/**
 * Get or create a consistent visitor ID
 * Uses cookie first, falls back to User-Agent hash for crawlers
 */
function getVisitorId(req, res) {
  let visitorId = req.cookies.convert_sdk_vid;
  
  if (!visitorId) {
    // Check if this looks like a crawler (optional optimization)
    const userAgent = req.headers['user-agent'] || 'unknown';
    const isCrawler = /bot|crawler|spider|llm|gpt|claude|perplexity/i.test(userAgent);
    
    if (isCrawler) {
      // Hash-based ID for crawlers (deterministic)
      const ip = req.ip || req.connection.remoteAddress || '0.0.0.0';
      visitorId = crypto
        .createHash('sha256')
        .update(userAgent + ip)
        .digest('hex')
        .substring(0, 32);
    } else {
      // Random UUID for regular users
      visitorId = crypto.randomUUID();
    }
    
    // Set cookie (crawlers may not accept, but users will)
    res.cookie('convert_sdk_vid', visitorId, {
      maxAge: 365 * 24 * 60 * 60 * 1000,
      httpOnly: true,
      secure: process.env.NODE_ENV === 'production',
      sameSite: 'lax',
      domain: process.env.COOKIE_DOMAIN || undefined
    });
  }
  
  return visitorId;
}

// ========================================
// 4. SDK DECISION MIDDLEWARE
// ========================================

/**
 * Understanding context.runExperiences() return value:
 * 
 * Returns: Array<BucketedVariation | RuleError | BucketingError>
 * 
 * Each BucketedVariation object contains:
 * - experienceKey: string      // The experiment's unique key
 * - experienceId: string        // The experiment's ID
 * - experienceName: string      // The experiment's display name
 * - key: string                 // The variation's key (e.g., 'variation-1')
 * - id: string                  // The variation's ID
 * - name: string                // The variation's display name
 * - traffic_allocation: number  // Traffic percentage (0-10000)
 * - changes: Array              // Array of changes/features in this variation
 * 
 * To access a specific experiment, use Array.find():
 *   const variation = variations.find(v => v.experienceKey === 'my-experiment-key');
 */

/**
 * This middleware runs on EVERY request to ensure
 * consistent experiment decisions across all routes
 */
async function sdkMiddleware(req, res, next) {
  if (!convertSDK) {
    console.error('SDK not initialized');
    return res.status(503).send('Service temporarily unavailable');
  }
  
  try {
    // 1. Update dataStore with current request/response for cookie operations
    //    The dataStore instance was passed to SDK config at initialization
    dataStore.req = req;
    dataStore.res = res;
    dataStore.data = req.cookies || {}; // Read existing cookies
    
    // 2. Get/create visitor ID
    const visitorId = getVisitorId(req, res);
    
    // 3. Gather visitor attributes (optional but recommended)
    const visitorAttributes = {
      userAgent: req.headers['user-agent'],
      language: req.headers['accept-language'],
      // Add more attributes as needed for targeting
    };
    
    // 4. Create context for this visitor
    const context = convertSDK.createContext(visitorId, visitorAttributes);
    
    if (!context) {
      console.error('Failed to create context for visitor:', visitorId);
      return next(); // Continue without SDK
    }
    
    // 5. Run ALL active experiments to get consistent decisions
    //    This ensures /llms.txt and /page use the same variations
    //    Returns: Array<BucketedVariation | RuleError | BucketingError>
    //    The SDK will automatically call dataStore.set() to persist decisions
    const variations = context.runExperiences();
    
    // 6. Store in request object for route handlers
    req.convertSDK = {
      visitorId,
      context,
      variations, // Array of BucketedVariation objects
    };
    
    next();
  } catch (error) {
    console.error('SDK middleware error:', error);
    // Fail gracefully - continue without experiment decisions
    req.convertSDK = { visitorId: 'error', variations: [] };
    next();
  }
}

// Apply SDK middleware to all routes
app.use(sdkMiddleware);

// ========================================
// 5. /llms.txt ROUTE HANDLER
// ========================================

app.get('/llms.txt', async (req, res) => {
  try {
    const { variations, visitorId } = req.convertSDK || {};
    
    // Get the decision for our specific llms.txt experiment
    // variations is Array<BucketedVariation>, so we need to find our experiment
    const experienceKey = 'llms-txt-content-test'; // Your experiment key in Convert
    const variation = variations?.find(v => v.experienceKey === experienceKey);
    
    // Determine which file to serve
    let variationKey = 'variation-1'; // Default fallback
    
    if (variation && variation.key) {
      variationKey = variation.key; // e.g., 'variation-1' or 'variation-2'
      console.log(`Serving ${variationKey} for visitor ${visitorId}`);
    } else {
      console.log(`No valid decision, serving default for visitor ${visitorId}`);
    }
    
    // Read the appropriate variation file
    const filePath = path.join(__dirname, 'llms', `${variationKey}.txt`);
    const content = await fs.readFile(filePath, 'utf-8');
    
    // === CRITICAL: Set proper headers ===
    res.setHeader('Content-Type', 'text/plain; charset=utf-8');
    
    // Option A: Consistent caching (same visitor gets same variant on repeat crawls)
    res.setHeader('Cache-Control', 'private, max-age=86400'); // 24 hours
    
    // Option B: No caching (re-evaluate on every hit - not recommended for llms.txt)
    // res.setHeader('Cache-Control', 'no-store');
    
    // Send the content
    res.status(200).send(content);
    
  } catch (error) {
    console.error('Error serving /llms.txt:', error);
    
    // Fallback to default file
    try {
      const fallbackPath = path.join(__dirname, 'llms', 'variation-1.txt');
      const fallbackContent = await fs.readFile(fallbackPath, 'utf-8');
      res.setHeader('Content-Type', 'text/plain; charset=utf-8');
      res.status(200).send(fallbackContent);
    } catch (fallbackError) {
      res.status(500).send('# Error loading content\n\nPlease try again later.');
    }
  }
});

// ========================================
// 6. EXAMPLE REGULAR PAGE ROUTE
// ========================================

app.get('/pricing', (req, res) => {
  const { variations, visitorId } = req.convertSDK || {};
  
  // Find specific experiments from the variations array
  const llmsExperience = variations?.find(v => v.experienceKey === 'llms-txt-content-test');
  const ctaExperience = variations?.find(v => v.experienceKey === 'cta-button-test');
  
  // Example: Render different pricing page based on experiments
  res.send(`
    <!DOCTYPE html>
    <html>
    <head><title>Pricing</title></head>
    <body>
      <h1>Pricing Page</h1>
      <p>Visitor ID: ${visitorId}</p>
      <p>LLMs.txt variation: ${llmsExperience?.key || 'default'}</p>
      <p>CTA variation: ${ctaExperience?.key || 'default'}</p>
      
      <!-- Your actual page content here -->
      
      <script>
        // Optional: Client-side tracking
        console.log('Experiments:', ${JSON.stringify(variations || [])});
      </script>
    </body>
    </html>
  `);
});

// ========================================
// 7. HEALTH CHECK
// ========================================

app.get('/health', (req, res) => {
  res.json({
    status: 'healthy',
    sdk: convertSDK ? 'ready' : 'not initialized',
    timestamp: new Date().toISOString()
  });
});

// ========================================
// 8. ERROR HANDLING
// ========================================

app.use((err, req, res, next) => {
  console.error('Unhandled error:', err);
  res.status(500).json({ error: 'Internal server error' });
});

4.2 Example llms.txt Variation Files

File: llms/variation-1.txt (Technical Focus)

# Convert Experiences - A/B Testing Platform

> Technical Documentation & Developer Resources

## Overview

Convert Experiences is an enterprise A/B testing and experimentation platform designed for high-traffic websites and applications. Our platform enables data-driven optimization through rigorous statistical analysis and privacy-first architecture.

## Core Capabilities

- **Server-Side Testing**: Node.js SDK for backend experiments
- **Client-Side Testing**: JavaScript SDK with sub-50ms performance overhead
- **Feature Flags**: GitOps-compatible feature management with gradual rollouts
- **Privacy Compliance**: GDPR/CCPA ready, cookieless mode, on-premise deployment options

## Developer Resources

- API Documentation: https://developers.convert.com/api?utm_source=llm-v1&utm_medium=llms-txt&utm_campaign=llm-discovery
- JavaScript SDK: https://github.com/convertcom/javascript-sdk?utm_source=llm-v1
- Integration Guides: https://support.convert.com/integrations?utm_source=llm-v1
- Status Page: https://status.convert.com

## Technical Architecture

Our platform uses:
- Statistical Engine: Sequential Testing (always valid inference)
- Data Processing: Real-time stream processing with <100ms latency
- Edge Delivery: Global CDN with 99.99% uptime SLA

## Contact

Enterprise inquiries: enterprise@convert.com
Developer support: https://community.convert.com?utm_source=llm-v1

File: llms/variation-2.txt (Marketing Focus)

# Convert Experiences - Increase Revenue Through Smarter Testing

> The A/B Testing Platform Trusted by 5,000+ Brands Worldwide

## Why Convert?

Convert Experiences helps ambitious companies like yours turn more visitors into customers. Our platform makes experimentation accessible to marketers while providing the advanced features developers demand.

## What We Do

🚀 **Boost Conversions**: Run unlimited A/B tests on any part of your website or app
📊 **Make Data-Driven Decisions**: Get reliable results 30% faster with our advanced stats engine
🔒 **Stay Compliant**: GDPR & CCPA ready out-of-the-box
⚡ **Lightning Fast**: Your visitors won't notice the 50ms performance impact

## Who Uses Convert?

- **E-commerce Brands**: Optimize product pages, checkout flows, and upsells
- **SaaS Companies**: Test pricing pages, onboarding flows, and feature launches
- **Media Publishers**: Improve engagement, reduce bounce rates, grow subscriptions
- **Enterprise Teams**: Collaborate at scale with role-based access and approval workflows

## Get Started

Free 15-Day Trial: https://www.convert.com/free-trial?utm_source=llm-v2&utm_medium=llms-txt&utm_campaign=llm-discovery
See Pricing: https://www.convert.com/pricing?utm_source=llm-v2
Customer Stories: https://www.convert.com/customers?utm_source=llm-v2

## Recognition

- ⭐ 4.7/5 on G2 (250+ reviews)
- 🏆 Leader in A/B Testing Software (G2, 2024)
- 💎 Trusted by 5,000+ websites in 80+ countries

## Contact

Sales: sales@convert.com | +1 (844) 266-8378
Live Chat: https://www.convert.com/contact?utm_source=llm-v2

4.3 Implementing Cookie-Based DataStore

The SDK expects a dataStore object with get(key) and set(key, value) methods. For server-side persistence, we create a stateful singleton that gets updated per-request with req/res objects.

File: cookieDataStore.js

/**
 * Custom DataStore implementation for Express cookies
 * 
 * IMPORTANT: This is a stateful singleton pattern.
 * - Create ONE instance and pass to SDK config at initialization
 * - Update req/res/data properties in middleware on each request
 * - The SDK calls get() and set() methods automatically during bucketing
 */
class CookieDataStore {
  constructor(options = {}) {
    this.req = null;           // Will be set per-request in middleware
    this.res = null;           // Will be set per-request in middleware
    this.data = {};            // Will be populated from req.cookies per-request
    this.expire = options.expire || 365 * 24 * 60 * 60 * 1000; // 1 year default
  }
  
  /**
   * Get stored data for a key
   * Called automatically by SDK when checking for existing bucketing decisions
   * @param {string} key - Storage key (e.g., 'accountId-projectId-visitorId')
   * @returns {Object|null}
   */
  get(key) {
    if (!key) return this.data; // Return all data if no key specified
    return this.data[key.toString()] || null;
  }
  
  /**
   * Set data for a key
   * Called automatically by SDK when storing new bucketing decisions
   * @param {string} key - Storage key
   * @param {Object} value - Data to store (bucketing decisions + segments)
   */
  set(key, value) {
    if (!key) throw new Error('Invalid DataStore key!');
    
    // Update in-memory data
    this.data[key.toString()] = value;
    
    // Persist to cookie if response object is available
    if (this.res) {
      try {
        this.res.cookie(key.toString(), value, {
          maxAge: this.expire,
          httpOnly: true,
          secure: process.env.NODE_ENV === 'production',
          sameSite: 'lax'
        });
      } catch (error) {
        console.error('Error setting cookie:', error);
      }
    }
  }
}

module.exports = CookieDataStore;

How it works:

Initialization (once at app startup):

const dataStore = new CookieDataStore();
const convertSDK = new ConvertSDK({
  sdkKey: 'xxx',
  dataStore: dataStore  // Pass the singleton instance
});

Middleware (on every request):

function sdkMiddleware(req, res, next) {
  // Update the singleton dataStore with current request context
  dataStore.req = req;
  dataStore.res = res;
  dataStore.data = req.cookies || {};  // Load existing cookies
  
  // Now the SDK can read/write cookies for this request
  const context = convertSDK.createContext(visitorId);
  const decisions = context.runExperiences(); // Automatically persists to cookies
  
  next();
}

SDK's automatic behavior:
- When running experiments, SDK calls dataStore.get(key) to check for existing decisions
- If found, SDK reuses the same variation (consistent bucketing)
- If not found, SDK generates new decision and calls dataStore.set(key, value) to persist it

Alternative: Separate cookies per visitor

If you prefer one cookie per visitor (rather than one per storage key), modify the set method:

set(key, value) {
  if (!key) throw new Error('Invalid DataStore key!');
  
  this.data[key.toString()] = value;
  
  if (this.res) {
    try {
      // Use a single cookie name for all Convert data
      const cookieName = '_convert_data';
      
      // Merge with existing cookie data
      const existingData = this.req.cookies[cookieName] 
        ? JSON.parse(this.req.cookies[cookieName]) 
        : {};
      existingData[key.toString()] = value;
      
      // Write merged data back to single cookie
      this.res.cookie(cookieName, JSON.stringify(existingData), {
        maxAge: this.expire,
        httpOnly: true,
        secure: process.env.NODE_ENV === 'production',
        sameSite: 'lax'
      });
    } catch (error) {
      console.error('Error setting cookie:', error);
    }
  }
}

Important notes:

✅ Do: Create ONE dataStore instance, pass to SDK config
✅ Do: Update req, res, data properties in middleware per request
❌ Don't: Create new dataStore instances per request
❌ Don't: Try to pass dataStore to createContext() (it doesn't accept it)

The SDK's DataManager automatically uses the dataStore you configured at initialization.

5. Caching Recommendations

5.1 Why Caching Matters

AI crawlers often:

Crawl infrequently (days/weeks between visits)
Respect cache headers (to reduce load on origin servers)
Don't execute JavaScript (no dynamic cache-busting)

Your caching strategy determines whether a crawler sees consistent content across visits.

5.2 Recommended Cache-Control Headers

Strategy	Header	Use Case
Consistent (Recommended)	`Cache-Control: private, max-age=86400`	Same crawler sees same variant for 24h. Enables stat-sig.
Long-term	`Cache-Control: private, max-age=604800`	Same variant for 7 days. Reduces re-bucketing.
No cache	`Cache-Control: no-store`	Re-evaluate on every request. Destroys consistency. ❌
Public cache	`Cache-Control: public, max-age=3600`	CDN caches one variant for ALL crawlers. Don't use. ❌

5.3 Implementation

app.get('/llms.txt', async (req, res) => {
  // ... serve variation content ...
  
  // Set caching headers
  res.setHeader('Content-Type', 'text/plain; charset=utf-8');
  
  if (process.env.LLMS_CACHE_STRATEGY === 'consistent') {
    // Recommended: Private cache for 24 hours
    res.setHeader('Cache-Control', 'private, max-age=86400');
    res.setHeader('Vary', 'Cookie'); // Cache varies by visitor
  } else if (process.env.LLMS_CACHE_STRATEGY === 'none') {
    // No caching - use sparingly
    res.setHeader('Cache-Control', 'no-store, no-cache, must-revalidate');
    res.setHeader('Pragma', 'no-cache');
  }
  
  res.send(content);
});

5.4 CDN Considerations

If using a CDN (Cloudflare, Fastly, CloudFront):

⚠️ Warning: Default CDN behavior caches one version for all visitors. This breaks experimentation.

Solution A: Vary by Cookie

res.setHeader('Vary', 'Cookie');
res.setHeader('Cache-Control', 'private, max-age=86400');

Solution B: Bypass CDN for /llms.txt

// Cloudflare example
res.setHeader('Cache-Control', 'private, max-age=86400');
res.setHeader('CF-Cache-Status', 'BYPASS');

Solution C: Custom cache key

// Cloudflare Worker example
addEventListener('fetch', event => {
  const url = new URL(event.request.url);
  
  if (url.pathname === '/llms.txt') {
    const cookie = event.request.headers.get('Cookie') || '';
    const visitorId = extractVisitorId(cookie); // Your function
    
    // Custom cache key includes visitor ID
    const cacheKey = new Request(`${url.toString()}?vid=${visitorId}`);
    event.respondWith(fetch(cacheKey));
  }
});

6. Analytics & UTM Tagging

6.1 Why Tag llms.txt Content?

Since you can't track "views" of a plain text file server-side, embed tracking parameters in URLs within the file. When AI tools cite or link to these URLs, you can measure traffic in Google Analytics or your analytics platform.

6.2 Dynamic UTM Injection

Node.js example with template replacement:

app.get('/llms.txt', async (req, res) => {
  const { variations, visitorId } = req.convertSDK || {};
  const experienceKey = 'llms-txt-content-test';
  const variation = variations?.find(v => v.experienceKey === experienceKey);
  const variationKey = variation?.key || 'variation-1';
  
  // Read template file
  const templatePath = path.join(__dirname, 'llms', `${variationKey}.txt`);
  let content = await fs.readFile(templatePath, 'utf-8');
  
  // Replace placeholders with UTM parameters
  const utmParams = new URLSearchParams({
    utm_source: 'llm',
    utm_medium: 'llms-txt',
    utm_campaign: 'ai-discovery',
    utm_content: variationKey,
    utm_term: experienceKey,
    // Optional: Include visitor ID for cohort analysis
    vid: visitorId.substring(0, 8)
  });
  
  // Replace {{BASE_URL}} in template with actual URL + UTMs
  content = content.replace(
    /\{\{BASE_URL\}\}/g,
    `https://www.convert.com?${utmParams.toString()}`
  );
  
  // Replace {{DOCS_URL}} similarly
  content = content.replace(
    /\{\{DOCS_URL\}\}/g,
    `https://developers.convert.com?${utmParams.toString()}`
  );
  
  res.setHeader('Content-Type', 'text/plain; charset=utf-8');
  res.setHeader('Cache-Control', 'private, max-age=86400');
  res.send(content);
});

Updated template file with placeholders:

# Convert Experiences

Visit our website: {{BASE_URL}}
Developer docs: {{DOCS_URL}}

6.3 Tracking in Google Analytics 4

Step 1: Define custom dimensions

In GA4, create custom dimensions:

experiment_id → Event scope
variation_id → Event scope
visitor_id → User scope

Step 2: Track incoming traffic

When users land on your site from AI referrals:

// Example: gtag.js snippet
app.get('*', (req, res, next) => {
  const utmContent = req.query.utm_content; // e.g., 'variation-2'
  const utmTerm = req.query.utm_term;       // e.g., 'llms-txt-content-test'
  
  if (utmContent && utmTerm) {
    // Send custom event to GA4
    res.locals.gaEvent = {
      event_name: 'llm_referral',
      experiment_id: utmTerm,
      variation_id: utmContent,
      traffic_source: req.query.utm_source
    };
  }
  
  next();
});

Step 3: Analyze in GA4

Create an Exploration report:

Dimensions: utm_content, utm_source, landing_page
Metrics: sessions, conversions, revenue
Filters: utm_medium = llms-txt

This shows which llms.txt variation drives more valuable traffic.

7. Troubleshooting & Edge Cases

7.1 Common Issues

Issue: "Visitors see different variations on /llms.txt vs /page"

Cause: Not running experiments consistently across routes.

Solution: Call context.runExperiences() in middleware before all route handlers:

// ✅ Correct: Middleware runs first
app.use(sdkMiddleware); // Decides ALL experiments
app.get('/llms.txt', handleLLMsRoute);
app.get('/page', handlePageRoute);

// ❌ Wrong: Separate context per route
app.get('/llms.txt', (req, res) => {
  const context = sdk.createContext(...); // New context
  context.runExperience('llms-txt-test'); // Only this one
});

Issue: "SDK returns error: 'Experience not found'"

Cause: Experiment key mismatch or experiment not active.

Check:

Verify experienceKey matches exactly (case-sensitive)
Check experiment status in Convert dashboard (must be "Running")
Confirm SDK fetched latest config: convertSDK.onReady().then(() => { /* check data */ })

Debug:

const allExperiences = context.runExperiences();
console.log('Available experiments:', Object.keys(allExperiences));

Issue: "Cookies not persisting for crawlers"

Reality: Most AI crawlers don't accept cookies.

Solution: Use User-Agent + IP hash as fallback visitor ID (see 3.3).

function getVisitorId(req, res) {
  // Try cookie
  let visitorId = req.cookies.convert_sdk_vid;
  
  if (!visitorId) {
    // Fallback to deterministic hash
    const userAgent = req.headers['user-agent'] || 'unknown';
    const ip = req.ip || 'unknown';
    visitorId = crypto.createHash('sha256')
      .update(`${userAgent}:${ip}`)
      .digest('hex')
      .substring(0, 32);
  }
  
  return visitorId;
}

Issue: "Experiment reached statistical significance, but shows no traffic difference"

Cause: UTM parameters not included in llms.txt file.

Check:

Open https://yourdomain.com/llms.txt in browser
Verify URLs include ?utm_source=llm&utm_content=variation-1
Check GA4 for traffic with these parameters

Issue: "SDK initialization fails: 'Invalid SDK key'"

Check:

SDK key format: 10xxxxxx/10yyyyyy (account ID / project ID)
Environment variable is set: process.env.CONVERT_SDK_KEY
Key is active in Convert dashboard

Debug:

console.log('SDK Key:', process.env.CONVERT_SDK_KEY);
convertSDK.onReady()
  .then(() => console.log('SDK ready'))
  .catch(err => console.error('SDK init error:', err));

7.2 Edge Case: Stale Config Bundle

Scenario: You update an experiment in Convert dashboard, but server still serves old variations.

Cause: SDK caches config data (default 5-minute refresh).

Solution A: Manual refresh

// Force immediate config refresh
await convertSDK.fetchConfig();

Solution B: Shorter refresh interval

const convertSDK = new ConvertSDK({
  sdkKey: '...',
  dataRefreshInterval: 60000 // 1 minute instead of 5
});

Solution C: Webhook-triggered refresh

// Endpoint that Convert webhooks can call
app.post('/webhook/convert-config-update', async (req, res) => {
  console.log('Config update webhook received');
  await convertSDK.fetchConfig();
  res.sendStatus(200);
});

7.3 Edge Case: Race Conditions (Concurrent Requests)

Scenario: Same visitor makes simultaneous requests to /llms.txt and /page before SDK finishes bucketing.

Risk: Might get assigned to different variations if cookie write is delayed.

Mitigation:

// Use in-memory cache with TTL for inflight bucketing decisions
const bucketingCache = new Map();
const CACHE_TTL = 5000; // 5 seconds

async function sdkMiddleware(req, res, next) {
  const visitorId = getVisitorId(req, res);
  const cacheKey = visitorId;
  
  // Check if bucketing is in-flight
  if (bucketingCache.has(cacheKey)) {
    const cached = bucketingCache.get(cacheKey);
    console.log(`Using cached variations for ${visitorId}`);
    req.convertSDK = cached;
    return next();
  }
  
  // Perform bucketing
  const context = convertSDK.createContext(visitorId);
  const variations = context.runExperiences(); // Array<BucketedVariation>
  
  // Cache for concurrent requests
  const result = { visitorId, context, variations };
  bucketingCache.set(cacheKey, result);
  
  setTimeout(() => bucketingCache.delete(cacheKey), CACHE_TTL);
  
  req.convertSDK = result;
  next();
}

7.4 Debugging Checklist

When things don't work:

Check SDK initialization:

curl http://localhost:3000/health
# Should show: {"status":"healthy","sdk":"ready"}

Verify visitor ID persistence:

# First request
curl -c cookies.txt http://localhost:3000/llms.txt
# Second request with cookies
curl -b cookies.txt http://localhost:3000/llms.txt
# Should serve same variation

Inspect SDK variations:

console.log('Variations:', JSON.stringify(variations, null, 2));

Check experiment status:
- Convert Dashboard → Experiences → Status: "Running" ✓
- Traffic allocation: 50/50 or as desired ✓
- No audience targeting that blocks all traffic ✓

Test with curl:

# Simulate AI crawler
curl -H "User-Agent: GPTBot/1.0" http://localhost:3000/llms.txt

8. FAQ

8.1 General Questions

Q: Does running experiments server-side affect statistical significance?

A: No, statistical calculations remain valid as long as:

✅ Visitor assignment is random (hash-based bucketing ensures this)
✅ Decisions are consistent (same visitor always sees same variation)
✅ Sample size is sufficient (depends on traffic and effect size)

Q: Will this slow down my server?

A: Minimal impact. Benchmarks:

SDK initialization: ~100-200ms (once at startup)
Context creation + decision: ~1-5ms per request
Cookie read/write: <1ms

For high-traffic sites, consider:

Edge computing (Cloudflare Workers, Vercel Edge Functions)
Caching decisions in Redis (TTL = 24h)

Q: How many experiments can I run simultaneously?

A: The SDK handles unlimited experiments, but practical limits:

Cookie size: Browsers limit cookies to ~4KB. Each bucketing decision adds ~20-50 bytes.
Performance: runExperiences() evaluates all experiments. With >50 experiments, consider selective evaluation:

// Instead of runExperiences() (all experiments):
const decision1 = context.runExperience('exp-1');
const decision2 = context.runExperience('exp-2');
// Only run experiments needed for this route

Q: Can I A/B test the structure of llms.txt, not just content?

A: Yes! Example variations:

Variation 1: Markdown format with H1/H2 headings
Variation 2: Plain text with ASCII art hierarchy
Variation 3: JSON-LD structured data

Serve different files as before. Test which format AI tools parse better.

8.2 Technical Questions

Q: What if a crawler doesn't send cookies or User-Agent?

A: Fallback chain:

Try convert_sdk_vid cookie
Hash User-Agent + IP
Hash IP + request path
Use fixed default ID (= "control group")

function getVisitorIdWithFallback(req) {
  return req.cookies.convert_sdk_vid
    || hashUserAgent(req.headers['user-agent'], req.ip)
    || hashIP(req.ip, req.path)
    || 'default-visitor'; // Always gets variation-1
}

Q: How do I test this locally before deploying?

Option 1: Simulate crawler with curl

# Request 1: No cookies
curl -v http://localhost:3000/llms.txt
# Note the Set-Cookie header

# Request 2: With cookies (should get same variation)
curl -v -H "Cookie: convert_sdk_vid=abc123" http://localhost:3000/llms.txt

Option 2: Browser dev tools

// In browser console:
fetch('/llms.txt').then(r => r.text()).then(console.log);
// Check Network tab → Response Headers → Cache-Control

Option 3: Unit tests

const request = require('supertest');
const app = require('./server');

describe('/llms.txt experimentation', () => {
  it('serves consistent variation for same visitor', async () => {
    const agent = request.agent(app); // Persists cookies
    
    const res1 = await agent.get('/llms.txt');
    const res2 = await agent.get('/llms.txt');
    
    expect(res1.text).toBe(res2.text); // Same content
  });
  
  it('tracks variation in UTM parameters', async () => {
    const res = await request(app).get('/llms.txt');
    expect(res.text).toMatch(/utm_content=variation-[12]/);
  });
});

Q: Can I use this with Next.js Edge Runtime?

A: Yes, with caveats:

✅ SDK works in Edge Runtime (Vercel, Cloudflare Workers)
⚠️ No filesystem access → Store variation content in KV store or inline
⚠️ Limited crypto APIs → Use crypto.subtle.digest() for hashing

// File: pages/llms.txt.js (Next.js Edge Function)
import ConvertSDK from '@convertcom/js-sdk';

export const config = { runtime: 'edge' };

const sdk = new ConvertSDK({ sdkKey: process.env.CONVERT_SDK_KEY });

export default async function handler(req) {
  const visitorId = req.cookies.get('convert_sdk_vid')?.value || crypto.randomUUID();
  const context = sdk.createContext(visitorId);
  const variations = context.runExperiences(); // Array<BucketedVariation>
  
  // Find the specific experiment from the array
  const bucketedVariation = variations.find(v => v.experienceKey === 'llms-txt-test');
  const variationKey = bucketedVariation?.key || 'variation-1';
  
  // Inline content (no fs.readFile in Edge)
  const content = VARIATIONS[variationKey];
  
  return new Response(content, {
    headers: {
      'Content-Type': 'text/plain; charset=utf-8',
      'Cache-Control': 'private, max-age=86400',
      'Set-Cookie': `convert_sdk_vid=${visitorId}; Max-Age=31536000; HttpOnly`
    }
  });
}

const VARIATIONS = {
  'variation-1': '# Technical Content...',
  'variation-2': '# Marketing Content...'
};

8.3 Business Questions

Q: How long should I run the experiment?

A: Until statistical significance and stability:

Minimum: 2 weeks (captures weekly patterns)
Target: >1,000 unique visitors per variation
Stability: Winner stays winner for 3+ days

Q: What should I measure as "success"?

Primary metrics:

Traffic quality: Bounce rate, pages/session of UTM-tagged traffic
Conversions: Sign-ups, purchases from AI-referred visitors
Engagement: Time on site, return visits

Secondary metrics: 4. Citation frequency: How often AI tools link to your site 5. Brand mentions: Track site:yourdomain.com in AI chat exports

Q: What if I see no significant difference?

This is a valid result! It means:

✅ Your current llms.txt is "good enough"
✅ Content style doesn't matter (focus on other optimizations)
💡 Consider testing radically different hypotheses:
- Long-form (500+ words) vs. short (100 words)
- Question-answer format vs. narrative
- Product-focused vs. problem-focused

9. Additional Resources

9.1 Convert SDK Documentation

JavaScript SDK Wiki: github.com/convertcom/javascript-sdk/wiki
Core Module: ConvertSDK / Core
Context API: Context
Data Management: DataManager
Bucketing Logic: BucketingManager

9.2 llms.txt Specification

Official Spec: llmstxt.org
Adoption Analysis: SearchEngineLand - llms.txt Proposed Standard
Critique: PPC Land - Adoption Stalls
Case Study: Mintlify - The Value of llms.txt

Appendix A: Complete Minimal Example

For quick copy-paste testing:

File: minimal-server.js

const express = require('express');
const cookieParser = require('cookie-parser');
const crypto = require('crypto');
const ConvertSDK = require('@convertcom/js-sdk');

const app = express();
app.use(cookieParser());

// Simple DataStore for cookie persistence
class SimpleDataStore {
  constructor() {
    this.req = null;
    this.res = null;
    this.data = {};
  }
  get(key) {
    return key ? this.data[key.toString()] : this.data;
  }
  set(key, value) {
    if (!key) return;
    this.data[key.toString()] = value;
    if (this.res) {
      this.res.cookie(key.toString(), value, {
        maxAge: 31536000000,
        httpOnly: true
      });
    }
  }
}

const dataStore = new SimpleDataStore();

// Initialize SDK with dataStore
const sdk = new ConvertSDK({
  sdkKey: process.env.CONVERT_SDK_KEY || '10123456/10234567',
  dataStore: dataStore // Pass the singleton instance
});

// Middleware: Decide experiments
app.use((req, res, next) => {
  sdk.onReady().then(() => {
    // Update dataStore with current request context
    dataStore.req = req;
    dataStore.res = res;
    dataStore.data = req.cookies || {};
    
    const visitorId = req.cookies._v || crypto.randomUUID();
    res.cookie('_v', visitorId, { maxAge: 31536000000, httpOnly: true });
    
    const context = sdk.createContext(visitorId);
    req.variations = context.runExperiences(); // Auto-persists via dataStore
    
    next();
  }).catch(next);
});

// Serve /llms.txt
app.get('/llms.txt', (req, res) => {
  // Find the specific experiment from the variations array
  const bucketedVariation = req.variations?.find(v => v.experienceKey === 'llms-txt-test');
  const variationKey = bucketedVariation?.key || 'variation-1';
  const content = variationKey === 'variation-1'
    ? '# Technical Content\n\nDocs: https://example.com/docs?utm_content=v1'
    : '# Marketing Content\n\nProduct: https://example.com/product?utm_content=v2';
  
  res.type('text/plain').send(content);
});

// Start server
sdk.onReady().then(() => {
  app.listen(3000, () => console.log('Server running on :3000'));
});

Run:

npm install express cookie-parser @convertcom/js-sdk
CONVERT_SDK_KEY=your_key_here node minimal-server.js
curl http://localhost:3000/llms.txt

Appendix B: Cookie Structure Reference

When using a custom cookie-based dataStore implementation (like the one in section 4.3), bucketing decisions are stored in this format:

Cookie name: convert_sdk_vid

Value (JSON):

{
  "10123456-10234567-visitor-abc123": {
    "bucketing": {
      "10345678": "10456789",
      "10345679": "10456790"
    },
    "segments": {
      "country": "US",
      "device": "desktop"
    }
  }
}

Key structure: {accountId}-{projectId}-{visitorId}

Bucketing format: {experienceId}: {variationId} (numeric IDs)

Size estimate:

Base overhead: ~50 bytes
Per experiment: ~20 bytes
10 experiments: ~250 bytes total
50 experiments: ~1,050 bytes total

Cookie limits:

Per cookie: 4,096 bytes
Per domain: 50 cookies (browser-dependent)

Appendix C: Visual Decision Tree

┌─────────────────────────────────────────────┐
│  Request arrives (any route)                │
└──────────────────┬──────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────┐
│  SDK Middleware: getVisitorId(req, res)     │
└──────────────────┬──────────────────────────┘
                   │
        ┌──────────┴──────────┐
        │                     │
        ▼                     ▼
   Has Cookie?            No Cookie
    convert_sdk_vid    (first visit)
        │                     │
        │              ┌──────┴──────┐
        │              │             │
        │              ▼             ▼
        │         AI Crawler?   Regular User
        │          (UA match)    (random ID)
        │              │             │
        │              ▼             │
        │        Hash(UA + IP)       │
        │              │             │
        └──────────────┴─────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────┐
│  context = sdk.createContext(visitorId)     │
└──────────────────┬──────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────┐
│  variations = context.runExperiences()      │
│  Returns: Array<BucketedVariation>          │
│  (Checks cookie for existing bucketing)     │
└──────────────────┬──────────────────────────┘
                   │
        ┌──────────┴──────────┐
        │                     │
        ▼                     ▼
   Has stored         No stored decision
   decision?          (new experiment)
        │                     │
        │                     ▼
        │         ┌─────────────────────────┐
        │         │ BucketingManager:       │
        │         │ Hash(visitorId + expId) │
        │         │ → Assign variation      │
        │         └───────────┬─────────────┘
        │                     │
        │                     ▼
        │         ┌─────────────────────────┐
        │         │ Store in cookie:        │
        │         │ {exp123: var2}          │
        │         └───────────┬─────────────┘
        │                     │
        └─────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────┐
│  Route Handler: Use variations array        │
│  - Find specific experiment using           │
│    variations.find(v => v.experienceKey)    │
│  - /llms.txt → serve variant file           │
│  - /page → render with variant changes      │
└─────────────────────────────────────────────┘

Core Modules

Integration Guides

Server-Side llms.txt Testing

LLMTesting

Integrating llms.txt with the Server-Side JS SDK

Table of Contents

1. Purpose & Context

1.1 What is llms.txt?

1.2 Why Test llms.txt Variations?

1.3 Why Server-Side SDK Integration Matters

2. Architectural Overview

2.1 Request Flow Diagram

2.2 Sequence: Simultaneous /page and /llms.txt Requests

2.3 Key Principles

3. Prerequisites

3.1 SDK Requirements

3.2 Convert Account Setup

3.3 Visitor ID Strategy

3.4 Domain & Cookie Scope

4. Implementation: Node.js

4.1 Complete Express.js Example

4.2 Example llms.txt Variation Files

4.3 Implementing Cookie-Based DataStore

5. Caching Recommendations

5.1 Why Caching Matters

5.2 Recommended Cache-Control Headers

5.3 Implementation

5.4 CDN Considerations

6. Analytics & UTM Tagging

6.1 Why Tag llms.txt Content?

6.2 Dynamic UTM Injection

6.3 Tracking in Google Analytics 4

7. Troubleshooting & Edge Cases

7.1 Common Issues

Issue: "Visitors see different variations on /llms.txt vs /page"

Issue: "SDK returns error: 'Experience not found'"

Issue: "Cookies not persisting for crawlers"

Issue: "Experiment reached statistical significance, but shows no traffic difference"

Issue: "SDK initialization fails: 'Invalid SDK key'"

7.2 Edge Case: Stale Config Bundle

7.3 Edge Case: Race Conditions (Concurrent Requests)

7.4 Debugging Checklist

8. FAQ

8.1 General Questions

8.2 Technical Questions

8.3 Business Questions

9. Additional Resources

9.1 Convert SDK Documentation

9.2 llms.txt Specification

Appendix A: Complete Minimal Example

Appendix B: Cookie Structure Reference

Appendix C: Visual Decision Tree

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally