Skip to content

Add Schema Versioning & Evolution Guide #1

@rz1989s

Description

@rz1989s

Problem Statement

Context7 Benchmark Impact: Question 2 scored 72/100 (lowest score for docs-lumos)

Q2 Feedback (72/100):

"The context lacks explicit guidance on versioning strategies, backward compatibility considerations, schema evolution patterns, and how to handle breaking changes when updating schemas—which are critical for the specific question asked."

Critical Gaps:

  • No versioning strategies documentation
  • No backward compatibility guides
  • No schema evolution patterns
  • No breaking change handling

This is the same gap as lumos core, but for documentation site.

Proposed Solution

Add comprehensive versioning documentation to the docs site.

1. New Page: docs/guide/versioning.md

---
title: Schema Versioning
description: Strategies for versioning LUMOS schemas in production
---

# Schema Versioning

## Overview

As your Solana program evolves, you'll need to update your schemas. This guide covers versioning strategies, backward compatibility, and migration patterns.

## Versioning Strategies

### Semantic Versioning for Schemas

Apply semantic versioning to your schemas:

- **Major (1.0.0 → 2.0.0):** Breaking changes
  - Field removed
  - Field type changed
  - Field renamed
  - Required field added

- **Minor (1.0.0 → 1.1.0):** Non-breaking additions
  - Optional field added (`Option<T>`)
  - New enum variant appended
  - Documentation updates

- **Patch (1.0.0 → 1.0.1):** No schema changes
  - Comment updates
  - Code generation improvements

### Version Tracking

Track schema versions in your repository:

```rust
// schemas/v1/player.lumos
#[solana]
#[account]
struct PlayerAccount {
    wallet: PublicKey,
    score: u32,
}
// schemas/v2/player.lumos
#[solana]
#[account]
struct PlayerAccount {
    wallet: PublicKey,
    score: u64,        // Type changed (BREAKING)
    level: u16,        // New field (BREAKING - not optional)
}

Backward Compatibility

Non-Breaking Changes

Safe schema updates that maintain compatibility:

Adding Optional Fields

// v1
struct Player {
    wallet: PublicKey,
    score: u32,
}

// v2 (backward compatible)
struct Player {
    wallet: PublicKey,
    score: u32,
    level: Option<u16>,  // Optional - existing data still valid
}

Migration: Existing accounts work without changes. New field defaults to None.

Appending Enum Variants

// v1
enum GameState {
    Active,    // discriminant: 0
    Paused,    // discriminant: 1
}

// v2 (backward compatible)
enum GameState {
    Active,    // discriminant: 0 (unchanged)
    Paused,    // discriminant: 1 (unchanged)
    Finished,  // discriminant: 2 (new)
}

Migration: Existing data deserializes correctly. Old code ignores new variant.

Breaking Changes

Changes requiring data migration:

Removing Fields

// v1
struct Account {
    wallet: PublicKey,
    email: String,      // Deprecated
    score: u32,
}

// v2 (BREAKING)
struct Account {
    wallet: PublicKey,
    // email removed (BREAKING)
    score: u32,
}

Migration Required: Must read v1 data, transform, write as v2.

Changing Field Types

// v1
struct Account {
    balance: u32,  // Max: ~4 billion
}

// v2 (BREAKING)
struct Account {
    balance: u64,  // Max: ~18 quintillion
}

Migration Required: Cannot deserialize v1 data with v2 schema.

Schema Evolution Patterns

Pattern 1: Deprecation with Transition Period

Mark fields as deprecated before removal:

// v1.5 (transition)
struct Account {
    wallet: PublicKey,
    #[deprecated("Use new_email instead. Will be removed in v2.0")]
    email: String,
    new_email: Option<String>,
}

// v2.0 (removal)
struct Account {
    wallet: PublicKey,
    email: String,  // Renamed from new_email
}

Timeline:

  1. v1.5: Add new_email, deprecate email
  2. Migrate data over 2-4 weeks
  3. v2.0: Remove old email, rename new_emailemail

Pattern 2: Dual-Schema Support

Support both old and new schemas simultaneously:

// Program supports both
enum AccountVersion {
    V1(AccountV1),
    V2(AccountV2),
}

impl AccountVersion {
    pub fn deserialize(data: &[u8]) -> Result<Self> {
        // Try v2 first
        if let Ok(v2) = AccountV2::try_from_slice(data) {
            return Ok(Self::V2(v2));
        }
        // Fallback to v1
        let v1 = AccountV1::try_from_slice(data)?;
        Ok(Self::V1(v1))
    }
    
    pub fn to_v2(self) -> AccountV2 {
        match self {
            Self::V2(acc) => acc,
            Self::V1(acc) => acc.migrate_to_v2(),
        }
    }
}

Pattern 3: Version Discriminator

Add explicit version field:

struct Account {
    version: u8,  // Schema version
    wallet: PublicKey,
    // ... other fields
}

impl Account {
    pub fn deserialize_any(data: &[u8]) -> Result<Self> {
        let version = data[8]; // After discriminator
        match version {
            1 => Self::from_v1(data),
            2 => Self::from_v2(data),
            _ => Err(Error::UnsupportedVersion),
        }
    }
}

Migration Strategies

Strategy 1: One-Time Migration Script

Migrate all accounts in one transaction:

// migration.rs
use anchor_lang::prelude::*;

pub fn migrate_accounts(ctx: Context<MigrateAccounts>) -> Result<()> {
    let old_data = AccountV1::try_from_slice(&ctx.accounts.account.data)?;
    
    let new_data = AccountV2 {
        wallet: old_data.wallet,
        score: old_data.score as u64,  // u32 → u64
        level: 1,  // Default value
    };
    
    // Realloc if needed
    ctx.accounts.account.realloc(new_data.size(), false)?;
    
    // Write new data
    new_data.serialize(&mut *ctx.accounts.account.data.borrow_mut())?;
    Ok(())
}

Strategy 2: Lazy Migration

Migrate on first access:

pub fn update_score(ctx: Context<UpdateScore>, amount: u64) -> Result<()> {
    let account = &mut ctx.accounts.player;
    
    // Check if migration needed
    if account.version < 2 {
        migrate_to_v2(account)?;
    }
    
    account.score += amount;
    Ok(())
}

Strategy 3: Gradual Migration

Allow both versions during transition:

pub fn process_account(data: &[u8]) -> Result<Score> {
    // Try new schema first
    if let Ok(v2) = AccountV2::try_from_slice(data) {
        return Ok(v2.score);
    }
    
    // Fall back to old schema
    let v1 = AccountV1::try_from_slice(data)?;
    Ok(v1.score as u64)
}

Best Practices

1. Plan Breaking Changes Carefully

  • Announce breaking changes 2-4 weeks in advance
  • Provide migration tools for users
  • Document migration path clearly
  • Consider if breaking change is truly necessary

2. Use Feature Flags

struct Account {
    wallet: PublicKey,
    score: u32,
    #[cfg(feature = "v2")]
    level: u16,
}

3. Test Migrations Thoroughly

#[cfg(test)]
mod tests {
    #[test]
    fn test_v1_to_v2_migration() {
        let v1 = AccountV1 {
            wallet: Pubkey::new_unique(),
            score: 100,
        };
        
        let v2 = migrate_to_v2(v1);
        
        assert_eq!(v2.score, 100);
        assert_eq!(v2.level, 1); // Default
    }
}

4. Document Version History

Keep a changelog in your schema directory:

# Schema Changelog

## v2.0.0 (2025-12-01)
- **BREAKING:** Changed `score` from u32 to u64
- **BREAKING:** Added required `level` field
- Migration guide: [docs/migrations/v1-to-v2.md]

## v1.1.0 (2025-11-01)
- Added optional `achievements` field

## v1.0.0 (2025-10-01)
- Initial release

Related


### 2. New Page: `docs/guide/migrations.md`

```markdown
---
title: Data Migrations
description: Patterns for migrating on-chain data between schema versions
---

# Data Migrations

## Migration Workflow

### Step 1: Identify Breaking Changes

Use `lumos diff` to compare schemas:

```bash
lumos diff schemas/v1/player.lumos schemas/v2/player.lumos

Output:

Breaking changes detected:
- Field 'email' removed
- Field 'score' type changed: u32 → u64

Step 2: Write Migration Code

pub fn migrate_player_v1_to_v2(old: PlayerV1) -> PlayerV2 {
    PlayerV2 {
        wallet: old.wallet,
        score: old.score as u64,  // Widen type
        level: calculate_level(old.score),  // Derive new field
    }
}

Step 3: Deploy Migration Instruction

pub fn migrate_account(ctx: Context<MigrateAccount>) -> Result<()> {
    let old = PlayerV1::try_from_slice(&ctx.accounts.player.data)?;
    let new = migrate_player_v1_to_v2(old);
    
    // Realloc if size changed
    let new_size = 8 + new.try_to_vec()?.len();
    ctx.accounts.player.realloc(new_size, false)?;
    
    // Write new data
    new.serialize(&mut *ctx.accounts.player.data.borrow_mut())?;
    Ok(())
}

Step 4: Migrate Accounts

// TypeScript migration script
for (const accountPubkey of accountsToMigrate) {
  await program.methods
    .migrateAccount()
    .accounts({ player: accountPubkey })
    .rpc();
}

Complete Example

See /examples/migrations/v1-to-v2/ for full migration example.


### 3. New Directory: `docs/examples/versioning/`

Create example files showing:
- Simple version update (additive)
- Breaking change migration
- Dual-schema support pattern

### 4. Update Navigation

Add to `.vitepress/config.ts`:

```typescript
sidebar: {
  '/guide/': [
    // ... existing items
    {
      text: 'Advanced',
      items: [
        { text: 'Versioning', link: '/guide/versioning' },
        { text: 'Migrations', link: '/guide/migrations' },
      ]
    }
  ]
}

Acceptance Criteria

  • docs/guide/versioning.md created with comprehensive guide
  • docs/guide/migrations.md created with migration patterns
  • docs/examples/versioning/ directory with 3 examples:
    • Additive change example
    • Breaking change migration
    • Dual-schema support
  • Navigation updated in .vitepress/config.ts
  • Cross-links added to related pages
  • Target: Q2: 72→90 (+18 points)

Impact

Context7 Benchmark:

  • Q2: 72 → 90 (+18 points)

Overall Score: 88.0 → 89.8 (+1.8 points)

User Value:

  • Clear versioning strategies
  • Production migration patterns
  • Reduced risk in schema updates
  • Better long-term maintainability

Related

  • Context7 Benchmark Question 2
  • lumos core issue #79 (same topic)
  • Existing deprecation attribute support

Priority Justification

🔴 CRITICAL - Lowest docs score (72), critical for production usage, aligns with core repo improvements

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions