Skip to content

v3.0.0.b4

Latest
Latest

Choose a tag to compare

View all tags
@JSv4 JSv4 released this 08 Feb 17:23
v3.0.0.b4
a2667b4
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

OpenContracts v3.0.0.b4

Highlights

This beta release includes significant new features, security hardening, performance optimizations, and UI modernization across the platform.

Auth0 Authentication for Django Admin

  • Django admin now supports Auth0 SSO login with password fallback
  • Admin claims synchronization via Auth0 token claims (is_staff, is_superuser)
  • Open redirect prevention, CSRF protection, and in-memory token storage
  • 50+ security tests covering edge cases

Runtime-Configurable Pipeline Settings

  • PipelineSettings singleton model: Database-backed configuration for parsers, embedders, and thumbnailers -- no more Django settings file changes for runtime config
  • Encrypted secrets storage: Fernet-encrypted API keys and credentials, never exposed via GraphQL
  • Visual pipeline configuration UI: Replaces JSON editing with clickable component cards, per-stage MIME type selectors, and auto-expanding settings
  • Management command migrate_pipeline_settings: Self-documenting component discovery and settings migration
  • GraphQL mutations for updatePipelineSettings, resetPipelineSettings, updateComponentSecrets, and deleteComponentSecrets

Personal Corpus ("My Documents")

  • Each user automatically receives a personal "My Documents" corpus
  • All uploads without a specified corpus default to personal corpus
  • Data migration creates personal corpuses for existing users (irreversible)

Document Processing Pipeline Hardening

  • Processing status tracking with PENDING/PROCESSING/COMPLETED/FAILED states
  • Typed parsing exceptions with transient/permanent distinction
  • Automatic retry with exponential backoff (up to 3 retries)
  • Manual retry via RetryDocumentProcessing GraphQL mutation
  • Failure notifications to document creator

Bifurcated Conversation Permissions (CHAT vs THREAD)

  • CHAT type: Restrictive permissions (creator + explicit + public)
  • THREAD type: Context-based permissions inherited from corpus/document
  • ConversationQueryOptimizer for request-level caching

Corpus Forking Improvements

  • Folder hierarchy preservation during fork
  • Relationship preservation with annotation remapping
  • Multiple bug fixes for edge cases (missing label sets, slug collisions, etc.)

Corpus-Scoped MCP Endpoints

  • New /mcp/corpus/{corpus_slug}/ endpoint for single-corpus shareable links
  • Auto-injected corpus context for all tool calls
  • TTL-based cache with LRU eviction

Multimodal Embedding Support

  • CLIP ViT-L-14 multimodal embedder (768-dim shared text/image space)
  • 2048-dimensional embedding support for newer models
  • Pre-extracted image content for annotations (~10-20x faster embeddings)
  • Corpus-specific dual embedding strategy

Frontend Modernization

  • Unified Upload Modal: Consolidated bulk and single upload into one component with @os-legal/ui design system
  • Corpus List View redesign: Hero section, stats grid, filter tabs, CollectionCard components
  • Extract View: Route-based architecture with WebSocket real-time updates (replacing polling)
  • NavMenu migration: @os-legal/ui NavBar with built-in responsive hamburger menu
  • Landing page redesign: Corpus categories, CompactLeaderboard, skeleton loading states
  • Annotations Panel: Reusable shared component with semantic search, filtering, and pagination
  • GraphQL query optimization: Eliminated ~39 N+1 queries on landing page via subquery-backed count fields

Security

  • WebSocket agent permission vulnerability fixed: Three-layer permission model prevents unauthorized access
  • JWT error message hardening: Generic messages to prevent information disclosure
  • Sensitive data redaction in logs: API keys, secrets, passwords redacted from logs
  • IDOR prevention hardened across GraphQL mutations and REST endpoints
  • TOCTOU fix for agent thread actions with select_for_update()

Other Notable Changes

  • LlamaParse document parser integration
  • Thread/message triggered corpus actions for automated moderation
  • Moderation dashboard with rollback features
  • Real-time notification system via WebSocket
  • Unified JWT authentication utility
  • Bulk document selection and removal
  • Agent chat processing indicator
  • MCP telemetry tracking (privacy-preserving)
  • Secure zip import with folder structure preservation

Breaking Changes

  • ModernBERT embedders removed: Switch to SentenceTransformerEmbedder, OpenAIEmbedder, or VoyageAIEmbedder
  • NLM Ingest Parser removed: Use Docling (default) or LlamaParse
  • Migration 0038 is irreversible: Creates personal corpuses for all users
  • Migration 0039: Removes legacy corpus.documents M2M field (use DocumentPath)

Upgrade Notes

  1. Back up your database before upgrading
  2. Run migrations: docker compose -f production.yml --profile migrate up migrate
  3. If using ModernBERT embedders, update PipelineSettings before upgrading
  4. If using NLM parser, switch to PDF_PARSER=docling or PDF_PARSER=llamaparse
  5. Review docs/pipelines/pipeline_configuration.md for new runtime configuration options

Full changelog: https://github.com/Open-Source-Legal/OpenContracts/blob/main/CHANGELOG.md

Assets 2
Loading