v2.1.0 (AI Ingestor)

Rosmarium COS v2.1.0

We are thrilled to announce the release of Rosmarium v2.1.0!

This major minor release introduces the Extensible Data Ingestion Pipeline, a complete architectural leap that transforms Rosmarium from a traditional CMS into a true AI-native headless data aggregation platform.

🚀 Key Features

1. Extensible Ingestion Extractors

The ingestion pipeline is no longer limited to web crawling. The backend AI Worker now dynamically handles multiple source types, turning raw unstructured data from anywhere into clean, validated GraphQL content types:

• File System Support: Ingest structured JSON arrays, XML files, or unstructured text files directly from local storage.
• Database Synchronization: Connect directly to PostgreSQL or MongoDB. Rosmarium will execute your SELECT queries or collections sweeps and map each row/document into the AI classifier pipeline.
• Cloud Storage: Native integration with Amazon S3 and MinIO endpoints, allowing you to bulk-ingest content directly from cloud buckets.

2. Hardware-Aware Dynamic AI Models

The system now intelligently probes the hardware it's running on (evaluating CUDA availability and system RAM) to automatically recommend the optimal classification and embedding models.

• The Ingestion UI now features a dynamic model selector that defaults to the recommended model for your cluster's hardware capabilities.
• Fallback logic has been introduced to smoothly default to sentence-transformers CPU embedding if an external Ollama cluster is unavailable.

3. Prompt Overrides & AI Tweaking

You now have fine-grained control over the AI extraction process directly from the Admin UI:

• Custom System Prompt overrides allow you to redefine the persona and rules of the extraction LLM on a per-job basis.
• Custom User Prompt Templates allow you to instruct the LLM precisely on how to map your raw data fields to your Rosmarium Content Types.

🐛 Bug Fixes & Optimizations

• Rate Limit Bypassing: Internal traffic between the Python AI worker and the Node.js API server is now correctly whitelisted, fixing the 429 Too Many Requests errors during high-frequency bulk ingestion jobs.
• UI Form Validation: Fixed an issue where the "Start Import" button in the Admin UI would remain disabled when using File or Database ingestion modes.
• RBAC API Scopes: Internal wildcard * API keys correctly inherit system-level permissions for job polling and webhook executions.

🛠️ Upgrading

To upgrade to v2.1.0, pull the latest images and run the migration scripts:

git pull origin main
pnpm install
pnpm db:migrate
pnpm infra:up

Note: If you plan on using the new MongoDB or S3 extractors, ensure your Python environment installs the new motor and aioboto3 optional dependencies in rosmarium-ai-worker.