core: Cairo agent migration from starknet-agent repo

[ijusttookadnatest]

Migration from Starknet Agent to Cairo Coder

This PR involved creating Cairo Coder, a specialized version of Starknet Agent, focusing exclusively on the Cairo code generation functionality. Here's a summary of the main changes:

Major Architectural Changes

1. Application Refocus: Transformation from a complete RAG search tool (Starknet Agent) into a Cairo code generator via API (Cairo Coder)

2. UI Removal: Complete elimination of the frontend and WebSocket communication in favor of a pure REST API

3. Infrastructure Simplification:

   • Removal of non-essential configuration endpoints
   • Standardization on hosted mode (HOSTED_MODE) as the only operating option
   • Removal of dynamic configuration options via UI

4. API Adaptation:

   • Creation of a /generate endpoint compatible with the OpenAI format
   • Standardized response format that follows the OpenAI API structure

Technical Modifications

1. Backend Refactoring:

   • Replacement of WebSocket communication with standard Express routes
   • Restructuring of the RAG pipeline to focus on code generation

2. Module Simplification:

   • Removal of the /config endpoint and all its associated functionality
   • Code optimization with the assumption of always operating in hosted mode

3. Updated Documentation:

   • README update to reflect the new mission of the tool
   • Addition of a "Credits" section acknowledging the origin from Starknet Agent
   • Rewriting of installation instructions and API usage

4. Preserved Ingestion Process:

   • Retention of the document ingestion infrastructure for RAG
   • Maintenance of the ingester architecture for different Cairo documentation sources

Rationale for Migration

This migration allowed:

1. Creating a specialized tool focused on a single task: Cairo code generation
2. Simplifying the architecture to facilitate maintenance and future evolution
3. Offering an API compatible with existing standards (OpenAI format) for easy integration

These changes transformed a general research tool for the Starknet ecosystem into a specialized service for Cairo developers, providing a simple and standard interface for generating quality Cairo code from natural language descriptions.