Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
f90c5bc
migrate to DSPy
enitrat Jul 15, 2025
baaba89
--wip-- [skip ci]
enitrat Jul 15, 2025
3637e72
--wip-- [skip ci]
enitrat Jul 15, 2025
0282a2d
use DSPy PGVector
enitrat Jul 15, 2025
36070d1
feat: native PGVector + DSPY streaming
enitrat Jul 15, 2025
dacc515
feat: add optimizer for query processing
enitrat Jul 15, 2025
4f2b436
fix: properly filter sources in embedding search
enitrat Jul 15, 2025
5817747
cleanup and test fixes
enitrat Jul 16, 2025
0cdd0d1
enhance context in retriever
enitrat Jul 16, 2025
75b725e
write optimizer scripts for generation
enitrat Jul 16, 2025
ea58566
re-optimize retrieval
enitrat Jul 17, 2025
ecd5e31
force scarb nightly for faster compilation in runner-crate
enitrat Jul 17, 2025
4204816
remove useless asyncs
enitrat Jul 17, 2025
b943f6d
add optimizer on RAG run
enitrat Jul 17, 2025
1a3482d
add token usage tracking
enitrat Jul 17, 2025
9884b6e
feat: add starklings evaluator (py-based)
enitrat Jul 17, 2025
87f19c4
use multi-workers server
enitrat Jul 17, 2025
66b06f3
cleanup
enitrat Jul 17, 2025
5c53531
add chat/completions endpoint
enitrat Jul 17, 2025
023091a
feat: add langsmith tracing
enitrat Jul 17, 2025
464d727
update instructions
enitrat Jul 17, 2025
18268e2
migrate dockerfile
enitrat Jul 17, 2025
283d227
ruff fmt
enitrat Jul 17, 2025
cd3fc93
docs: maintainer guide
enitrat Jul 17, 2025
e8b9827
fix deprecated method
enitrat Jul 18, 2025
f601a02
fix types post-ruff lints
enitrat Jul 19, 2025
dd26593
minor tweaks
enitrat Jul 19, 2025
c0d0b5f
feat: add DSPY summarization for Corelib
enitrat Jul 20, 2025
07bacd1
dev: use Evaluator to establish baseline perf
enitrat Jul 20, 2025
9df5871
feat: support async
enitrat Jul 20, 2025
0773cf4
fix AgentFactory instanciation
enitrat Jul 20, 2025
adcfea2
dev: remove dead VectorStore
enitrat Jul 20, 2025
235bdbd
fix dockerfile
enitrat Jul 22, 2025
0642e70
add API Keys instructions
enitrat Jul 22, 2025
67880f9
committed uv.lock
enitrat Jul 22, 2025
ebed080
typecheck fixes, readme updates
enitrat Jul 22, 2025
0954724
update cairobook ingester using LLM-summarized file
enitrat Jul 22, 2025
0220634
add missing agent keys
enitrat Jul 22, 2025
398e69e
refactor: remove async dependencies and optimize RAG pipeline
enitrat Jul 24, 2025
ec040d6
feat: add retry logic for AdapterParseError in generation program
enitrat Jul 24, 2025
7220d73
feat(eval): add compilation error on initial prompt
enitrat Jul 24, 2025
07be2ca
fix types
enitrat Jul 25, 2025
2fadb3a
fix typechecks
enitrat Jul 25, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/trunk-check.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,11 @@ jobs:
- name: Checkout
uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v6

- name: Trunk Code Quality
uses: trunk-io/trunk-action@v1

- name: Ty Check
run: uv run ty check
9 changes: 7 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,13 @@ packages/**/dist

.trunk
!.trunk/trunk.yaml
!.trunk/configs
!.trunk/.gitignore

starklings/
debug/
debug/

fixtures/runner_crate/target

.snfoundry_cache

python/starklings_results/
2 changes: 2 additions & 0 deletions .trunk/configs/.markdownlint.yaml
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
# Prettier friendly markdownlint config (all formatting rules disabled)
extends: markdownlint/style/prettier

MD024: false
1 change: 1 addition & 0 deletions .trunk/trunk.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ runtimes:
# This is the section where you manage your linters. (https://docs.trunk.io/check/configuration)
lint:
enabled:
- ruff@0.12.3
- actionlint@1.7.7
- git-diff-check
- hadolint@2.12.1-beta
Expand Down
307 changes: 108 additions & 199 deletions README.md

Large diffs are not rendered by default.

63 changes: 63 additions & 0 deletions README.old.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Legacy TypeScript Backend Instructions

**Note:** These instructions are for the original TypeScript backend, which has been superseded by the Python implementation. The Python backend is now the recommended and default service. Use these instructions only if you need to run the legacy service for a specific reason.

## Installation (TypeScript)

1. **Prerequisites**: Ensure Docker is installed and running.

2. **Clone the Repository**:

```bash
git clone https://github.com/KasarLabs/cairo-coder.git
cd cairo-coder
```

3. **Install Dependencies**:

```bash
pnpm install
```

4. **Configure Backend (`packages/agents/config.toml`)**:
Inside the `packages/agents` package, copy `sample.config.toml` to `config.toml`. Fill in your OpenAI or Gemini API keys.

5. **Configure PostgreSQL Database**:

**a. Database Container Initialization (`.env` file):**
Create a `.env` file in the root directory with the following PostgreSQL configuration:

```toml
POSTGRES_USER="YOUR_POSTGRES_USER"
POSTGRES_PASSWORD="YOUR_POSTGRES_PASSWORD"
POSTGRES_DB="YOUR_POSTGRES_DB"
```

**b. Application Connection Settings (`config.toml` file):**
In `packages/agents/config.toml`, configure the database connection section to match the `.env` file:

```toml
[VECTOR_DB]
POSTGRES_USER="YOUR_POSTGRES_USER"
POSTGRES_PASSWORD="YOUR_POSTGRES_PASSWORD"
POSTGRES_DB="YOUR_POSTGRES_DB"
POSTGRES_HOST="postgres"
POSTGRES_PORT="5432"
```

6. **Configure LangSmith (Optional)**:
Create a `.env` file in `packages/backend` with your LangSmith credentials. See the main `README.md` for more details on the variables.

7. **Run the Application**:
```bash
docker compose up postgres backend
```
The API will be available at `http://localhost:3001/v1/chat/completions`.

## Running the Ingester (TypeScript)

After you have the main application running, run the ingester to process and embed documentation from various sources.

```bash
docker compose --profile ingester up
```
44 changes: 25 additions & 19 deletions backend.dockerfile
Original file line number Diff line number Diff line change
@@ -1,26 +1,32 @@
FROM node:20 AS base
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable
FROM python:3.12-slim-bookworm

WORKDIR /app
# Install UV
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/

# Copy root workspace files
COPY pnpm-workspace.yaml ./
COPY package.json ./
COPY pnpm-lock.yaml ./
COPY turbo.json ./
# Set working directory
WORKDIR /app

# Copy backend & agents packages
COPY packages/backend ./packages/backend
COPY packages/agents ./packages/agents
# Copy Python project files
COPY python/pyproject.toml python/uv.lock ./python/
COPY python/src ./python/src
COPY python/optimizers ./python/optimizers
COPY python/config.toml ./python/
COPY python/.env ./python/
COPY README.md ./python/

# Copy shared TypeScript config
COPY packages/typescript-config ./packages/typescript-config
# For psycopg2
RUN apt-get update && apt-get install -y --no-install-recommends \

Check warning on line 18 in backend.dockerfile

View workflow job for this annotation

GitHub Actions / Trunk Check

hadolint(DL3008)

[new] Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
libpq-dev \
gcc \
python3-dev \
&& rm -rf /var/lib/apt/lists/*

RUN mkdir /app/data
# Install dependencies using UV
WORKDIR /app/python
RUN uv sync --frozen

RUN pnpm install --frozen-lockfile
RUN pnpm install -g turbo
# Expose the port the app runs on
EXPOSE 3001

CMD ["turbo", "start"]
# Run the application
CMD ["uv", "run", "cairo-coder"]
25 changes: 25 additions & 0 deletions backend.old.dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
FROM node:20 AS base
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable

WORKDIR /app

# Copy root workspace files
COPY pnpm-workspace.yaml ./
COPY package.json ./
COPY pnpm-lock.yaml ./
COPY turbo.json ./

# Copy backend & agents packages
COPY packages/backend ./packages/backend
COPY packages/agents ./packages/agents

# Copy shared TypeScript config
COPY packages/typescript-config ./packages/typescript-config

RUN mkdir /app/data && \
pnpm install --frozen-lockfile && \
pnpm install -g turbo

CMD ["turbo", "start"]
1 change: 1 addition & 0 deletions fixtures/runner_crate/.tool-versions
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
scarb nightly-2025-07-16
145 changes: 145 additions & 0 deletions fixtures/runner_crate/Scarb.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
# Code generated by scarb DO NOT EDIT.
version = 1

[[package]]
name = "openzeppelin"
version = "2.0.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:5e4fdecc957cfca7854d95912dc72d9f725517c063b116512900900add29fd77"
dependencies = [
"openzeppelin_access",
"openzeppelin_account",
"openzeppelin_finance",
"openzeppelin_governance",
"openzeppelin_introspection",
"openzeppelin_merkle_tree",
"openzeppelin_presets",
"openzeppelin_security",
"openzeppelin_token",
"openzeppelin_upgrades",
"openzeppelin_utils",
]

[[package]]
name = "openzeppelin_access"
version = "2.0.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:511681dd26d814ee2bc996d44ff8cb4aaa5ae9d14272130def7eb901cf004850"
dependencies = [
"openzeppelin_introspection",
]

[[package]]
name = "openzeppelin_account"
version = "2.0.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:fb3381c50d68b028d3801feb43df378e2bd62137b6884844f8f60aefe796188b"
dependencies = [
"openzeppelin_introspection",
"openzeppelin_utils",
]

[[package]]
name = "openzeppelin_finance"
version = "2.0.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:e9456ef69502a87c4c99bf50145351b50950f8b11244847d92935c466c4ba787"
dependencies = [
"openzeppelin_access",
"openzeppelin_token",
]

[[package]]
name = "openzeppelin_governance"
version = "2.0.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:056e6d6f3d48193b53f06283884f8a9675f986fc85425f6a40e8c1aeb3b3ecfa"
dependencies = [
"openzeppelin_access",
"openzeppelin_account",
"openzeppelin_introspection",
"openzeppelin_token",
"openzeppelin_utils",
]

[[package]]
name = "openzeppelin_introspection"
version = "2.0.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:87773ed6cd2318f169283ecbbb161890d1996260a80302d81ec45b70ee5e54c1"

[[package]]
name = "openzeppelin_merkle_tree"
version = "2.0.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:47f80c9ce59557774243214f8e75c5e866f30f3d8daa755855f6ffd01c89ca89"

[[package]]
name = "openzeppelin_presets"
version = "2.0.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:36c761ee923f1dc0887c0eab8c224b49ac242dbfe9163fbb0b08562042ab3d98"
dependencies = [
"openzeppelin_access",
"openzeppelin_account",
"openzeppelin_finance",
"openzeppelin_introspection",
"openzeppelin_token",
"openzeppelin_upgrades",
"openzeppelin_utils",
]

[[package]]
name = "openzeppelin_security"
version = "2.0.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:902932ec296c2f400e0ac7c579edeaafd6067b6ce6d9854c1191de28e396ffe3"

[[package]]
name = "openzeppelin_token"
version = "2.0.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:6fe61f63b5a6706018265fb7373b6e5bd3ff829bdc760b2b90296b1e708d180c"
dependencies = [
"openzeppelin_access",
"openzeppelin_account",
"openzeppelin_introspection",
"openzeppelin_utils",
]

[[package]]
name = "openzeppelin_upgrades"
version = "2.0.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:560d57a9c3f3ec5a476e82fec8963c93c8df63a4ff9ff134f64ab8383bde3c61"

[[package]]
name = "openzeppelin_utils"
version = "2.0.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:bf799c794139837f397975ffdf6a7ed5032d198bbf70e87a8f44f144a9dfc505"

[[package]]
name = "runner_crate"
version = "0.1.0"
dependencies = [
"openzeppelin",
"openzeppelin_access",
"openzeppelin_token",
"snforge_std",
]

[[package]]
name = "snforge_scarb_plugin"
version = "0.44.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:ec8c7637b33392a53153c1e5b87a4617ddcb1981951b233ea043cad5136697e2"

[[package]]
name = "snforge_std"
version = "0.44.0"
source = "registry+https://scarbs.xyz/"
checksum = "sha256:d4affedfb90715b1ac417b915c0a63377ae6dd69432040e5d933130d65114702"
dependencies = [
"snforge_scarb_plugin",
]
27 changes: 27 additions & 0 deletions fixtures/runner_crate/Scarb.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[package]
name = "runner_crate"
version = "0.1.0"
edition = "2024_07"

# See more keys and their definitions at https://docs.swmansion.com/scarb/docs/reference/manifest.html

[scripts]
test = "snforge test"

[tool.scarb]
allow-prebuilt-plugins = ["snforge_std"]

# Core Starknet and OpenZeppelin dependencies
[dependencies]
starknet = "2.11.4"
openzeppelin = "2.0.0"
openzeppelin_token = "2.0.0"
openzeppelin_access = "2.0.0"

[dev-dependencies]
snforge_std = "0.44.0"
assert_macros = "2.11.4"

# Starknet contract compilation target
[[target.starknet-contract]]
sierra = true
Empty file.
3 changes: 3 additions & 0 deletions ingester.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ COPY packages/backend ./packages/backend
COPY packages/ingester ./packages/ingester
COPY packages/agents ./packages/agents

# Copy ingester files generated from python
COPY python/scripts/summarizer/generated ./python/scripts/summarizer/generated

# Copy shared TypeScript config
COPY packages/typescript-config ./packages/typescript-config

Expand Down
Loading
Loading