Shimmy boilerplate

.github/workflows/production-deploy.yml

@@ -31,7 +31,7 @@ jobs:
       environment: "production"
       version-bump: ${{ inputs.version-bump }}
       branch: ${{ inputs.branch }}
-      deployed-environment-variables: '[\"OPENAI_API_KEY\",\"OPENAI_MODEL\",\"GOOGLE_AI_API_KEY\",\"GOOGLE_AI_MODEL\"]'
+      deployed-environment-variables: '[\"OPENAI_API_KEY\",\"OPENAI_MODEL\",\"GOOGLE_AI_API_KEY\",\"GOOGLE_AI_MODEL\",\"OPENROUTER_API_KEY\",\"OPENROUTER_MODEL\",\"OPENROUTER_BASE_URL\"]'
     secrets:
       aws-key-id: ${{ secrets.LAMBDA_CONTAINER_PIPELINE_AWS_ID }}
       aws-secret-key: ${{ secrets.LAMBDA_CONTAINER_PIPELINE_AWS_SECRET }}

.github/workflows/staging-deploy.yml

@@ -23,6 +23,9 @@ jobs:
       OPENAI_MODEL: ${{ vars.OPENAI_MODEL }}
       GOOGLE_AI_API_KEY: ${{ secrets.GOOGLE_AI_API_KEY }}
       GOOGLE_AI_MODEL: ${{ vars.GOOGLE_AI_MODEL }}
+      OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
+      OPENROUTER_MODEL: ${{ vars.OPENROUTER_MODEL }}
+      OPENROUTER_BASE_URL: ${{ vars.OPENROUTER_BASE_URL }}
     steps:
       - name: Checkout Code
         uses: actions/checkout@v4
@@ -67,7 +70,7 @@ jobs:
     with:
       template-repository-name: "lambda-feedback/chat-function-boilerplate"
       environment: "staging"
-      deployed-environment-variables: '[\"OPENAI_API_KEY\",\"OPENAI_MODEL\",\"GOOGLE_AI_API_KEY\",\"GOOGLE_AI_MODEL\"]'
+      deployed-environment-variables: '[\"OPENAI_API_KEY\",\"OPENAI_MODEL\",\"GOOGLE_AI_API_KEY\",\"GOOGLE_AI_MODEL\",\"OPENROUTER_API_KEY\",\"OPENROUTER_MODEL\",\"OPENROUTER_BASE_URL\"]'
     secrets:
       aws-key-id: ${{ secrets.LAMBDA_CONTAINER_PIPELINE_AWS_ID }}
       aws-secret-key: ${{ secrets.LAMBDA_CONTAINER_PIPELINE_AWS_SECRET }}

.github/workflows/test-lint.yml

@@ -21,6 +21,9 @@ jobs:
       OPENAI_MODEL: ${{ vars.OPENAI_MODEL }}
       GOOGLE_AI_API_KEY: ${{ secrets.GOOGLE_AI_API_KEY }}
       GOOGLE_AI_MODEL: ${{ vars.GOOGLE_AI_MODEL }}
+      OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
+      OPENROUTER_MODEL: ${{ vars.OPENROUTER_MODEL }}
+      OPENROUTER_BASE_URL: ${{ vars.OPENROUTER_BASE_URL }}
     steps:
       - name: Checkout
         uses: actions/checkout@v4

AGENTS.md

@@ -4,13 +4,13 @@ This file provides guidance to AI agents when working with code in this reposito
 
 ## Project Overview
 
-This is a chat function connecting students to an AI educational chatbot that is integrated with the **Lambda-Feedback** educational platform. It deploys as an AWS Lambda function (containerized via Docker) that receives student chat messages with educational context and returns LLM-powered chatbot responses. Incoming requests follow the [muEd API](https://mued.org/) schema (`context`, `user`, `messages`).
+This is a chat function connecting students to an AI educational chatbot that is integrated with the **Lambda-Feedback** educational platform. It's containerized via Docker and deployed behind [shimmy](https://github.com/lambda-feedback/shimmy), a shim that spawns this function as a persistent JSON-RPC worker process and exposes it as the muEd `/chat` / `/chat/health` HTTP API (both locally and as an AWS Lambda container). It receives student chat messages with educational context and returns LLM-powered chatbot responses. Incoming requests follow the [muEd API](https://mued.org/) schema (`context`, `user`, `messages`).
 
 ## Commands
 
 **Testing:**
 ```bash
-pytest                              # Run all unit tests
+PYTHONPATH=. pytest                 # Run all unit tests (CI sets PYTHONPATH=. too)
 python tests/manual_agent_run.py   # Test agent locally with example inputs
 python tests/manual_agent_requests.py  # Test running Docker container
 ```
@@ -23,39 +23,45 @@ docker run --env-file .env -p 8080:8080 llm_chat
 
 **Manual API test (while Docker is running):**
 ```bash
-curl -X POST http://localhost:8080/2015-03-31/functions/function/invocations \
+curl -X POST http://localhost:8080/chat \
   -H 'Content-Type: application/json' \
-  -d '{"body":"{\"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}]}"}'
+  -H 'X-Api-Version: 0.1.0' \
+  -d '{"messages": [{"role": "USER", "content": "hi"}]}'
+
+curl http://localhost:8080/chat/health -H 'X-Api-Version: 0.1.0'
 ```
 
 **Run a single test:**
 ```bash
 pytest tests/test_module.py        # Run specific test file
-pytest tests/test_index.py::test_function_name  # Run specific test
+pytest tests/test_module.py::TestChatModuleFunction::test_response_format  # Run specific test
 ```
 
 ## Architecture
 
 ### Request Flow
 
 ```
-Lambda event → index.py (handler)
-  → validates via lf_toolkit ChatRequest schema
-  → src/module.py (chat_module)
-    → extracts muEd API context (messages, conversationId, question context, user type)
-    → parses educational context to prompt text via src/agent/context.py
-    → src/agent/agent.py (BaseAgent / LangGraph)
-      → routes to call_llm or summarize_conversation node
-      → calls LLM provider (OpenAI / Google / Azure / Ollama)
-  → returns ChatResponse (output, summary, conversationalStyle, processingTime)
+shimmy (shim, container entrypoint)
+  → spawns index.py as a persistent worker subprocess (lf_toolkit RPC server)
+  → forwards POST /chat / GET /chat/health as JSON-RPC "chat" / "chat/health" calls
+  → index.py registers src/module.py's chat_module / chat_health_module as handlers
+    → lf_toolkit validates the request body against the muEd ChatRequest schema
+    → src/module.py (chat_module)
+      → extracts muEd API context (messages, conversationId, question context, user type)
+      → parses educational context to prompt text via src/agent/context.py
+      → src/agent/agent.py (BaseAgent / LangGraph)
+        → routes to call_llm or summarize_conversation node
+        → calls LLM provider (OpenAI / Google / Azure / Ollama)
+    → returns ChatResponse (output, summary, conversationalStyle, processingTime)
 ```
 
 ### Key Files
 
 | File | Role |
 |------|------|
-| `index.py` | AWS Lambda entry point; parses event body, validates schema |
-| `src/module.py` | Transforms muEd API request → invokes agent → builds ChatResponse |
+| `index.py` | Worker entrypoint; registers `chat_module`/`chat_health_module` with `lf_toolkit`'s RPC server (`create_server()` + `run()`) |
+| `src/module.py` | Transforms muEd API request → invokes agent → builds ChatResponse; also exposes `chat_health_module()` |
 | `src/agent/agent.py` | LangGraph stateful graph; manages message history and summarization |
 | `src/agent/prompts.py` | System prompts for tutor behavior, summarization, style detection |
 | `src/agent/llm_factory.py` | Factory classes for each LLM provider (OpenAI, Google, Azure, Ollama) |

CLAUDE.md

@@ -4,13 +4,13 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## Project Overview
 
-This is a chat function connecting students to an AI educational chatbot that is integrated with the **Lambda-Feedback** educational platform. It deploys as an AWS Lambda function (containerized via Docker) that receives student chat messages with educational context and returns LLM-powered chatbot responses. Incoming requests follow the [muEd API](https://mued.org/) schema (`context`, `user`, `messages`).
+This is a chat function connecting students to an AI educational chatbot that is integrated with the **Lambda-Feedback** educational platform. It's containerized via Docker and deployed behind [shimmy](https://github.com/lambda-feedback/shimmy), a shim that spawns this function as a persistent JSON-RPC worker process and exposes it as the muEd `/chat` / `/chat/health` HTTP API (both locally and as an AWS Lambda container). It receives student chat messages with educational context and returns LLM-powered chatbot responses. Incoming requests follow the [muEd API](https://mued.org/) schema (`context`, `user`, `messages`).
 
 ## Commands
 
 **Testing:**
 ```bash
-pytest                              # Run all unit tests
+PYTHONPATH=. pytest                 # Run all unit tests (CI sets PYTHONPATH=. too)
 python tests/manual_agent_run.py   # Test agent locally with example inputs
 python tests/manual_agent_requests.py  # Test running Docker container
 ```
@@ -23,39 +23,45 @@ docker run --env-file .env -p 8080:8080 llm_chat
 
 **Manual API test (while Docker is running):**
 ```bash
-curl -X POST http://localhost:8080/2015-03-31/functions/function/invocations \
+curl -X POST http://localhost:8080/chat \
   -H 'Content-Type: application/json' \
-  -d '{"body":"{\"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}]}"}'
+  -H 'X-Api-Version: 0.1.0' \
+  -d '{"messages": [{"role": "USER", "content": "hi"}]}'
+
+curl http://localhost:8080/chat/health -H 'X-Api-Version: 0.1.0'
 ```
 
 **Run a single test:**
 ```bash
 pytest tests/test_module.py        # Run specific test file
-pytest tests/test_index.py::test_function_name  # Run specific test
+pytest tests/test_module.py::TestChatModuleFunction::test_response_format  # Run specific test
 ```
 
 ## Architecture
 
 ### Request Flow
 
 ```
-Lambda event → index.py (handler)
-  → validates via lf_toolkit ChatRequest schema
-  → src/module.py (chat_module)
-    → extracts muEd API context (messages, conversationId, question context, user type)
-    → parses educational context to prompt text via src/agent/context.py
-    → src/agent/agent.py (BaseAgent / LangGraph)
-      → routes to call_llm or summarize_conversation node
-      → calls LLM provider (OpenAI / Google / Azure / Ollama)
-  → returns ChatResponse (output, summary, conversationalStyle, processingTime)
+shimmy (shim, container entrypoint)
+  → spawns index.py as a persistent worker subprocess (lf_toolkit RPC server)
+  → forwards POST /chat / GET /chat/health as JSON-RPC "chat" / "chat/health" calls
+  → index.py registers src/module.py's chat_module / chat_health_module as handlers
+    → lf_toolkit validates the request body against the muEd ChatRequest schema
+    → src/module.py (chat_module)
+      → extracts muEd API context (messages, conversationId, question context, user type)
+      → parses educational context to prompt text via src/agent/context.py
+      → src/agent/agent.py (BaseAgent / LangGraph)
+        → routes to call_llm or summarize_conversation node
+        → calls LLM provider (OpenAI / Google / Azure / Ollama)
+    → returns ChatResponse (output, summary, conversationalStyle, processingTime)
 ```
 
 ### Key Files
 
 | File | Role |
 |------|------|
-| `index.py` | AWS Lambda entry point; parses event body, validates schema |
-| `src/module.py` | Transforms muEd API request → invokes agent → builds ChatResponse |
+| `index.py` | Worker entrypoint; registers `chat_module`/`chat_health_module` with `lf_toolkit`'s RPC server (`create_server()` + `run()`) |
+| `src/module.py` | Transforms muEd API request → invokes agent → builds ChatResponse; also exposes `chat_health_module()` |
 | `src/agent/agent.py` | LangGraph stateful graph; manages message history and summarization |
 | `src/agent/prompts.py` | System prompts for tutor behavior, summarization, style detection |
 | `src/agent/llm_factory.py` | Factory classes for each LLM provider (OpenAI, Google, Azure, Ollama) |

Dockerfile

@@ -1,18 +1,14 @@
-ARG PYTHON_VERSION=3.13
+ARG BASE_VERSION=python:3.12
 
-FROM public.ecr.aws/lambda/python:${PYTHON_VERSION}
+# evaluation-function-base's python image bundles the shimmy binary,
+# the Lambda RIE, and the entrypoint.sh that picks between them.
+FROM ghcr.io/lambda-feedback/evaluation-function-base/${BASE_VERSION}
 
-# Set working directory
-WORKDIR ${LAMBDA_TASK_ROOT}
+RUN apt-get update && apt-get install -y \
+      build-essential \
+    && rm -rf /var/lib/apt/lists/*
 
-RUN pip install --upgrade pip 
-RUN dnf install -y git \
-    && dnf install -y \
-      gcc \
-      gcc-c++ \
-      make \
-      python3-devel \
-    && dnf clean all
+RUN pip install --upgrade pip
 
 COPY requirements.txt .
 RUN pip install -r requirements.txt
@@ -27,5 +23,15 @@ COPY index.py .
 
 COPY tests ./tests
 
-# Set the Lambda function handler
-CMD ["index.handler"]
+# Command shimmy uses to start the chat function worker
+ENV FUNCTION_COMMAND="python"
+
+# Args to start the chat function worker with
+ENV FUNCTION_ARGS="index.py"
+
+# The transport to use for the RPC server
+ENV FUNCTION_RPC_TRANSPORT="ipc"
+
+ENV FUNCTION_WORKER_SEND_TIMEOUT="170s"
+
+ENV LOG_LEVEL="debug"

README.md

@@ -22,19 +22,15 @@ OPENAI_MODEL
 GOOGLE_AI_API_KEY
 GOOGLE_AI_MODEL
 ```
-
-> [!Note]
-> If you decide to use another endpoint such as Azure or Ollama or any other, please update the github workflow files to use the right secrets and variables for testing.
+> If you use OpenRouter:
 ```bash
-> If you use Azure-OpenAI:
-AZURE_OPENAI_API_KEY
-AZURE_OPENAI_ENDPOINT
-AZURE_OPENAI_API_VERSION
-AZURE_OPENAI_CHAT_DEPLOYMENT_NAME
-AZURE_OPENAI_EMBEDDING_3072_DEPLOYMENT
-AZURE_OPENAI_EMBEDDING_1536_DEPLOYMENT
-AZURE_OPENAI_EMBEDDING_3072_MODEL
-AZURE_OPENAI_EMBEDDING_1536_MODEL
+OPENROUTER_API_KEY
+OPENROUTER_MODEL
+OPENROUTER_BASE_URL
+```
+
+> [!NOTE]
+> If you decide to use other providers like Azure OpenAI or Ollama, you will need to update the workflow files and the `llm_factory.py` file to include the necessary environment variables for those providers.
 
 > For monitoring of the LLM calls (follow instructions on how to set up on langsmith online):
 LANGCHAIN_TRACING_V2
@@ -117,7 +113,6 @@ The agent uses **two separate LLM instances** — `self.llm` for chat responses
     ├── manual_agent_run.py               # allows testing of any LLM agent on a couple of example inputs
     ├── utils.py                          # shared test helpers
     ├── test_example_inputs.py            # pytests for the example input files
-    ├── test_index.py                     # pytests
     └── test_module.py                    # pytests
 ```
 
@@ -128,18 +123,18 @@ To test your function, you can run the unit tests, call the code directly throug
 
 ### Run Unit Tests
 
-You can run the unit tests using `pytest`.
+You can run the unit tests using `pytest`. Run it from the repository root with `PYTHONPATH=.` set (as CI does) so the `tests` and `src` packages resolve correctly:
 
 ```bash
-pytest
+PYTHONPATH=. pytest
 ```
 
 ### Run the Chat Script
 
-You can run the Python function itself. Make sure to have a main function in either `src/module.py` or `index.py`.
+You can run the Python function itself directly — `index.py` wires `chat_module`/`chat_health_module` into `lf_toolkit`'s RPC server, the same way shimmy invokes it inside the container. This requires the `EVAL_IO`/`EVAL_RPC_TRANSPORT` environment variables shimmy would normally set (see `lf_toolkit`'s docs), so prefer the Docker or `manual_agent_run.py` routes below for everyday testing.
 
 ```bash
-python src/module.py
+python index.py
 ```
 
 You can also use the `manual_agent_run.py` script to test the agents with example inputs from Lambda Feedback questions and synthetic conversations.
@@ -171,33 +166,41 @@ docker run -e OPENAI_API_KEY={your key} -e OPENAI_MODEL={your LLM model name} -p
 docker run --env-file .env -it --name my-lambda-container -p 8080:8080 llm_chat
 ```
 
-This will start the chat function and expose it on port `8080` and it will be open to be curl:
+This starts shimmy (the [Lambda Feedback shim](https://github.com/lambda-feedback/shimmy)) as the container's entrypoint, which spawns this function as a worker subprocess and exposes it on port `8080` as the muEd chat API:
 
 ```bash
-curl --location 'http://localhost:8080/2015-03-31/functions/function/invocations' \
+curl --location 'http://localhost:8080/chat' \
 --header 'Content-Type: application/json' \
---data '{"body":"{\"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}]}"}'
+--header 'X-Api-Version: 0.1.0' \
+--data '{"messages": [{"role": "USER", "content": "hi"}]}'
+```
+
+Health check:
+
+```bash
+curl --location 'http://localhost:8080/chat/health' \
+--header 'X-Api-Version: 0.1.0'
 ```
 
 #### Call Docker Container
 ##### A. Call Docker with Python Requests
 
-In the `tests/` folder you can find the `manual_agent_requests.py` script that calls the POST URL of the running docker container. It reads any kind of input files with the expected schema. You can use this to test your curl calls of the chatbot.
+In the `tests/` folder you can find the `manual_agent_requests.py` script that calls the `/chat` and `/chat/health` routes of the running docker container. It reads any kind of input files with the expected schema. You can use this to test your curl calls of the chatbot.
 
 ##### B. Call Docker Container through API request
 
 POST URL:
 
 ```bash
-http://localhost:8080/2015-03-31/functions/function/invocations
+http://localhost:8080/chat
 ```
 
-Per the [muEd `ChatRequest` schema](https://mued.org/), only `messages` is required; `conversationId`, `user`, `context`, and `configuration` are all optional.
+Per the [muEd `ChatRequest` schema](https://mued.org/), only `messages` is required; `conversationId`, `user`, `context`, and `configuration` are all optional. Requests may include an `X-Api-Version: 0.1.0` header.
 
-**Minimal request — only required components** (stringified within `body` for the AWS Lambda Runtime Interface Emulator):
+**Minimal request — only required components:**
 
 ```JSON
-{"body":"{\"messages\": [{\"role\": \"USER\", \"content\": \"hi\"}]}"}
+{"messages": [{"role": "USER", "content": "hi"}]}
 ```
 
 **Full request as Lambda Feedback sends it** — all optional fields populated: