Teleprompter is a prompt management system for Large Language Model (LLM) running on Cloudflare and Cloudflare Workers. It provides versioning, metadata tracking, and runtime editing and updating of prompts.
Teleprompter has three components that work together.
- Worker: The worker that manages the prompts and pushes updates
- CLI Tool: A command-line tool for viewing and managing prompts
- SDK: A Typescript SDK for fetching your prompts
- Every prompt is a Mustache template.
- Prompts are append only, there's no such thing as an update just a full replacement.
- Prompts are versioned using the UNIX timestamp when they were created
- Prompts can be rolled back to previous versions
- Prompts are updated and edited by the Teleprompter worker. You can talk to it with the CLI.
- Updates are sent to your applications via Cloudflare Queues
- Your application consumes the update and stashes the result in it's local KV namespace
- You can either fetch it out or render the template directly
- Fin
See also: Theory of Operation for the full architecture and update flow.
interface Prompt {
id: TEXT // a unique ID for the prompt
text: TEXT // the text of the Mustache template
version: INTEGER // this is always just the UNIX timestamp of when this prom,pt was edited
namespace: STRING // the name of the QUEUE to send updates to
}I want you to look at some text and decide:
- Is this code written in a programming language?
- What language is it written in?
- What is the purpose of the code?
Write a short one sentence description of the nature and purpose of the code including the programming language it was written in.
You are narrating an article to a person that cannot see the code so begin the description by explaining that "The article shows a code sample" then explain the purpose of the code example.
Here are some examples of code and good descriptions.
Example 1:
Code:
interface CodeSummary {
code: boolean
language: string
summary: string
}
Description: The article shows a TypeScript interface definition for a CodeSummary object.
Example 2:
Code:
func toDefaultValueType(dataType string, s string) (any, error) {
switch SqliteType(dataType) {
case SqliteNull:
return nil, nil
case SqliteInteger:
return strconv.ParseInt(s, 10, 64)
case SqliteReal:
return strconv.ParseFloat(s, 64)
case SqliteText:
return s, nil
case SqliteBlob:
return []byte(s), nil
case SqliteBoolean:
i, err := strconv.ParseInt(s, 10, 32)
if err != nil {
return false, err
}
return i != 0, nil
}
return nil, fmt.Errorf("unknown sqlite type: %s", dataType)
}
Description: The article shows a Go function that converts a string to a value of the correct type based on a given SQLite data type.
Here is the code to analyze:
{{code}}- Node.js v16 or higher
- npm or yarn
- Cloudflare account (for deploying/workers)
- Wrangler CLI (
npm install -g wrangler)
Teleprompter has no authentication system of its own. It uses Cloudflare Warp for access control. The authentication token retrieved from Cloudflare is stored in $HOME/.teleprompter/token. The token file permissions are set to 0600 to keep it private but that's all there is for security.
-
Clone the repository:
git clone https://github.com/britt/teleprompter.git cd teleprompter -
Install dependencies:
npm install # or yarn install
- Copy and edit
wrangler.example.toml(or similar) towrangler.tomland adjust for your Cloudflare account and your desired settings. - Bindings for Durable Objects and Queues should be declared in your
wrangler.toml.
You can use the Wrangler CLI to run the Cloudflare Worker locally:
wrangler devThis will start the development server. The prompts Durable Object will be available for testing at local endpoints (e.g., http://localhost:8787/prompts).
npm testor with yarn:
yarn testTo build the documentation using TypeDoc:
npm run docsThe generated documentation will be found in the doc directory.
After configuring your credentials and wrangler.toml, deploy using:
wrangler deployFor day-to-day prompt management, use the Teleprompter CLI Tool. You can install and use it by following instructions at that repository, allowing you to create, edit, and roll back prompts in production or during development.
COMING SOON (Add examples and instructions for using Teleprompter)
We welcome contributions to Teleprompter! Here's how you can help:
- Fork the repository
- Create a new branch for your feature:
git checkout -b feature/your-feature-name - Make your changes
- Write or update tests as needed
- Run tests to ensure everything works
- Commit your changes:
git commit -m "Add some feature" - Push to your fork:
git push origin feature/your-feature-name - Open a Pull Request
- Write thorough and comprehensive tests for your changes
- Include both positive and negative test cases
- Test edge cases and error conditions
- Try to have better test coverage than I did when I first wrote it
- Consider integration tests where appropriate
- Add comments for complex logic
- Update documentation for any changed functionality
- Keep commits focused and atomic
This project is licensed under the MIT License. See the LICENSE file for details.
- Update the README.md with details of changes if needed
- Update the version numbers following Semantic Versioning
- Your PR will be reviewed by the maintainer. This is not guaranteed (or even likely) to be timely since it's just me.
- Once approved, your PR will be merged
When filing an issue, please include:
- A clear description of the problem
- Steps to reproduce
- Expected vs actual behavior
- Version information
- Any relevant logs or error messages
- Be respectful and inclusive
- Keep discussions constructive
- Focus on the technical merits
- Follow the maintainer's decisions
This project is licensed under the MIT License. See the LICENSE file for details.
This documentation was co-authored by doc.holiday.