Develop TJBot Recipes

This guide explains how to create a new TJBot recipe using TJBot's core library.

Create a new recipe

The TJBot CLI can be used to scaffold a new recipe.

tjbot create my_new_recipe

The CLI will prompt you for various types of information about your recipe, including its display name, description, and which language should be used (TypeScript or Python).

TJBot uses the mise task runner to manage dependency installation and running the recipe. You can run your recipe using the command mise run start inside your recipe's directory.

Once your recipe has been created, refer to TJBot's API docs for documentation on TJBot's SDKs.

Define recipe-specific configuration

Recipe-specific configuration settings should be kept in a file named recipe.toml inside your recipe's directory. The TJBot library provides a static convenience method for loading recipe-specific config from this file:

// read recipe-specific config
const config: Record<String, unknown> = TJBot.getRecipeConfig();

Instantiate the TJBot API

The primary entry point to the TJBot API is the TJBot class. Since the TJBot libary manages communications with several hardware devices, it uses a singleton pattern to ensure only one instance of this class is created at runtime, using the .getInstance() method.

Additionally, because many hardware operations are asynchronous, the .initialize() method must be called to initialize the hardware. Here is an example of how to initialize the TJBot singleton for a recipe that requires an led, microphone, and speaker:

import TJBot from 'tjbot-ce';

const tj = await TJBot.getInstance().initialize({
    hardware: {
        led: true,
        microphone: true,
        speaker: true,
        camera: false,
        servo: false,
    },
});

Tip

The configuration object passed to .initialize() overrides any of the TJBot settings defined in tjbot.toml.

Core API methods

TJBot's core capabilities -- listening, seeing, speaking, shining, waving -- are implemented through various methods in the TJBot class.

• listen() listens for audio on the microphone and transcribes it to text via speech-to-text.
• speak(text) speaks the given text via text-to-speech.
• shine(color) and pulse(color, duration) change the color of the LED.
• wave(), raiseArm(), lowerArm(), armBack() control servo movement.
• see() captures an image from the camera and returns it as an in-memory buffer.
• look(path?) captures an image from the camera and saves it to the specified file path (or a temporary path if no path is specified).
• detectObjects(image) performs object detection on the given image.
• detectFaces(image) detects faces within the given image.
• classifyImage(image) performs classification on the given image.
• describeImage(image) produces a natural language description of the given image (only enabled when using the Azure vision backend).
• sleep(seconds) pauses the recipe for the given duration.

Several utility methods can also be useful when writing a recipe:

• TJBot.getRecipeConfig(path?) loads recipe-specific settings from recipe.toml.
• getLocalModels(modelType?, installedOnly?) lists the AI models that can be (or have been) downloaded.
• setLogLevel(level) sets the logging level for console messages.

Tip

Check out TJBot's full API documentation for a more complete description of API methods and types.