diff --git a/docs/_guides.yaml b/docs/_guides.yaml index c51b9eac27..7842638e8e 100644 --- a/docs/_guides.yaml +++ b/docs/_guides.yaml @@ -29,9 +29,7 @@ toc: path: /docs/genkit/models - title: Creating flows path: /docs/genkit/flows - - title: Prompting models - path: /docs/genkit/prompts - - title: Managing prompts + - title: Managing prompts with Dotprompt path: /docs/genkit/dotprompt - title: Tool calling path: /docs/genkit/tool-calling diff --git a/docs/dotprompt.md b/docs/dotprompt.md index 706ec906e4..bbcb2d5245 100644 --- a/docs/dotprompt.md +++ b/docs/dotprompt.md @@ -1,72 +1,255 @@ # Managing prompts with Dotprompt -Firebase Genkit provides the Dotprompt plugin and text format to help you write -and organize your generative AI prompts. +Prompt engineering is the primary way that you, as an app developer, influence +the output of generative AI models. For example, when using LLMs, you can craft +prompts that influence the tone, format, length, and other characteristics of +the models' responses. + +The way you write these prompts will depend on the model you're using; a prompt +written for one model might not perform well when used with another model. +Similarly, the model parameters you set (temperature, top-k, and so on) will +also affect output differently depending on the model. + +Getting all three of these factors—the model, the model parameters, and +the prompt—working together to produce the output you want is rarely a +trivial process and often involves substantial iteration and experimentation. +Genkit provides a library and file format called Dotprompt, that aims to make +this iteration faster and more convenient. + +Dotprompt is designed around the premise that **prompts are code**. You define +your prompts along with the models and model parameters they're intended for +separately from your application code. Then, you (or, perhaps someone not even +involved with writing application code) can rapidly iterate on the prompts and +model parameters using the Genkit Developer UI. Once your prompts are working +the way you want, you can import them into your application and run them using +Genkit. + +Your prompt definitions each go in a file with a `.prompt` extension. Here's an +example of what these files look like: + +```handlebars +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/prompts/ex01.prompt" %} +``` + +The portion in the triple-dashes is YAML front matter, similar to the front +matter format used by GitHub Markdown and Jekyll; the rest of the file is the +prompt, which can optionally use +[Handlebars](https://handlebarsjs.com/guide/){:.external} templates. The +following sections will go into more detail about each of the parts that make a +`.prompt` file and how to use them. + +## Before you begin + +Before reading this page, you should be familiar with the content covered on the +[Generating content with AI models](models) page. + +If you want to run the code examples on this page, first complete the steps in +the [Getting started](get-started) guide. All of the examples assume that you +have already installed Genkit as a dependency in your project. + +## Creating prompt files -Dotprompt is designed around the premise that _prompts are code_. You write and -maintain your prompts in specially-formatted files called dotprompt files, track -changes to them using the same version control system that you use for your -code, and you deploy them along with the code that calls your generative AI -models. +Although Dotprompt provides several [different ways](#alternatives) to create +and load prompts, it's optimized for projects that organize their prompts as +`.prompt` files within a single directory (or subdirectories thereof). This +section shows you how to create and load prompts using this recommended setup. -To use Dotprompt, first create a `prompts` directory in your project root and -then create a `.prompt` file in that directory. Here's a simple example you -might call `greeting.prompt`: +### Creating a prompt directory + +The Dotprompt library expects to find your prompts in a directory at your +project root and automatically loads any prompts it finds there. By default, +this directory is named `prompts`. For example, using the default directory +name, your project structure might look something like this: ```none ---- -model: vertexai/gemini-1.5-flash -config: - temperature: 0.9 -input: - schema: - location: string - style?: string - name?: string - default: - location: a restaurant ---- +your-project/ +├── lib/ +├── node_modules/ +├── prompts/ +│ └── hello.prompt +├── src/ +├── package-lock.json +├── package.json +└── tsconfig.json +``` -You are the world's most welcoming AI assistant and are currently working at {{location}}. +If you want to use a different directory, you can specify it when you configure +Genkit: -Greet a guest{{#if name}} named {{name}}{{/if}}{{#if style}} in the style of {{style}}{{/if}}. +```ts +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="promptDir" adjust_indentation="auto" %} ``` -To use this prompt, install the `dotprompt` plugin, and import the `promptRef` function from -the `@genkit-ai/dotprompt` library: +### Creating a prompt file + +There are two ways to create a `.prompt` file: using a text editor, or with the +developer UI. + +#### Using a text editor + +If you want to create a prompt file using a text editor, create a text file with +the `.prompt` extension in your prompts directory: for example, +`prompts/hello.prompt`. + +Here is a minimal example of a prompt file: + +```handlebars +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/prompts/ex02.prompt" %} +``` + +The portion in the dashes is YAML front matter, similar to the front matter +format used by GitHub markdown and Jekyll; the rest of the file is the prompt, +which can optionally use Handlebars templates. The front matter section is +optional, but most prompt files will at least contain metadata specifying a +model. The remainder of this page shows you how to go beyond this, and make use +of Dotprompt's features in your prompt files. + +#### Using the developer UI + +You can also create a prompt file using the model runner in the developer UI. +Start with application code that imports the Genkit library and configures it to +use the model plugin you're interested in. For example: ```ts -import { dotprompt, promptRef } from '@genkit-ai/dotprompt'; +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/minimal.ts" region_tag="mini" adjust_indentation="auto" %} +``` + +It's okay if the file contains other code, but the above is all that's required. + +Load the developer UI in the same project: + +```posix-terminal +export GENKIT_ENV=dev + +npx genkit ui:start + +npx tsx your-code.ts +``` + +In the Models section, choose the model you want to use from the list of models +provided by the plugin. + +![Genkit developer UI model runner](resources/developer_ui_model_runner.png) + +Then, experiment with the prompt and configuration until you get results you're +happy with. When you're ready, press the Export button and save the file to your +prompts directory. + +## Running prompts -configureGenkit({ plugins: [dotprompt()] }); +After you've created prompt files, you can run them from your application code, +or using the tooling provided by Genkit. Regardless of how you want to run your +prompts, first start with application code that imports the Genkit library and +the model plugins you're interested in. For example: + +```ts +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/minimal.ts" region_tag="mini" adjust_indentation="auto" %} ``` -Then, load the prompt using `promptRef('file_name')`: +It's okay if the file contains other code, but the above is all that's required. +If you're storing your prompts in a directory other than the default, be sure to +specify it when you configure Genkit. + +### Run prompts from code + +To use a prompt, first load it using the `prompt('file_name')` method: ```ts -const greetingPrompt = promptRef('greeting'); +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="loadPrompt" adjust_indentation="auto" %} +``` -const result = await greetingPrompt.generate({ - input: { - location: 'the beach', - style: 'a fancy pirate', - }, -}); +Once loaded, you can call the prompt like a function: -console.log(result.text); +```ts +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="callPrompt" adjust_indentation="auto" %} ``` -Dotprompt's syntax is based on the [Handlebars](https://handlebarsjs.com/guide/) -templating language. You can use the `if`, `unless`, and `each` helpers to add -conditional portions to your prompt or iterate through structured content. The -file format utilizes YAML frontmatter to provide metadata for a prompt inline -with the template. +A callable prompt takes two optional parameters: the input to the prompt (see +the section below on [specifying input schemas](#schemas)), and a configuration +object, similar to that of the `generate()` method. For example: + +```ts +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="callPromptOpts" adjust_indentation="auto" %} +``` + +Any parameters you pass to the prompt call will override the same parameters +specified in the prompt file. + +See [Generate content with AI models](models) for descriptions of the available +options. -## Defining Input/Output Schemas +### Using the developer UI -Dotprompt includes a compact, YAML-optimized schema definition format called -Picoschema to make it easy to define the most important attributes of a schema -for LLM usage. Here's an example of a schema for an article: +As you're refining your app's prompts, you can run them in the Genkit developer +UI to quickly iterate on prompts and model configurations, independently from +your application code. + +Load the developer UI from your project directory: + +```posix-terminal +export GENKIT_ENV=dev + +npx genkit ui:start + +npx tsx your-code.ts +``` + +![Genkit developer UI prompt runner](resources/prompts-in-developer-ui.png) + +Once you've loaded prompts into the developer UI, you can run them with +different input values, and experiment with how changes to the prompt wording or +the configuration parameters affect the model output. When you're happy with the +result, you can click the **Export prompt** button to save the modified prompt +back into your project directory. + +## Model configuration + +In the front matter block of your prompt files, you can optionally specify model +configuration values for your prompt: + +```yaml +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/prompts/ex03.prompt" %} +``` + +These values map directly to the `config` parameter accepted by the callable +prompt: + +```ts +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="callPromptCfg" adjust_indentation="auto" %} +``` + +See [Generate content with AI models](models) for descriptions of the available +options. + +## Input and output schemas {:#schemas} + +You can specify input and output schemas for your prompt by defining them in the +front matter section: + +```handlebars +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/prompts/ex04.prompt" %} +``` + +These schemas are used in much the same way as those passed to a `generate()` +request or a flow definition. For example, the prompt defined above produces +structured output: + +```ts +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="outSchema" adjust_indentation="auto" %} +``` + +You have several options for defining schemas in a `.prompt` file: Dotprompt's +own schema definition format, Picoschema; standard JSON Schema; or, as +references to schemas defined in your application code. The following sections +describe each of these options in more detail. + +### Picoschema + +The schemas in the example above are defined in a format called Picoschema. +Picoschema is a compact, YAML-optimized schema definition format that makes it +easy to define the most important attributes of a schema for LLM usage. Here's a +longer example of a schema, which specifies the information an app might store +about an article: ```yaml schema: @@ -113,21 +296,24 @@ interface Article { /** arbitrary extra data */ extra?: any; /** wildcard field */ - [additionalField: string]: string; + } ``` -Picoschema supports scalar types `string`, `integer`, `number`, `boolean`, and `any`. -For objects, arrays, and enums they are denoted by a parenthetical after the field name. +Picoschema supports scalar types `string`, `integer`, `number`, `boolean`, and +`any`. Objects, arrays, and enums are denoted by a parenthetical after the field +name. + +Objects defined by Picoschema have all properties required unless denoted +optional by `?`, and do not allow additional properties. When a property is +marked as optional, it is also made nullable to provide more leniency for LLMs +to return null instead of omitting a field. -Objects defined by Picoschema have all properties as required unless denoted optional -by `?`, and do not allow additional properties. When a property is marked as optional, -it is also made nullable to provide more leniency for LLMs to return null instead of -omitting a field. +In an object definition, the special key `(*)` can be used to declare a +"wildcard" field definition. This will match any additional properties not +supplied by an explicit key. -In an object definition, the special key `(*)` can be used to declare a "wildcard" -field definition. This will match any additional properties not supplied by an -explicit key. +### JSON Schema Picoschema does not support many of the capabilities of full JSON schema. If you require more robust schemas, you may supply a JSON Schema instead: @@ -142,32 +328,26 @@ output: minimum: 20 ``` -### Leveraging Reusable Schemas +### Zod schemas defined in code -In addition to directly defining schemas in the `.prompt` file, you can reference -a schema registered with `defineSchema` by name. To register a schema: +In addition to directly defining schemas in the `.prompt` file, you can +reference a schema registered with `defineSchema()` by name. If you're using +TypeScript, this approach will let you take advantage of the language's static +type checking features when you work with prompts. -```ts -import { defineSchema } from '@genkit-ai/core'; -import { z } from 'zod'; +To register a schema: -const MySchema = defineSchema( - 'MySchema', - z.object({ - field1: z.string(), - field2: z.number(), - }) -); +```ts +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="MenuItemSchema" adjust_indentation="auto" %} ``` -Within your prompt, you can provide the name of the registered schema: +Within your prompt, provide the name of the registered schema: -```yaml -# myPrompt.prompt +```none --- -model: vertexai/gemini-1.5-flash +model: googleai/gemini-1.5-flash-latest output: - schema: MySchema + schema: MenuItemSchema --- ``` @@ -176,196 +356,114 @@ registered Zod schema. You can then utilize the schema to strongly type the output of a Dotprompt: ```ts -import { promptRef } from "@genkit-ai/dotprompt"; - -const myPrompt = promptRef("myPrompt"); - -const result = await myPrompt.generate({...}); - -// now strongly typed as MySchema -result.output; +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="outSchema2" adjust_indentation="auto" %} ``` -## Overriding Prompt Metadata +## Prompt templates -While `.prompt` files allow you to embed metadata such as model configuration in -the file itself, you can also override these values on a per-call basis: +The portion of a `.prompt` file that follows the front matter (if present) is +the prompt itself, which will be passed to the model. While this prompt could be +a simple text string, very often you will want to incorporate user input into +the prompt. To do so, you can specify your prompt using the +[Handlebars](https://handlebarsjs.com/guide/){:.external} templating language. +Prompt templates can include placeholders that refer to the values defined by +your prompt's input schema. -```ts -const result = await greetingPrompt.generate({ - model: 'vertexai/gemini-1.5-pro', - config: { - temperature: 1.0, - }, - input: { - location: 'the beach', - style: 'a fancy pirate', - }, -}); -``` - -## Structured output - -You can set the format and output schema of a prompt to coerce into JSON: - -```none ---- -model: vertexai/gemini-1.5-flash -input: - schema: - theme: string -output: - format: json - schema: - name: string - price: integer - ingredients(array): string ---- +You already saw this in action in the section on input and output schemas: -Generate a menu item that could be found at a {{theme}} themed restaurant. +```handlebars +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/prompts/ex03.prompt" %} ``` -When generating a prompt with structured output, use the `output()` helper to -retrieve and validate it: +In this example, the Handlebars expression, `{{theme}}`, +resolves to the value of the input's `theme` property when you run the +prompt. To pass input to the prompt, call the prompt as in the following +example: ```ts -const createMenuPrompt = promptRef('create_menu'); - -const menu = await createMenuPrompt.generate({ - input: { - theme: 'banana', - }, -}); - -console.log(menu.output); +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="inSchema" adjust_indentation="auto" %} ``` -Output conformance is achieved by inserting additional instructions into the -prompt. By default, it is appended to the end of the last message generated -by the prompt. You can manually reposition it using the `{{section "output"}}` -helper. - -```none -This is a prompt that manually positions output instructions. - -== Output Instructions - -{{section "output"}} +Note that because the input schema declared the `theme` property to be optional +and provided a default, you could have omitted the property, +and the prompt would have resolved using the default value. -== Other Instructions +Handlebars templates also support some limited logical constructs. For example, +as an alternative to providing a default, you could define the prompt using +Handlebars's `#if` helper: -This will come after the output instructions. +```handlebars +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/prompts/ex05.prompt" %} ``` -## Multi-message prompts +In this example, the prompt renders as "Invent a menu item for a restaurant" +when the `theme` property is unspecified. -By default, Dotprompt constructs a single message with a `"user"` role. Some -prompts are best expressed as a combination of multiple messages, such as a -system prompt. +See the [Handlebars +documentation](https://handlebarsjs.com/guide/builtin-helpers.html){:.external} +for information on all of the built-in logical helpers. -The `{{role}}` helper provides a simple way to construct multi-message prompts: +In addition to properties defined by your input schema, your templates can also +refer to values automatically defined by Genkit. The next few sections describe +these automatically-defined values and how you can use them. -```none ---- -model: vertexai/gemini-1.5-flash -input: - schema: - userQuestion: string ---- +### Multi-message prompts -{{role "system"}} -You are a helpful AI assistant that really loves to talk about food. Try to work -food items into all of your conversations. -{{role "user"}} -{{userQuestion}} -``` +By default, Dotprompt constructs a single message with a "user" role. +However, some prompts are best expressed as a combination of multiple messages, +such as a system prompt. -## Multi-Turn Prompts and History +The `{{role}}` helper provides a simple way to +construct multi-message prompts: -Dotprompt supports multi-turn prompts by passing the `history` option into the -`generate` method: - -```ts -const result = await multiTurnPrompt.generate({ - messages: [ - { role: 'user', content: [{ text: 'Hello.' }] }, - { role: 'model', content: [{ text: 'Hi there!' }] }, - ], -}); +```handlebars +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/prompts/ex06.prompt" %} ``` -By default, history will be inserted before the final message generated by -the prompt. However, you can manually position history using the `{{history}}` -helper: - -```none -{{role "system"}} -This is the system prompt. -{{history}} -{{role "user"}} -This is a user message. -{{role "model"}} -This is a model message. -{{role "user"}} -This is the final user message. -``` +### Multi-modal prompts -## Multi-modal prompts - -For models that support multimodal input such as images alongside text, you can +For models that support multimodal input, such as images alongside text, you can use the `{{media}}` helper: -```none ---- -model: vertexai/gemini-1.5-flash -input: - schema: - photoUrl: string ---- - -Describe this image in a detailed paragraph: - -{{media url=photoUrl}} +```handlebars +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/prompts/ex08.prompt" %} ``` -The URL can be `https://` or base64-encoded `data:` URIs for "inline" image -usage. In code, this would be: +The URL can be `https:` or base64-encoded `data:` URIs for "inline" image usage. +In code, this would be: ```ts -const describeImagePrompt = promptRef('describe_image'); - -const result = await describeImagePrompt.generate({ - input: { - photoUrl: 'https://example.com/image.png', - }, -}); - -console.log(result.text); +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="multiModalPrompt" adjust_indentation="auto" %} ``` -## Partials +See also [Multimodal input](/docs/genkit/models#multimodal-input), on the Models +page, for an example of constructing a `data:` URL. + +### Partials Partials are reusable templates that can be included inside any prompt. Partials can be especially helpful for related prompts that share common behavior. -When loading a prompt directory, any file prefixed with `_` is considered a -partial. So a file `_personality.prompt` might contain: +When loading a prompt directory, any file prefixed with an underscore (`_`) is +considered a partial. So a file `_personality.prompt` might contain: -```none + +```handlebars You should speak like a {{#if style}}{{style}}{{else}}helpful assistant.{{/else}}. ``` + This can then be included in other prompts: -```none + +```handlebars --- -model: vertexai/gemini-1.5-flash +model: googleai/gemini-1.5-flash input: schema: name: string style?: string --- - {{ role "system" }} {{>personality style=style}} @@ -375,79 +473,69 @@ Give the user a friendly greeting. User's Name: {{name}} ``` -Partials are inserted using the `{{>NAME_OF_PARTIAL args...}}` syntax. If no -arguments are provided to the partial, it executes with the same context as the -parent prompt. + +Partials are inserted using the +`{{>NAME_OF_PARTIAL args...}}` +syntax. If no arguments are provided to the partial, it executes with the same +context as the parent prompt. Partials accept both named arguments as above or a single positional argument -representing the context. This can be helpful for e.g. rendering members of a list. +representing the context. This can be helpful for tasks such as rendering +members of a list. -``` -# _destination.prompt -- {{name}} ({{country}}) +**_destination.prompt** -# chooseDestination.prompt -Help the user decide between these vacation destinations: -{{#each destinations}} -{{>destination this}}{{/each}} -``` -### Defining Partials in Code +```handlebars +- {{name}} ({{country}}) +``` -You may also define partials in code using `definePartial`: -```ts -import { definePartial } from '@genkit-ai/dotprompt'; +**chooseDestination.prompt** -definePartial( - 'personality', - 'Talk like a {{#if style}}{{style}}{{else}}helpful assistant{{/if}}.' -); -``` -Code-defined partials are available in all prompts. - -## Prompt Variants +```handlebars +--- +model: googleai/gemini-1.5-flash-latest +input: + schema: + destinations(array): + name: string + country: string +--- +Help the user decide between these vacation destinations: -Because prompt files are just text, you can (and should!) commit them to your -version control system, allowing you to compare changes over time easily. -Often times, tweaked versions of prompts can only be fully tested in a -production environment side-by-side with existing versions. Dotprompt supports -this through its **variants** feature. +{{#each destinations}} +{{>destination this}} +{{/each}} +``` -To create a variant, create a `[name].[variant].prompt` file. For instance, if -you were using Gemini 1.5 Flash in your prompt but wanted to see if Gemini 1.5 -Pro would perform better, you might create two files: -- `my_prompt.prompt`: the "baseline" prompt -- `my_prompt.gemini15pro.prompt`: a variant named "gemini15pro" +#### Defining partials in code -To use a prompt variant, specify the `variant` option when loading: +You can also define partials in code using `definePartial`: ```ts -const myPrompt = promptRef('my_prompt', { variant: 'gemini15pro' }); +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="definePartial" adjust_indentation="auto" %} ``` -The name of the variant is included in the metadata of generation traces, so you -can compare and contrast actual performance between variants in the Genkit trace -inspector. +Code-defined partials are available in all prompts. -## Defining Custom Helpers +### Defining Custom Helpers -You can define custom helpers to process and manage data inside of a prompt. Helpers -are registered globally using `defineHelper`: +You can define custom helpers to process and manage data inside of a prompt. +Helpers are registered globally using `defineHelper`: ```ts -import { defineHelper } from '@genkit-ai/dotprompt'; - -defineHelper('shout', (text: string) => text.toUpperCase()); +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="defineHelper" adjust_indentation="auto" %} ``` Once a helper is defined you can use it in any prompt: -```none + +```handlebars --- -model: vertexai/gemini-1.5-flash +model: googleai/gemini-1.5-flash input: schema: name: string @@ -456,48 +544,50 @@ input: HELLO, {{shout name}}!!! ``` -For more information about the arguments passed into helpers, see the -[Handlebars documentation](https://handlebarsjs.com/guide/#custom-helpers) on creating -custom helpers. -## Alternate ways to load and define prompts +## Prompt variants + +Because prompt files are just text, you can (and should!) commit them to your +version control system, allowing you to compare changes over time easily. Often, +tweaked versions of prompts can only be fully tested in a production environment +side-by-side with existing versions. Dotprompt supports this through its +variants feature. + +To create a variant, create a `[name].[variant].prompt` file. For instance, if +you were using Gemini 1.5 Flash in your prompt but wanted to see if Gemini 1.5 +Pro would perform better, you might create two files: + +* `my_prompt.prompt`: the "baseline" prompt +* `my_prompt.gemini15pro.prompt`: a variant named `gemini15pro` + +To use a prompt variant, specify the variant option when loading: + +```ts +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="loadPromptVariant" adjust_indentation="auto" %} +``` + +The name of the variant is included in the metadata of generation traces, so you +can compare and contrast actual performance between variants in the Genkit trace +inspector. + +## Defining prompts in code {:#alternatives} + +All of the examples discussed so far have assumed that your prompts are defined +in individual `.prompt` files in a single directory (or subdirectories thereof), +accessible to your app at runtime. Dotprompt is designed around this setup, and +its authors consider it to be the best developer experience overall. -Dotprompt is optimized for organization in the prompt directory. However, there -are a few other ways to load and define prompts: +However, if you have use cases that are not well supported by this setup, +you can also define prompts in code using the `definePrompt()` function: -- `loadPromptFile`: Load a prompt from a file in the prompt directory. -- `loadPromptUrl`: Load a prompt from a URL. -- `defineDotprompt`: Define a prompt in code. +The first parameter to this function is analogous to the front matter block of a +`.prompt` file; the second parameter can either be a Handlebars template string, +as in a prompt file, or a function that returns a `GenerateRequest`: -Examples: +```ts +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="definePromptTempl" adjust_indentation="auto" %} +``` ```ts -import { - loadPromptFile, - loadPromptUrl, - defineDotprompt, -} from '@genkit-ai/dotprompt'; -import path from 'path'; -import { z } from 'zod'; - -// Load a prompt from a file -const myPrompt = await loadPromptFile( - path.resolve(__dirname, './path/to/my_prompt.prompt') -); - -// Load a prompt from a URL -const myPrompt = await loadPromptUrl('https://example.com/my_prompt.prompt'); - -// Define a prompt in code -const myPrompt = defineDotprompt( - { - model: 'vertexai/gemini-1.5-flash', - input: { - schema: z.object({ - name: z.string(), - }), - }, - }, - `Hello {{name}}, how are you today?` -); +{% includecode github_path="firebase/genkit/js/doc-snippets/src/dotprompt/index.ts" region_tag="definePromptFn" adjust_indentation="auto" %} ``` diff --git a/docs/prompts.md b/docs/prompts.md deleted file mode 100644 index f3e9155704..0000000000 --- a/docs/prompts.md +++ /dev/null @@ -1,118 +0,0 @@ -# Prompts - -Prompt engineering is the primary way that you, as an app developer, influence -the output of generative AI models. For example, when using LLMs, you can craft -prompts that influence the tone, format, length, and other characteristics of -the models’ responses. - -Genkit is designed around the premise that **prompts are code**. You write and -maintain your prompts in source files, track changes to them using the same version -control system that you use for your code, and you deploy them along with the code -that calls your generative AI models. - -Genkit has a comprehensive tool for developing complex prompts called [Dotprompt](./dotprompt.md). See that documentation for the complete listing of features. - -This document shows several ways prompts can be used in Genkit, starting from the simpliest and moving toward more complex. - -## Defining prompts - -Genkit's `generate()` helper function accepts string prompts, and you can -call models this way for straightforward use cases. - -```ts -import { generate } from '@genkit-ai/ai'; - -generate({ - model: 'googleai/gemini-1.5-flash-latest', - prompt: 'You are a helpful AI assistant named Walt. Say hello.', -}); -``` - -In some cases you will need to include some customer provided inputs in your prompt. -You could use a template literal to render them like this: - -```ts -const name = "Fred"; -generate({ - model: 'googleai/gemini-1.5-flash-latest', - prompt: `You are a helpful AI assistant named Walt. Say hello to ${name}.`, -}); -``` - -However, Genkit provides a way to define your prompts in a standardized format that facilitates more advanced templating and rapid testing in the Developer UI. - -Use the `defineDotprompt` function to define these structured prompts. - -```ts -import { defineDotprompt } from '@genkit-ai/dotprompt' -import z from 'zod'; - -const helloPrompt = defineDotprompt( - { - name: 'helloPrompt', - model: 'googleai/gemini-1.5-flash-latest', - input: { - schema: z.object({ name: z.string() }), - }, - }, - `You are a helpful AI assistant named Walt. Say hello to {{name}}` -); -``` - -And then call the prompt using its `generate()` method: - -```ts -helloPrompt.generate({ input: { name: 'Fred' } }); -// Example output: Hello Fred! 👋 It's nice to meet you. How can I help you today? 😊 -``` - -As shown above, prompts defined this way can specify the structured inputs they accept through the `input.schema` configuration. This allows you a typesafe way to ensure that prompts can only be invoked with valid sets of inputs. - -Dotprompts can also specify an output, which they will pass along to call to the LLM as a directive (either as an in-context message or as an API parameter for LLMs which support a structured output mode). This guarantees that you'll either get a conforming response, or an exception you can deal with cleanly. - -```ts -const outputSchema = z.object({ - short: z.string(), - friendly: z.string(), - likeAPirate: z.string(), -}); - -const threeGreetingsPrompt = defineDotprompt( - { - name: 'threeGreetingsPrompt', - model: 'googleai/gemini-1.5-flash-latest', - input: { - schema: z.object({ name: z.string() }), - }, - output: { - format: 'json', - schema: outputSchema, - }, - }, - `You are a helpful AI assistant named Walt. Say hello to {{name}}, write a response for each of the styles requested` -); -``` - -You can then call `generate` on that prompt and work with the structured output in the response: - -```ts -const response = await (threeGreetingsPrompt.generate( - { input: { name: 'Fred' } } -)); - -response.output?.likeAPirate -// "Ahoy there, Fred! May the winds be ever in your favor!" -``` - -In the Genkit Developer UI, you can run any prompt you have defined in this way. This allows you to experiment with individual prompts outside of the scope of places in your code where they might be used. - -The developer UI showing JSON response to the threeGreetingsPrompt - -## Dotprompt - -See the [Dotprompt](./dotprompt.md) page for more features of the Dotprompt library, including - -- Loading prompts from `.prompt` source files -- Handlebars-based templates -- Support for multi-turn prompt templates and multimedia content -- Concise input and output schema definitions diff --git a/docs/resources/developer_ui_model_runner.png b/docs/resources/developer_ui_model_runner.png index 4440aa73ed..4d1f9c0d95 100644 Binary files a/docs/resources/developer_ui_model_runner.png and b/docs/resources/developer_ui_model_runner.png differ diff --git a/docs/resources/prompts-in-developer-ui.png b/docs/resources/prompts-in-developer-ui.png index 6b24d82cb1..2e430170a3 100644 Binary files a/docs/resources/prompts-in-developer-ui.png and b/docs/resources/prompts-in-developer-ui.png differ diff --git a/js/doc-snippets/package.json b/js/doc-snippets/package.json index a5c6433eb1..ea4036e58d 100644 --- a/js/doc-snippets/package.json +++ b/js/doc-snippets/package.json @@ -7,7 +7,6 @@ "license": "ISC", "dependencies": { "genkit": "workspace:*", - "@genkit-ai/firebase": "workspace:*", "@genkit-ai/googleai": "workspace:*" }, "devDependencies": { diff --git a/js/doc-snippets/src/dotprompt/index.ts b/js/doc-snippets/src/dotprompt/index.ts new file mode 100644 index 0000000000..04522aae17 --- /dev/null +++ b/js/doc-snippets/src/dotprompt/index.ts @@ -0,0 +1,199 @@ +/** + * Copyright 2024 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { GenerateRequest, genkit, loadPromptFile } from 'genkit'; + +// [START promptDir] +const ai = genkit({ + promptDir: './llm_prompts', + // (Other settings...) +}); +// [END promptDir] + +// [START MenuItemSchema] +import { z } from 'genkit'; + +const MenuItemSchema = ai.defineSchema( + 'MenuItemSchema', + z.object({ + dishname: z.string(), + description: z.string(), + calories: z.coerce.number(), + allergens: z.array(z.string()), + }) +); +// [END MenuItemSchema] + +async function fn02() { + // [START loadPrompt] + const helloPrompt = await ai.prompt('hello'); + // [END loadPrompt] + + // [START loadPromptVariant] + const myPrompt = await ai.prompt('my_prompt', { variant: 'gemini15pro' }); + // [END loadPromptVariant] + + // [START callPrompt] + const response = await helloPrompt(); + + // Alternatively, use destructuring assignments to get only the properties + // you're interested in: + const { text } = await helloPrompt(); + // [END callPrompt] + + // [START callPromptOpts] + const response2 = await helloPrompt( + // Prompt input: + { name: 'Ted' }, + + // Generation options: + { + config: { + temperature: 0.4, + }, + } + ); + // [END callPromptOpts] +} + +async function fn03() { + const helloPrompt = await ai.prompt('hello'); + + // [START callPromptCfg] + const response3 = await helloPrompt( + {}, + { + config: { + temperature: 1.4, + topK: 50, + topP: 0.4, + maxOutputTokens: 400, + stopSequences: ['', ''], + }, + } + ); + // [END callPromptCfg] +} + +async function fn04() { + // [START outSchema] + // [START inSchema] + const menuPrompt = await ai.prompt('menu'); + const { data } = await menuPrompt({ theme: 'medieval' }); + // [END inSchema] + + const dishName = data['dishname']; + const description = data['description']; + // [END outSchema] +} + +async function fn05() { + // [START outSchema2] + const menuPrompt = await ai.prompt< + z.ZodTypeAny, // Input schema + typeof MenuItemSchema, // Output schema + z.ZodTypeAny // Custom options schema + >('menu'); + const { data } = await menuPrompt({ theme: 'medieval' }); + + // Now data is strongly typed as MenuItemSchema: + const dishName = data?.dishname; + const description = data?.description; + // [END outSchema2] +} + +async function fn06() { + // [START multiTurnPrompt] + const multiTurnPrompt = await ai.prompt('multiTurnPrompt'); + const result = await multiTurnPrompt({ + messages: [ + { role: 'user', content: [{ text: 'Hello.' }] }, + { role: 'model', content: [{ text: 'Hi there!' }] }, + ], + }); + // [END multiTurnPrompt] +} + +async function fn07() { + // [START multiModalPrompt] + const multimodalPrompt = await ai.prompt('multimodal'); + const { text } = await multimodalPrompt({ + photoUrl: 'https://example.com/photo.jpg', + }); + // [END multiModalPrompt] +} + +async function fn08() { + // [START definePartial] + ai.definePartial( + 'personality', + 'Talk like a {{#if style}}{{style}}{{else}}helpful assistant{{/if}}.' + ); + // [END definePartial] + + // [START defineHelper] + ai.defineHelper('shout', (text: string) => text.toUpperCase()); + // [END defineHelper] +} + +function fn09() { + // [START definePromptTempl] + const myPrompt = ai.definePrompt( + { + name: 'myPrompt', + model: 'googleai/gemini-1.5-flash', + input: { + schema: z.object({ + name: z.string(), + }), + }, + }, + 'Hello, {{name}}. How are you today?' + ); + // [END definePromptTempl] +} + +function fn10() { + // [START definePromptFn] + const myPrompt = ai.definePrompt( + { + name: 'myPrompt', + model: 'googleai/gemini-1.5-flash', + input: { + schema: z.object({ + name: z.string(), + }), + }, + }, + async (input): Promise => { + return { + messages: [ + { + role: 'user', + content: [{ text: `Hello, ${input.name}. How are you today?` }], + }, + ], + }; + } + ); + // [END definePromptFn] +} + +async function fn01() { + // [START loadPromptFile] + const helloPrompt = loadPromptFile(ai.registry, './llm_prompts/hello.prompt'); + // [END loadPromptFile] +} diff --git a/js/doc-snippets/src/dotprompt/minimal.ts b/js/doc-snippets/src/dotprompt/minimal.ts new file mode 100644 index 0000000000..e035fc815e --- /dev/null +++ b/js/doc-snippets/src/dotprompt/minimal.ts @@ -0,0 +1,31 @@ +/** + * Copyright 2024 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +// [START mini] +import { genkit } from 'genkit'; + +// Import the model plugins you want to use. +import { googleAI } from '@genkit-ai/googleai'; + +const ai = genkit({ + // Initialize and configure the model plugins. + plugins: [ + googleAI({ + apiKey: 'your-api-key', // Or (preferred): export GOOGLE_GENAI_API_KEY=... + }), + ], +}); +// [END mini] diff --git a/js/doc-snippets/src/dotprompt/prompts/ex01.prompt b/js/doc-snippets/src/dotprompt/prompts/ex01.prompt new file mode 100644 index 0000000000..808989a655 --- /dev/null +++ b/js/doc-snippets/src/dotprompt/prompts/ex01.prompt @@ -0,0 +1,16 @@ +--- +model: googleai/gemini-1.5-flash +config: + temperature: 0.9 +input: + schema: + location: string + style?: string + name?: string + default: + location: a restaurant +--- + +You are the world's most welcoming AI assistant and are currently working at {{location}}. + +Greet a guest{{#if name}} named {{name}}{{/if}}{{#if style}} in the style of {{style}}{{/if}}. \ No newline at end of file diff --git a/js/doc-snippets/src/dotprompt/prompts/ex02.prompt b/js/doc-snippets/src/dotprompt/prompts/ex02.prompt new file mode 100644 index 0000000000..32f5b8eca2 --- /dev/null +++ b/js/doc-snippets/src/dotprompt/prompts/ex02.prompt @@ -0,0 +1,4 @@ +--- +model: vertexai/gemini-1.5-flash +--- +You are the world's most welcoming AI assistant. Greet the user and offer your assistance. \ No newline at end of file diff --git a/js/doc-snippets/src/dotprompt/prompts/ex03.prompt b/js/doc-snippets/src/dotprompt/prompts/ex03.prompt new file mode 100644 index 0000000000..b9e53b4a2d --- /dev/null +++ b/js/doc-snippets/src/dotprompt/prompts/ex03.prompt @@ -0,0 +1,11 @@ +--- +model: googleai/gemini-1.5-flash +config: + temperature: 1.4 + topK: 50 + topP: 0.4 + maxOutputTokens: 400 + stopSequences: + - "" + - "" +--- \ No newline at end of file diff --git a/js/doc-snippets/src/dotprompt/prompts/ex04.prompt b/js/doc-snippets/src/dotprompt/prompts/ex04.prompt new file mode 100644 index 0000000000..54f23637dd --- /dev/null +++ b/js/doc-snippets/src/dotprompt/prompts/ex04.prompt @@ -0,0 +1,15 @@ +--- +model: googleai/gemini-1.5-flash +input: + schema: + theme?: string + default: + theme: "pirate" +output: + schema: + dishname: string + description: string + calories: integer + allergens(array): string +--- +Invent a menu item for a {{theme}} themed restaurant. \ No newline at end of file diff --git a/js/doc-snippets/src/dotprompt/prompts/ex05.prompt b/js/doc-snippets/src/dotprompt/prompts/ex05.prompt new file mode 100644 index 0000000000..4099ba82ea --- /dev/null +++ b/js/doc-snippets/src/dotprompt/prompts/ex05.prompt @@ -0,0 +1,7 @@ +--- +model: googleai/gemini-1.5-flash +input: + schema: + theme?: string +--- +Invent a menu item for a {{#if theme}}{{theme}} themed{{/if}} restaurant. \ No newline at end of file diff --git a/js/doc-snippets/src/dotprompt/prompts/ex06.prompt b/js/doc-snippets/src/dotprompt/prompts/ex06.prompt new file mode 100644 index 0000000000..6c55a29454 --- /dev/null +++ b/js/doc-snippets/src/dotprompt/prompts/ex06.prompt @@ -0,0 +1,11 @@ +--- +model: vertexai/gemini-1.5-flash +input: + schema: + userQuestion: string +--- +{{role "system"}} +You are a helpful AI assistant that really loves to talk about food. Try to work +food items into all of your conversations. +{{role "user"}} +{{userQuestion}} \ No newline at end of file diff --git a/js/doc-snippets/src/dotprompt/prompts/ex07.prompt b/js/doc-snippets/src/dotprompt/prompts/ex07.prompt new file mode 100644 index 0000000000..119bef79cd --- /dev/null +++ b/js/doc-snippets/src/dotprompt/prompts/ex07.prompt @@ -0,0 +1,10 @@ + +{{role "system"}} +This is the system prompt. +{{history}} +{{role "user"}} +This is a user message. +{{role "model"}} +This is a model message. +{{role "user"}} +This is the final user message. \ No newline at end of file diff --git a/js/doc-snippets/src/dotprompt/prompts/ex08.prompt b/js/doc-snippets/src/dotprompt/prompts/ex08.prompt new file mode 100644 index 0000000000..3df55e7876 --- /dev/null +++ b/js/doc-snippets/src/dotprompt/prompts/ex08.prompt @@ -0,0 +1,9 @@ +--- +model: vertexai/gemini-1.5-flash +input: + schema: + photoUrl: string +--- +Describe this image in a detailed paragraph: + +{{media url=photoUrl}} \ No newline at end of file diff --git a/js/pnpm-lock.yaml b/js/pnpm-lock.yaml index 66d54f4dcd..c435067ef8 100644 --- a/js/pnpm-lock.yaml +++ b/js/pnpm-lock.yaml @@ -136,9 +136,6 @@ importers: doc-snippets: dependencies: - '@genkit-ai/firebase': - specifier: workspace:* - version: link:../plugins/firebase '@genkit-ai/googleai': specifier: workspace:* version: link:../plugins/googleai