10.0: Smart interactions with OpenAI

[jstanden]

Introduction to OpenAI and GPT-3

OpenAI provides a beta API for arbitrary text completion tasks. It is built on top of the GPT-3 (Generative Pre-trained Transformer 3) language model.

GPT-3 was trained on hundreds of gigabytes of text from Internet sources like Common Crawl, Wikipedia, and collections of books. The data set contains more than 300 billion "tokens" (word fragments) which resulted in 175 billion "parameters" (probabilistic learning).

As a result, GPT-3 has learned an incredible amount about language: vocabulary, grammar, synonyms/antonyms, parts of speech, word order, gender, classification, nuance, sentiment, translation, etc.

To use GPT-3, you provide a prompt with a few examples of the kinds of output you would like, and the model predicts what comes next within the constraints you give it.

For instance, without any extra training or refinement, I prompted GPT-3 with the beginning of this sentence:

Cerb is

And it returned:

Cerb is a free, open source, web-based project management and group collaboration tool. It is written in PHP and uses MySQL for storage.

According to Google and DuckDuckGo, as of the time of writing this, that exact phrase doesn't show up anywhere online (of course, after posting this it will link here).

GPT-3 isn't just repeating back blocks of text it previously read on a website; it's summarizing information it has learned from many public sources.

Aside from having it recall facts, you can "teach" the model to perform textual tasks in natural language.

For instance, returning the color of fruits (bold text was generated):

strawberry: red
watermelon: green
pumpkin: orange
orange: orange
banana: yellow
lime: green
tomato: red

Or performing English<->German translation:

en: hello
de: hallo
en: good day
de: Guten Tag
en: so you later
de: bis später
en: house
de: Haus
en: happy
de: zufrieden
en: sad
de: traurig

Not only has the model performed a translation, but it has learned to mimic our format by prefixing the language codes in front of phrases.

If we let it continue, it will generate both new English prompts and German translations entirely on its own:

en: hungry
de: hungrig
en: I
de: ich
en: I am
de: ich bin
en: I have
de: ich habe
en: I want
de: ich will

It can also perform question and answer sessions; generating new questions on its own and then answering them:

Q: What is the capital of the United States of America?
A: Washington D.C.
Q: Who was the first president of the United States of America?
A: George Washington
Q: When was America founded?
A: 1776
Q: What is the national anthem of the United States of America?
A: The Star-Spangled Banner
Q: What is the name of the president of the United States of America?
A: Barack Obama

The above example also demonstrates some important limitations of GPT-3:

• GPT-3 has learned through probability and repetition. Its answers should not be inherently trusted as factual.
• GPT-3 was trained on data through October 2019. It doesn't know anything about very current events (e.g. Coronavirus).
• GPT-3 is subject to biases within its source data.

Despite these limitations, there are many use cases where GPT-3 can spark creativity or perform grunt work for you.

Cerb

Let's look at some examples where we could use the power of GPT-3 within Cerb.

As previously mentioned, OpenAI's API is in a private beta. If you join the waitlist and are accepted, you will receive a unique API token.

Create a connected service for OpenAI

In Cerb, you can create a 'Token Bearer' Connected Service for 'OpenAI'.

Navigate to Search -> Connected Services -> (+).

Select the Build tab.

Field	Value
Name:	OpenAI
URI:	openai
Type:	Token Bearer
Token Name:	Bearer

Click the Save Changes button.

Create a connected account

Now create a Connected Account named 'OpenAI' with the token.

Navigate to Search -> Connected Accounts -> (+).

Select OpenAI.

Field	Value
Name:	OpenAI
URI:	openai
Service:	OpenAI
Owner:	Cerb
Token:	(paste your token from OpenAI API)

Automations

In Cerb 9.x you can use bot behaviors to interact with OpenAI's API. In 10.x you should start using automations.

We're going to teach GPT-3 how to generate KATA for interactions. This way we can simply describe what we want to build without having to remember the syntax.

Navigate to Search -> Automations -> (+).

Field	Value
Name:	example.openai.interaction
Description:	Generate interaction KATA using OpenAI
Trigger:	interaction.worker

start:
  set:
    prompt_text@text:
      Ask "What is your favorite ice cream flavor?"
    training@text:
      Q: Ask "What is your name?""
      A: text/prompt_name:
       label: What is your name?
       type: freeform
      ~~
      Q: Ask for email address
      A: text/prompt_email_address:
       label: Email Address:
       type: freeform
      ~~
      Q: Ask "Where are you located?""
      A: text/prompt_location:
       label: Where are you located?
       type: freeform
      ~~
      Q: Say "How can I help?"
      A: say/help:
       message@text:
        How can I help?
      ~~
      Q: Show a table named "language" that has rows for: English, Spanish, German
      A: sheet/prompt_language:
       label: Language
       data:
        0:
         key: english
         label: English
        1:
         key: spanish
         label: Spanish
        2:
         key: german
         label: German
       schema:
        layout:
         headings@bool: no
         style: table
        columns:
         selection/key:
         text/label:
      ~~
      Q: Ask about favorite color
      A: text/prompt_favorite_color:
       label: Favorite Color:
       type: freeform
      ~~

  http.request/prediction:
    output: http_response
    inputs:
      method: POST
      url: https://api.openai.com/v1/engines/curie/completions
      headers@text:
        Content-Type: application/json
      body@text:
        {% set json = {
          max_tokens: 255,
          temperature: 0,
          stop: ["~~\n"],
          prompt: training ~ "\nQ: " ~ prompt_text ~ "\n",
        } %}
        {{json|json_encode}}
      authentication: cerb:connected_account:openai
    on_success:
      set:
        response@key,json: http_response:body
      return:
        snippet@text:
          {{response.choices[0].text|slice(3)}}

We're setting a placeholder called training with several examples of how we'd describe a prompt and then how we'd write it in KATA.

We're then setting another placeholder called prompt_text with a description of what we want in KATA (Ask "What is your favorite ice cream flavor?").

We use the http.request: command to call the OpenAI API. This is a simple POST request that sends and receives JSON.

In our training data, we prefixed the natural language prompt with Q: and the resulting KATA with A:.

We delimited examples with ~~, which is just an arbitrary sequence that's unlikely to appear in the training or real-world responses. We use the delimiter to tell OpenAI to stop generating text once it encounters that token. This gives us back a single answer, rather than letting GPT-3 generate additional new prompts and answers.

The authentication: option securely authenticates our request with the OpenAI token from our connected account.

On success, we parse the response as JSON and extract the first prediction. We return that as a key named snippet.

In Cerb 10+, interactions have expected inputs and outputs. When an interaction is used in a toolbar above a text editor, it often may output a snippet that is then pasted at the current cursor location in the text. We'll use that capability in a moment.

Automations also require a policy to grant permissions. We'll use this simple one:

commands:
  http.request:
    deny@bool: {{inputs.url is not prefixed ("https://api.openai.com/")}}
    allow: yes

This allows the http.request: command only when the url: input is prefixed with https://api.openai.com/. We could also limit the method to POST, the content-type header, the connected account, etc.

When we run the automation with this prompt:

Ask "What is your favorite ice cream flavor?"

We get this result:

__exit: return
__return:
  snippet: |
    text/prompt_favorite_ice_cream:
     label: Favorite Ice Cream Flavor:
     type: freeform

That's pretty amazing. With only a few examples, the model has learned how to translate our natural language descriptions into code.

Some extra considerations

• OpenAI charges by token, so we try to use the smallest prompt that returns good results while still remaining flexible.

• The davinci engine is the most powerful, but it's also the most expensive. We built our example prompt using curie which returns useful results at 1/10th the cost. This should be able to return several hundred responses for about $1 USD.

• Using curie, the model occasionally returned inconsistent indentation levels for sibling keys. For this reason we trained it with single space indents rather than the usual double spaces.

Adding the interaction to a toolbar

In Cerb 10, interactions are launched from toolbars. Toolbars are also written in a KATA syntax. They display interactions as buttons and menu items that can be conditionally visible (enabled) or hidden (disabled). You can pass input data to interactions, and control what happens when they finish (e.g. refreshing a widget).

We can edit the toolbars for common UI elements in Cerb from Search->Toolbars.

We want to add a new button to the toolbar on the automation editor, so we'll edit the automation.editor record.

This KATA adds a new button for our interaction above, with a 'magic' icon, and only displays it for two automation types (triggers):

interaction/magic:
  uri: cerb:automation:example.openai.interaction
  icon: magic
  tooltip: Add form elements with OpenAI
  hidden@bool:
    {{trigger_name not in [
      'interaction.website',
      'interaction.worker',
    ]}}

When we create a new automation of a different type we see nothing special in the toolbar (as expected):

But when we switch the Trigger: to an interaction, we see our new button:

Making it interactive

Our example above has a hard-coded prompt, so it would always return the same "Favorite ice cream flavor" prompt.

To make this interactive, we'll remove the set:prompt_text: command, and above the set: add an interactive form:

start:
  await:
    form:
      title: OpenAI
      elements:
        text/prompt_text:
          label: What form element would you like to add?
          required@bool: yes
          placeholde: (e.g. ask for name)
          type: freeform

  set:
    training@text:
    #...

Now when we click our new button, we see:

And we get back the KATA:

More examples

Text prompts are easy enough, but OpenAI is capable of far more.

We gave the model one example of generating a sheet prompt. Sheets are a very flexible collection of rows and columns, with different layouts (table, fieldsets, buttons, scale, grid) and selection modes (single, multiple).

Let's try our favorite ice cream prompt again, with a finite set of choices:

OpenAI returns:

That's syntactically correct. We can start that interaction from any portal or website and see:

Now let's look at how we can leverage GPT-3's understanding of language to combine concepts. We can prompt for a favorite color with a sheet, but also describe the kinds of options we want without a literal list:

It generates:

Again, we can start that interaction from any website and see:

Perhaps we wanted all seven of the typical rainbow colors (ROYGBIV). The model understands natural language, so we can be more descriptive:

Which gives us:

The model took a little artistic liberty in generating a final color -- most likely because it was tripped up by counting from zero to seven. It's still fascinating that a few extra descriptive words got us much closer to what we want.

Let's try something completely different:

To answer that, the model needs to understand many concepts. What's a "smart phone"? What's a "maker"? What's "market share"? Should they be ordered?

GPT-3 doesn't hesitate:

What about literary knowledge?

I might have included 'Romeo and Juliet' or 'Titus Andronicus', but it knew exactly what we were talking about:

Maybe if we're more specific?

We get:

The Tempest is a tragicomedy, and appears twice. "The Winter's Tale" is considered a comedic romance. But overall these are very helpful results when we need a bit of creative help.

Creating a registration form with an AI helper

Here's a real-time video of creating a simple registration interaction from scratch using Cerb and OpenAI:

cerb10_openai_interactions.mov

Other ideas

This same approach can be used for other text-based tasks. OpenAI can:

• Rephrase long email messages into a summary.

• Extract key words from text.

• Classify documents into categories.

• Suggest or generate the remainder of a sentence when replying.

• Detect intent in a natural language sentence for triggering actions and interactions. For instance, "remind me", "schedule", "add a new", "show my open tasks", etc.

• Answer questions. This will work even better when you can refine the model with your own documentation, knowledgebase, and FAQs.

• Generate entire automations and packages from a simple written description.

Conclusion

These interactions aren't limited to simple web forms, either. Cerb's automation-based interactions are available in multiple channels: web, SMS, voice, email, chat, mobile, Slack, etc. Automations use "continuations" to remember all previous inputs and intelligently change the flow of a conversation (e.g. troubleshooters). Everything is written in the same approachable KATA syntax. As we've seen here, automations can even generate KATA to create entirely new automations on-the-fly.

While working on Cerb, it's not our intention to compete directly with buildings full of machine learning expert PhDs. Cerb features like automations, interactions, and KATA, make it easy to leverage any third-party service or future technological advances. We're a powerful, customizable interface between humans and machines; accessible where your workers are already signed in every day.