Validating AI-Generated Code with Live Programming

Table of Content

1. About
2. Prerequisites
3. How to Build
4. How to Run
5. How to Use

About

LEAP (Live Exploration of AI-Generated Programs) combines Projection Boxes with a Copilot-like tool that generates code completions within the current buffer. You can find an up-to-date version of this artifact on GitHub.

LEAP (similar to Projection Boxes) uses a modified version of Visual Studio Code, including Projeciton Boxes and LEAP as separate contributions in the editor. For code generation, it uses OpenAI's Completion API.

For any questions, please reach out to the authors of the paper Kasra Ferdowsi and Ruanqianqian (Lisa) Huang, or create an issue on GitHub.

Prerequisites

To build LEAP, you need to first install the following:

1. Node.js 16
2. Python 3.8 or above
3. The numpy Python package
4. The matplotlib Python package
5. yarn 1.22.x
6. An OpenAI API key to use for generating completions.

Once you satisfy the requirements please follow the next steps to setup your environment.

How to Build

Building from the terminal

The simplest way to build and run LEAP is directly within the terminal. To do so:

1. (Mac users only) Open up a terminal, type arch to ensure that you are using a x86_64 architecture. M1 is currently not supported.
2. Navigate to the directory containing this README file.
3. Run yarn or yarn install. This will install all the necessary node packages. Ensure that this command finishes successfully. If not, please look at the errors and address them before moving on.
4. Run yarn compile. This will compile the source code.

Building from VSCode

You can also build LEAP by opening this directory inside a regular instance of VSCode. To do so:

1. Follow steps 1-3 above.
2. Open the directory containing this README in VSCode.
3. Run Ctrl + Shift + B (Cmd + Shift + B on Mac) to compile the source code.

Unlike the terminal instructions which build the source once, this starts a build daemon which automatically recompiles the code after edits. You may find this useful if you are modifying the source code.

How to Run

LEAP depends on a number of environment variables that must be present and set correctly for it to work. These env vars are:

1. PYTHON3: Absolute path to the Python 3 executable.
2. RUNPY: Absolute path to the ./src/run.py file.
3. IMGSUM: Absolute path to the ./src/img-summary.py file.
4. LEAP_PROMPT: Absolute path to the ./src/implement_it.txt file.
5. OPENAI_API_KEY: Your OpenAI API key.

How you set these variables is up to you (e.g. you could set them globally) but here we include instructions for setting them for running from the terminal, and from VSCode.

Running from the terminal

We have provided a shell script for running LEAP in ./run.sh. First edit the script in a text editor and replace OPENAI API KEY HERE with your OpenAI API key. The other env vars should be set automatically, but if you run into issues, you may want to hard code them in this script as well.

After editing this file, save and close and run it with ./run.sh. Note that you must build LEAP before running this script, otherwise you will run into errors.

Running from VSCode

To run LEAP directly from VSCode:

1. Open the directory containing this README in VSCode.
2. In the launch.json under .vscode directory set the env vars above under Launch VS Code Internal
   • You may also configure these environment variables in your shell configuration (e.g., in ~/.profile).
   • Alternatively, you may configure the variables in ./.env, and declare in inputs in .vscode/launch.json the id's and keys of the configured variables. For example, if you have configured PYTHON3=/usr/local/bin/python3 in ./.env, then the inputs in ./vscode/launch.json should be

   	"inputs": [
   		{
   			"id": "envPYTHON3",
   			"type": "command",
   			"command": "extension.commandvariable.file.content",
   			"args": {
   			  "fileName": "${workspaceFolder}/.env",
   			  "key": "PYTHON3",
   			  "default": ""
   			}
   		}
   	]
   

   and the environment variable configuration in Launch VS Code Internal, under .vscode/launch.json, should be

   "env": {
   	"PYTHON3": "${input:envPYTHON3}"
   }
   

3. Run the build by pressing F5. Note that you must build LEAP before pressing F5, otherwise you will run into errors.

How to Use

To use LEAP, first open a python file or open a new file and save it with a .py extension. Then, as you write executable Python code, you should see Projection Boxes appearing to show your program's runtime values. To invoke the AI assistant, press Ctrl + Enter (Cmd + Enter on Mac) on a new line. This should open a side panel containing code suggestions.