LLM as Interpreter for Natural Language Programming, Pseudo-code Programming and Flow Programming of AI Agents

Since their inception, programming languages have trended towards greater readability and lower barriers for programmers. Following this trend, natural language can be a promising type of programming language that provides great flexibility and usability and helps towards the democracy of programming. However, the inherent vagueness, ambiguity, and verbosity of natural language pose significant challenges in developing an interpreter that can accurately understand the programming logic and execute instructions written in natural language. Fortunately, recent advancements in Large Language Models (LLMs) have demonstrated remarkable proficiency in interpreting complex natural language. Inspired by this, we develop a novel system for Code Representation and Execution (CoRE), which employs LLM as interpreter to interpret and execute natural language instructions. The proposed system unifies natural language programming, pseudo-code programming, and flow programming under the same representation for constructing language agents, while LLM serves as the interpreter to interpret and execute the agent programs. In this paper, we begin with defining the programming syntax that structures natural language instructions logically. During the execution, we incorporate external memory to minimize redundancy. Furthermore, we equip the designed interpreter with the capability to invoke external tools, compensating for the limitations of LLM in specialized domains or when accessing real-time information.

This package is mainly contributed by Shuyuan Xu (shuyuan.xu@rutgers.edu), Zelong Li (zelong.li@rutgers.edu), and Yongfeng Zhang (yongfeng.zhang@rutgers.edu). We welcome any issues and requests for model implementation and bug fix.

Reference

To be updated

Preparation

0. Clone this repo.

1. Create a conda virtual environment and install necessary packages:

conda create -n your_env_name python=3.9
conda activate your_env_name
pip install -r requirements.txt

2. Download the OpenAGI data from this Google Drive link, unzip it to the CoRE directory and rename it as openagi_data.

3. Download the database and unzip it to the CoRE directory (i.e., your/path/CoRE) and rename it as travel_database.

4. Make sure you are in the CoRE/src folder before running the codes. Otherwise,

cd src

Running Command Examples

OpenAGI on gpt-4-1106-preview:

python main.py \
--flow_name=OpenAGI_Flow.txt \
--tool_name=tools.txt \
--task=OpenAGI \
--log_file_name=OpenAGI_gpt_log.txt \
--model_name=gpt-4-1106-preview \
--openai_key="YOUR OPENAI KEY"

OpenAGI on TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ:

python main.py \
--flow_name=OpenAGI_Flow.txt \
--tool_name=tools.txt \
--task=OpenAGI \
--log_file_name=OpenAGI_mixtral_log.txt \
--model_name=TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ \
--openai_key="YOUR OPENAI KEY"

TravelPlanner on gpt-4-1106-preview:

python main.py \
--flow_name=TravelPlanner_Flow.txt \
--tool_name=tools.txt \
--task=TravelPlanner \
--log_file_name=TravelPlanner_gpt_log.txt \
--results_name=gpt \
--model_name=gpt-4-1106-preview \
--openai_key="YOUR OPENAI KEY"

TravelPlanner on TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ:

python main.py \
--flow_name=TravelPlanner_Flow.txt \
--tool_name=tools.txt \
--task=TravelPlanner \
--log_file_name=TravelPlanner_mixtral_log.txt \
--results_name=mixtral \
--model_name=TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ \
--openai_key="YOUR OPENAI KEY"

Reference

• We leveraged the dataset of OpenAGI projects to implement our experiment.