OpenELM is an open-source library by CarperAI, designed to enable evolutionary search with language models in both code and natural language.
The OpenELM project has the following goals:
- Release an open-source version of ELM with its associated diff models.
- Integrate with both open-source language models (run locally or on Colab) and with closed models via paid APIs, such as the OpenAI API. We want to support users with many different compute profiles!
- Provide a simple interface to a range of example environments for evolutionary search, to let users adapt these easily for their domain.
- Demonstrate the potential of evolution with LLMs.
pip install openelm
To use the sodarace environment, you must first
pip install swig.
pip install openelm[sodaracer]
See the pyproject.toml for further install options.
OpenELM supports the quality-diversity algorithms MAP-Elites, CVT-MAP-Elites, and Deep Grid MAP-Elites, as well as a simple genetic algorithm baseline.
- Prompt-based mutation with instruct models
- Diff models (specialised for code)
- Crossover with language models
OpenELM’s language models are instantiated as Langchain classes by default, which means that OpenELM can support practically any existing LLM API, as well as models run on your local GPU via HuggingFace Transformers.
We also provide optional Nvidia Triton Inference Server support, intended for use cases where low latency on 8 or more GPUs is important. Finally, for code generation domains, we provide a sandbox environment, consisting of a container server backed with gVisor (a container runtime that introduces an additional barrier between the host and the container) as well as a heuristic-based safety guard.
- Sodarace. A 2D physics-based simulation of robots moving across a variety of terrains. These robots are created by Python programs generated from an LLM.
- Image Generation. OpenELM can evolve over generated images by generating code that returns NumPy arrays containing the images. This serves as a simple test environment for code generation
- Programming Puzzles. OpenELM can be used to generate diverse solutions to programming puzzles. This environment supports co-evolution of both the problem and the solution at the same time.
- Prompts. OpenELM contains a generic environment suitable for evolving prompts for language models, customizable with Langchain templates to the desired domain.
- We also include a poetry environment, demonstrating the use of LLMs to evaluate both the quality and diversity of generated creative writing text, as described in a recent CarperAI blog post on Quality-Diversity with AI Feedback (QDAIF).
Roughly, ELM consists of a pipeline of different components:
Environmentclass. This class defines the mechanics of how to initialize members of the population, mutate them with the desired operator, and how to measure the fitness (and diversity) of individuals.
MAPElitesclass. This class describes how the evolutionary algorithm works, and can be viewed as a wrapper around the environment defining the selection algorithm for generated individuals.
MutationModelclass, which is responsible for running the LLM to actually generate new individuals. This functions as a wrapper around the LangChain API. The environment is expected to call the
MutationModelwhen a new individual is needed.
ELMclass calls the
MAPElitesalgorithm class and runs the search.
All options for these classes are defined in
configs.py, via dataclasses which are registered as a
hydra config, and can be overriden via the command line when running one of the example scripts such as
python run_elm.py will start an ELM evolutionary search using the defaults listed in
configs.py. These can be overriden via the command line. For example, you can use
run_elm.py env=image_evolution to run the Image Evolution environment.
To use the code execution sandbox, see the sandboxing readme for instructions to set it up in a Docker container with the gVisor runtime.
We also have code available to run models in Nvidia's Triton Inference Server. See the Triton Readme to get started
If you'd like to contribute or have questions, go to the #openelm channel on the CarperAI discord!