Centralized provisioning of
large language models
for a research community
--- HPC User Support Tools (HUST23)
Supplement for an accepted short paper in HUST23, "Centralized provisioning of large language models for a research community," an SC23 workshop. DOI: 10.1145/3624062.3624147.
This repository provides instructions, scripts, and example notebooks for recreating the centralized-HPC LLM efforts of the authors.
Our production environment exists on Arizona State University's flagship
supercomputer, Sol, which has a Rocky Linux 8 operating system. However,
reproduction should only depend on having the mamba (or with a simple
subsitution, conda) python package manager.
The shell script, INSTALL_PYTHON_ENV.sh, may be run to create a mamba
environment called genai that mirrors our own. The mamba export env
specification file genai23.09-mamba-environment.yml is provided for
completeness, but IS NOT recommended to rebuild from (due to over-specification
and build-order fragilities). Once the environment is created, create an
appropriate Jupyter kernel. An example is provided in the subdirectory
example-jupyter-kernel. In summary:
- Install
mamba, the python package manager and only dependency. - Run the
INSTALL_PYTHON_ENV.shshell script. - Create a Jupyter Kernel.
- Installation is complete, now run the example notebooks!
The subdirectory, example-notebooks, contains two generalized templated
Jupyter notebooks that have been shared with Research faculty to get them
started with LLMs on the Sol supercomputer.
The subdirectory, gpu-config/, provides details on the GPU configuration
necessary to replicate the presented results. For additional hardware details,
see our short paper on the Sol supercomputer.
LLM interaction may be empowered by gradio, which provides a
temporary public web app for a chat-like LLM interface. The example Jupyter
notebook, Chat_with_LLMs.ipynb, provides a template for launching the app,
and an example of the app is illustrated by the screenshot below.
