Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: improve README.md #673

Merged
merged 5 commits into from
Mar 26, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
90 changes: 75 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,65 @@

The Podman Desktop AI Studio extension simplifies getting started and developing with AI in a local environment. It provides key open-source technologies to start building on AI. A curated catalog of so-called recipes helps navigate the jungle of AI use cases and AI models. AI Studio further ships playgrounds: environments to experiment with and test AI models, for instance, a chat bot.



## Topics
- [Technology](#technology)
- [Extension features](#extension-features)
- [Requirements](#requiremements)
- [Installation](#installation)
- [Usage](#usage)
- [Contributing](#contributing)
- [Feedback](#feedback)



## Technology

The AI Studio extensions uses [Podman](https://podman.io) machines to run inference servers for LLM models and AI applications.
The AI models can be downloaded, and common formats like [GGUF](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md), [Pytorch](https://pytorch.org) or [Tensorflow](https://www.tensorflow.org).
jeffmaury marked this conversation as resolved.
Show resolved Hide resolved

## Extension features

### AI Models

AI Studio provides a curated list of open source AI models and LLMs. Once downloaded, the models are available to be used for AI applications, model services and playgrounds.

#### Model services

Once a model is downloaded, a model service can be started. A model service is an inference server that is running in a container and exposing the model through the well-known chat API common to many providers.

#### Playgrounds

The integrated Playground environments allow for experimenting with available models in a local environment. An intuitive user prompt helps in exploring the capabilities and accuracy of various models and aids in finding the best model for the use case at hand. The Playground interface further allows for parameterizing models to further optimize the settings and attributes of each model.

### AI applications

Once an AI model is available through a well know endpoint, it's easy to imaging a new world of applications that will connect and use the AI model. AI Studio support AI applications as a set of containers that are connected together.

AI Studio ships with a so-called Recipes Catalog that helps you navigate a number of core AI use cases and problem domains such as Chat Bots, Code Generators and Text Summarizers. Each recipe comes with detailed explanations and sample applications that can be run with various large language models (LLMs). Experimenting with multiple models allows finding the optimal one for your use case.

## Requirements

Disclaimer: This is **EXPERIMENTAL** and all features are subject to change as we develop the extension.

### Requirement 1. Software and hardware requirements

**OS:**

Compatible on Windows, macOS & Linux

**Software:**

- [Podman Desktop 1.8.0+](https://github.com/containers/podman-desktop)
- [Podman 4.9.0+](https://github.com/containers/podman)
vrothberg marked this conversation as resolved.
Show resolved Hide resolved

**Hardware**

LLMs AI models are heavy resource consumers both in terms of memory and CPU. Each of the provided models consumes about 4GiB of memory and requires at least 4 CPUs to run.

So we recommend that no more that 3 simultaneous models running concurrently and a minimum of 12Gb of memory and at least 4 CPUs for the Podman machine. Please note that this is not relevant for WSL on Windows as the WSL technology the memory and CPU with the host desktop.
jeffmaury marked this conversation as resolved.
Show resolved Hide resolved

## Installation

To install the extension, go to the Podman Desktop UI > ⚙ Settings > Extensions > Install a new extension from OCI Image.
Expand All @@ -12,27 +71,28 @@ To install a development version, use the `:nightly` tag as shown in the recordi

![](https://github.com/containers/podman-desktop-media/raw/ai-lab/gifs/installation.gif)

## Running in development mode

From the AI Studio packages/frontend folder:
## Contributing

```
$ yarn watch
```
Want to help develop and contribute to the AI Studio extension?

If you are not live editing the frontend package, you can just run (from the AI Studio sources folder):
You can use `yarn watch --extension-folder` from the Podman Desktop directory to automatically rebuild and test the AI Studio extension:

```
$ yarn build
```sh
git clone https://github.com/containers/podman-desktop
git clone https://github.com/projectatomic/ai-studio
cd ai-studio
jeffmaury marked this conversation as resolved.
Show resolved Hide resolved
yarn build
cd ../podman-desktop
yarn watch --extension-folder ../ai-studio/packages/backend
```

From the Podman Desktop sources folder:
If you are live editing the frontend package, from packages/frontend folder:

```
$ yarn watch --extension-folder path-to-extension-sources-folder/packages/backend
$ yarn watch
```

## Cleaning up resources
### Cleaning up resources

We'll be adding a way in AI Lab to let a user cleanup the environment: see issue https://github.com/projectatomic/ai-studio/issues/469.
For the time being, please consider the following actions:
Expand All @@ -42,17 +102,17 @@ For the time being, please consider the following actions:
4. Remove the containers related to AI
5. Cleanup your local clone of the recipes: `$HOME/podman-desktop/ai-studio`

## Providing a custom catalog
### Providing a custom catalog

The extension provides a default catalog, but you can build your own catalog by creating a file `$HOME/podman-desktop/ai-studio/catalog.json`.

The catalog provides lists of categories, recipes, and models.

Each recipe can belong to one or several categories. Each model can be used by one or several recipes.

The format of the catalog is not stable nor versioned yet, you can see the current catalog's format [in the sources of the extension](https://github.com/projectatomic/studio-extension/blob/main/packages/backend/src/ai.json).

## Packaging sample applications
### Packaging sample applications

Sample applications may be added to the catalog. See [packaging guide](PACKAGING-GUIDE.md) for detailed information.

Expand Down