Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 87 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,84 @@
# PUMA

**PUMA** aims to be a lightweight, high-performance inference engine for local AI. *Play for fun.*
A lightweight, high-performance inference engine for local AI. *Play for fun.*
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description still contains the template placeholder Fixes # with no issue number/link filled in. Either link the actual issue (e.g., Fixes #123) or remove the placeholder so the PR metadata is accurate.

Copilot uses AI. Check for mistakes.

## Features

- 🚀 **Model Management** - Download and manage AI models from multiple providers
- **Model Management** - Download and manage AI models from model providers like Hugging Face
- **System Detection** - Automatic GPU detection and system information reporting
- **Local Caching** - Efficient model storage with custom cache directories
- **Multiple Providers** - Support for Hugging Face with ModelScope coming soon

## Quick Start
## Installation

### Install from source
### From Source

```bash
make build
```

The binary will be available as `./puma`.

## Quick Start

### 1. Download a Model

```bash
# From Hugging Face (default)
puma pull InftyAI/tiny-random-gpt2
```

Comment on lines +27 to +30
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Installation section says the binary will be available as ./puma, but the Quick Start examples invoke puma ... (which assumes it’s on PATH). Update the examples to use ./puma or add an explicit step to install/add the binary to PATH before using puma without a relative path.

Copilot uses AI. Check for mistakes.
### 2. List Downloaded Models

```bash
puma ls
```

### 3. Check System Information

```bash
puma info
```

Example output:
```
System Information:
Operating System: Darwin
Architecture: arm64
CPU Cores: 14
Total Memory: 36.00 GiB
GPU: Apple M4 Max (Metal) - 32 GPU cores

PUMA Information:
PUMA Version: 0.0.1
Cache Directory: ~/.puma/cache
Cache Size: 799.88 MiB
Models: 1
Running Models: 0
```

## Commands

| Command | Description |
|---------|-------------|
| `pull` | Download a model from a provider |
| `ls` | List local models |
| `ps` | List running models |
| `run` | Create and run a model |
| `stop` | Stop a running model |
| `rm` | Remove a model |
| `info` | Display system-wide information |
| `inspect` | Return detailed information about a model |
| `version` | Show PUMA version |
| `help` | Show help information |
| Command | Status | Description | Example |
|---------|--------|-------------|---------|
| `pull` | ✅ | Download a model from a provider | `puma pull InftyAI/tiny-random-gpt2` |
| `ls` | ✅ | List local models | `puma ls` |
| `ps` | 🚧 | List running models | `puma ps` |
| `run` | 🚧 | Create and run a model | `puma run InftyAI/tiny-random-gpt2` |
| `stop` | 🚧 | Stop a running model | `puma stop <model-id>` |
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The README shows puma run InftyAI/tiny-random-gpt2, but the current CLI defines run as a subcommand with no arguments. This example will fail with an “unexpected argument” error unless the CLI is updated to accept a model argument (or the README example is changed).

Suggested change
| `stop` | 🚧 | Stop a running model | `puma stop <model-id>` |
| `stop` | 🚧 | Stop a running model | `TBD` |

Copilot uses AI. Check for mistakes.
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The README shows puma stop <model-id>, but the current CLI defines stop as a subcommand with no arguments. Either update the README example to match the current interface or update the CLI to accept a model/service identifier.

Suggested change
| `stop` | 🚧 | Stop a running model | `puma stop <model-id>` |
| `stop` | 🚧 | Stop a running model | `puma stop` |

Copilot uses AI. Check for mistakes.
| `rm` | ✅ | Remove a model | `puma rm InftyAI/tiny-random-gpt2` |
| `info` | ✅ | Display system-wide information | `puma info` |
| `inspect` | 🚧 | Return detailed information about a model or service | `puma inspect InftyAI/tiny-random-gpt2` |
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The README shows puma inspect InftyAI/tiny-random-gpt2, but the current CLI defines inspect as a subcommand with no arguments. This example will fail unless inspect is implemented to accept a target argument, or the README is adjusted.

Suggested change
| `inspect` | 🚧 | Return detailed information about a model or service | `puma inspect InftyAI/tiny-random-gpt2` |
| `inspect` | 🚧 | Return detailed inspection information | `puma inspect` |

Copilot uses AI. Check for mistakes.
| `version` | ✅ | Show PUMA version | `puma version` |
| `help` | ✅ | Show help information | `puma help` |

## Configuration

PUMA stores models in `~/.puma/cache` by default. This location is used for all downloaded models and metadata.

## Supported Providers

- **Hugging Face** - Full support with custom cache directories
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Features list mentions “ModelScope coming soon”, but the “Supported Providers” section only lists Hugging Face. Consider adding a ModelScope “coming soon” entry (or removing the mention) so the README is internally consistent.

Suggested change
- **Hugging Face** - Full support with custom cache directories
- **Hugging Face** - Full support with custom cache directories
- **ModelScope** - Coming soon

Copilot uses AI. Check for mistakes.

## Development

Expand All @@ -43,7 +94,24 @@ make build
make test
```

### Supported Providers
### Project Structure

```
puma/
├── src/
│ ├── cli/ # Command-line interface
│ ├── downloader/ # Model download logic
│ ├── registry/ # Model registry management
│ ├── system/ # System detection (CPU, GPU, memory)
│ └── utils/ # Utility functions
├── Cargo.toml # Rust dependencies
└── Makefile # Build commands
```

## License

Apache-2.0

## Contributing

- ✅ **Hugging Face** - Full support with custom cache directories
- 🚧 **ModelScope** - Coming soon
[![Star History Chart](https://api.star-history.com/svg?repos=inftyai/puma&type=Date)](https://www.star-history.com/#inftyai/puma&Date)
Loading