Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 108 additions & 64 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,27 @@
# PUMA
<div align="center">

A lightweight, high-performance inference engine for local AI. *Play for fun.*
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/images/logo.png">
<source media="(prefers-color-scheme: light)" srcset="docs/images/logo.png">
<img alt="PUMA Logo" src="docs/images/puma-logo-light.svg" width="240">
</picture>

## Features
**A lightweight, high-performance inference engine for local AI**

- **Model Management** - Download and manage AI models from model providers like Hugging Face
- **System Detection** - Automatic GPU detection and system information reporting
- **Local Caching** - Efficient model storage with custom cache directories
- **Multiple Providers** - Support for Hugging Face with ModelScope coming soon
[![Stability: Active](https://img.shields.io/badge/stability-active-brightgreen.svg)](https://github.com/InftyAI/PUMA)
[![Latest Release](https://img.shields.io/github/v/release/InftyAI/PUMA)](https://github.com/InftyAI/PUMA/releases)

## Installation
</div>

## ✨ Features

🔧 **Model Management** - Download, cache, and organize AI models from Hugging Face

🔍 **Advanced Filtering** - Search models with regex patterns and SQL-style queries

### From Source
💻 **System Detection** - Automatic GPU detection and resource reporting

## Installation

```bash
make build
Expand All @@ -21,76 +31,108 @@ The binary will be available as `./puma`.

## Quick Start

### 1. Download a Model

```bash
# From Hugging Face (default)
puma pull InftyAI/tiny-random-gpt2
```
# Download a model
puma pull inftyai/tiny-random-gpt2

### 2. List Downloaded Models

```bash
# List all models
puma ls
```

### 3. Check System Information
# Inspect model details
puma inspect inftyai/tiny-random-gpt2
Comment on lines +35 to +42

```bash
# Check system info
puma info
```

Example output:
```
System Information:
Operating System: Darwin
Architecture: arm64
CPU Cores: 14
Total Memory: 36.00 GiB
GPU: Apple M4 Max (Metal) - 32 GPU cores

PUMA Information:
PUMA Version: 0.0.1
Cache Directory: ~/.puma/cache
Cache Size: 799.88 MiB
Models: 1
Running Models: 0
# Remove a model
puma rm inftyai/tiny-random-gpt2
```

## Commands

| Command | Status | Description | Example |
|---------|--------|-------------|---------|
| `pull` | ✅ | Download a model from a provider | `puma pull InftyAI/tiny-random-gpt2` |
| `ls` | ✅ | List local models | `puma ls` |
| `ps` | 🚧 | List running models | `puma ps` |
| `run` | 🚧 | Create and run a model | `puma run InftyAI/tiny-random-gpt2` |
| `stop` | 🚧 | Stop a running model | `puma stop <model-id>` |
| `rm` | ✅ | Remove a model | `puma rm InftyAI/tiny-random-gpt2` |
| `info` | ✅ | Display system-wide information | `puma info` |
| `inspect` | ✅ | Return detailed information about a model or service | `puma inspect InftyAI/tiny-random-gpt2` |
| `version` | ✅ | Show PUMA version | `puma version` |
| `help` | ✅ | Show help information | `puma help` |
| Command | Status | Description |
|---------|--------|-------------|
| `pull <model>` | ✅ | Download model from provider |
| `ls` | ✅ | List models (supports regex, label filters) |
| `inspect <model>` | ✅ | Show detailed model information |
| `rm <model>` | ✅ | Remove model and cache |
| `info` | ✅ | Display system information |
| `version` | ✅ | Show PUMA version |
| `ps` | 🚧 | List running models |
| `run` | 🚧 | Start model inference |
| `stop` | 🚧 | Stop running model |

## Configuration
## Advanced Usage

PUMA stores models in `~/.puma/cache` by default. This location is used for all downloaded models and metadata.
### Pattern Matching

## Supported Providers
```bash
# Substring match
puma ls qwen

- **Hugging Face** - Full support with custom cache directories
# Prefix match
puma ls "^inftyai/"

## Development
# Alternation
puma ls "llama-(2|3)"
```

### Build
### Label Filtering

```bash
make build
# Single filter
puma ls -l author=inftyai

# Multiple filters (AND condition)
puma ls -l author=inftyai,license=mit

# Combine pattern + filter
puma ls llama -l author=meta
```

### Test
**Available filters:** `author`, `task`, `license`, `provider`, `model_series`

### Inspect Output

```bash
$ puma inspect inftyai/tiny-random-gpt2

name: inftyai/tiny-random-gpt2
kind: model
spec:
author: inftyai
task: text-generation
license: MIT
model_series: gpt2
context_window: 2.05K
safetensors:
total: 7.00B
parameters:
f32: 7.00B
artifact:
provider: huggingface
revision: abc123de
size: 1.24 GB
cache_path: ~/.puma/cache/...
Comment on lines +100 to +116
status:
created: 2 hours ago
updated: 2 hours ago
```

## Model Management

- **Database:** `~/.puma/models.db` (SQLite)
- **Cache:** `~/.puma/cache/` (model files)

Comment on lines +122 to +126
Models are stored with lowercase names for case-insensitive matching.

## Development

```bash
# Build
make build

# Run tests (67 unit + 22 integration)
make test
```

Expand All @@ -99,19 +141,21 @@ make test
```
puma/
├── src/
│ ├── cli/ # Command-line interface
│ ├── downloader/ # Model download logic
│ ├── registry/ # Model registry management
│ ├── system/ # System detection (CPU, GPU, memory)
│ └── utils/ # Utility functions
├── Cargo.toml # Rust dependencies
└── Makefile # Build commands
│ ├── cli/ # Command implementations (ls, rm, inspect)
│ ├── downloader/ # HuggingFace download logic
│ ├── registry/ # Model registry & metadata
│ ├── storage/ # SQLite storage backend
│ ├── system/ # System info detection
│ └── utils/ # Formatting & helpers
├── tests/ # Integration tests
├── Cargo.toml # Rust dependencies
└── Makefile # Build commands
```

## License

Apache-2.0

## Contributing
## Star History

[![Star History Chart](https://api.star-history.com/svg?repos=inftyai/puma&type=Date)](https://www.star-history.com/#inftyai/puma&Date)
Binary file added docs/images/logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/logo2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading