Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 16 additions & 72 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Cortex - CLI
# Cortex
<p align="center">
<img alt="cortex-cpplogo" src="https://raw.githubusercontent.com/janhq/cortex/dev/assets/cortex-banner.png">
</p>
Expand All @@ -11,86 +11,27 @@
> ⚠️ **Cortex is currently in Development**: Expect breaking changes and bugs!

## About
Cortex is an openAI-compatible local AI server that developers can use to build LLM apps. It is packaged with a Docker-inspired command-line interface and a Typescript client library. It can be used as a standalone server, or imported as a library.
Cortex is an OpenAI-compatible AI engine that developers can use to build LLM apps. It is packaged with a Docker-inspired command-line interface and client libraries. It can be used as a standalone server or imported as a library.

Cortex currently supports two inference engines:
Cortex currently supports 3 inference engines:

- Llama.cpp
- ONNX Runtime
- TensorRT-LLM

> Read more about Cortex at https://jan.ai/cortex

## Quicklinks
**Cortex**:
- [Website](https://jan.ai/)
- [GitHub](https://github.com/janhq/cortex)
- [User Guides](https://jan.ai/cortex)
- [API reference](https://jan.ai/api-reference)

## Prerequisites

### **Dependencies**

Before installation, ensure that you have installed the following:

- **Node.js**: version 18 and above is required to run the installation.
- **NPM**: Needed to manage packages.
- **CPU Instruction Sets**: Available for download from the [Cortex GitHub Releases](https://github.com/janhq/cortex/releases) page.


>💡 The **CPU instruction sets** are not required for the initial installation of Cortex. This dependency will be automatically installed during the Cortex initialization if they are not already on your system.


### **Hardware**

Ensure that your system meets the following requirements to run Cortex:

- **OS**:
- MacOSX 13.6 or higher.
- Windows 10 or higher.
- Ubuntu 12.04 and later.
- **RAM (CPU Mode):**
- 8GB for running up to 3B models.
- 16GB for running up to 7B models.
- 32GB for running up to 13B models.
- **VRAM (GPU Mode):**

- 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
- 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
- 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.

- **Disk**: At least 10GB for app and model download.
- [Homepage](https://cortex.jan.ai/)
- [Docs](https://cortex.jan.ai/docs/)

## Quickstart
To install Cortex CLI, follow the steps below:
1. Install the Cortex NPM package globally:
``` bash
npm i -g @janhq/cortex
```
> Cortex automatically detects your CPU and GPU, downloading the appropriate CPU instruction sets and required dependencies to optimize GPU performance.
2. Download a GGUF model from Hugging Face:
``` bash
# Pull a model most compatible with your hardware
cortex pull llama3

# Pull a specific variant with `repo_name:branch`
cortex pull llama3:7b

# Pull a model with the HuggingFace `model_id`
cortex pull microsoft/Phi-3-mini-4k-instruct-gguf
```
3. Load the model:
``` bash
cortex models start llama3:7b
```
Visit [Quickstart](https://cortex.jan.ai/docs/quickstart) to get started.

4. Start chatting with the model:
``` bash
cortex chat tell me a joke
npm i -g @janhq/cortex
cortex run llama3
```


## Run as an API server
To run Cortex as an API server:
```bash
cortex serve
Expand Down Expand Up @@ -120,9 +61,10 @@ chmod +x '[path-to]/cortex/cortex-js/dist/src/command.js'
npm link
```

## Cortex CLI Command
The following CLI commands are currently available:
> ⚠️ **Cortex is currently in Development**: More commands will be added soon!
## Cortex CLI Commands

The following CLI commands are currently available.
See [CLI Reference Docs](https://cortex.jan.ai/docs/cli) for more information.

```bash

Expand All @@ -142,6 +84,7 @@ The following CLI commands are currently available:
models stop Stop a specified model.
models update Update the configuration of a specified model.
```

## Uninstall Cortex

Run the following command to uninstall Cortex globally on your machine:
Expand All @@ -150,7 +93,8 @@ Run the following command to uninstall Cortex globally on your machine:
# Uninstall globally using NPM
npm uninstall -g @janhq/cortex
```

## Contact Support
- For support, please file a GitHub ticket.
- For questions, join our Discord [here](https://discord.gg/FTk2MvZwJH).
- For long-form inquiries, please email [hello@jan.ai](mailto:hello@jan.ai).
- For long-form inquiries, please email [hello@jan.ai](mailto:hello@jan.ai).