From 4f4c68584d2620781a89118664d6b21db02718f5 Mon Sep 17 00:00:00 2001 From: Nicole Zhu <69952136+0xSage@users.noreply.github.com> Date: Mon, 24 Jun 2024 18:57:21 -0700 Subject: [PATCH] docs: Update README.md --- README.md | 88 ++++++++++--------------------------------------------- 1 file changed, 16 insertions(+), 72 deletions(-) diff --git a/README.md b/README.md index aa2e3804c..3107bdb9d 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# Cortex - CLI +# Cortex

cortex-cpplogo

@@ -11,86 +11,27 @@ > ⚠️ **Cortex is currently in Development**: Expect breaking changes and bugs! ## About -Cortex is an openAI-compatible local AI server that developers can use to build LLM apps. It is packaged with a Docker-inspired command-line interface and a Typescript client library. It can be used as a standalone server, or imported as a library. +Cortex is an OpenAI-compatible AI engine that developers can use to build LLM apps. It is packaged with a Docker-inspired command-line interface and client libraries. It can be used as a standalone server or imported as a library. -Cortex currently supports two inference engines: +Cortex currently supports 3 inference engines: - Llama.cpp +- ONNX Runtime - TensorRT-LLM -> Read more about Cortex at https://jan.ai/cortex - ## Quicklinks -**Cortex**: -- [Website](https://jan.ai/) -- [GitHub](https://github.com/janhq/cortex) -- [User Guides](https://jan.ai/cortex) -- [API reference](https://jan.ai/api-reference) - -## Prerequisites - -### **Dependencies** - -Before installation, ensure that you have installed the following: - -- **Node.js**: version 18 and above is required to run the installation. -- **NPM**: Needed to manage packages. -- **CPU Instruction Sets**: Available for download from the [Cortex GitHub Releases](https://github.com/janhq/cortex/releases) page. - - ->💡 The **CPU instruction sets** are not required for the initial installation of Cortex. This dependency will be automatically installed during the Cortex initialization if they are not already on your system. - -### **Hardware** - -Ensure that your system meets the following requirements to run Cortex: - -- **OS**: - - MacOSX 13.6 or higher. - - Windows 10 or higher. - - Ubuntu 12.04 and later. -- **RAM (CPU Mode):** - - 8GB for running up to 3B models. - - 16GB for running up to 7B models. - - 32GB for running up to 13B models. -- **VRAM (GPU Mode):** - - - 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. - - 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. - - 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. - -- **Disk**: At least 10GB for app and model download. +- [Homepage](https://cortex.jan.ai/) +- [Docs](https://cortex.jan.ai/docs/) ## Quickstart -To install Cortex CLI, follow the steps below: -1. Install the Cortex NPM package globally: -``` bash -npm i -g @janhq/cortex -``` -> Cortex automatically detects your CPU and GPU, downloading the appropriate CPU instruction sets and required dependencies to optimize GPU performance. -2. Download a GGUF model from Hugging Face: -``` bash -# Pull a model most compatible with your hardware -cortex pull llama3 - -# Pull a specific variant with `repo_name:branch` -cortex pull llama3:7b -# Pull a model with the HuggingFace `model_id` -cortex pull microsoft/Phi-3-mini-4k-instruct-gguf -``` -3. Load the model: -``` bash -cortex models start llama3:7b -``` +Visit [Quickstart](https://cortex.jan.ai/docs/quickstart) to get started. -4. Start chatting with the model: ``` bash -cortex chat tell me a joke +npm i -g @janhq/cortex +cortex run llama3 ``` - - -## Run as an API server To run Cortex as an API server: ```bash cortex serve @@ -120,9 +61,10 @@ chmod +x '[path-to]/cortex/cortex-js/dist/src/command.js' npm link ``` -## Cortex CLI Command -The following CLI commands are currently available: -> ⚠️ **Cortex is currently in Development**: More commands will be added soon! +## Cortex CLI Commands + +The following CLI commands are currently available. +See [CLI Reference Docs](https://cortex.jan.ai/docs/cli) for more information. ```bash @@ -142,6 +84,7 @@ The following CLI commands are currently available: models stop Stop a specified model. models update Update the configuration of a specified model. ``` + ## Uninstall Cortex Run the following command to uninstall Cortex globally on your machine: @@ -150,7 +93,8 @@ Run the following command to uninstall Cortex globally on your machine: # Uninstall globally using NPM npm uninstall -g @janhq/cortex ``` + ## Contact Support - For support, please file a GitHub ticket. - For questions, join our Discord [here](https://discord.gg/FTk2MvZwJH). -- For long-form inquiries, please email [hello@jan.ai](mailto:hello@jan.ai). \ No newline at end of file +- For long-form inquiries, please email [hello@jan.ai](mailto:hello@jan.ai).