From 6d3d5aa40c773651a9dc77201601d56f978935e3 Mon Sep 17 00:00:00 2001 From: Louis Date: Thu, 23 May 2024 12:53:12 +0700 Subject: [PATCH] chore: update README.md --- README.md | 98 +++++++++------------------------------------ cortex-js/README.md | 28 ++++++++----- 2 files changed, 36 insertions(+), 90 deletions(-) diff --git a/README.md b/README.md index aef3ae290..e6b453e98 100644 --- a/README.md +++ b/README.md @@ -1,76 +1,7 @@ # Cortex Monorepo -This monorepo contains two projects: CortexJS and CortexCPP. - -## CortexJS: Stateful Business Backend - -* All of the stateful endpoints: - + /threads - + /messages - + /models - + /runs - + /vector_store - + /settings - + /?auth - + … -* Database & Filesystem -* API Gateway -* Authentication & Authorization -* Observability - -## CortexCPP: Stateless Embedding Backend - -* All of the high performance, stateless endpoints: - + /chat/completion - + /audio - + /fine_tuning - + /embeddings - + /load_model - + /unload_model -* Kernel - Hardware Recognition - -## Project Structure - -``` -. -├── cortex-js/ -│ ├── package.json -│ ├── README.md -│ ├── Dockerfile -│ ├── docker-compose.yml -│ ├── src/ -│ │ ├── controllers/ -│ │ ├── modules/ -│ │ ├── services/ -│ │ └── ... -│ └── ... -├── cortex-cpp/ -│ ├── app/ -│ │ ├── controllers/ -│ │ ├── models/ -│ │ ├── services/ -│ │ ├── ?engines/ -│ │ │ ├── llama.cpp -│ │ │ ├── tensorrt-llm -│ │ │ └── ... -│ │ └── ... -│ ├── CMakeLists.txt -│ ├── config.json -│ ├── Dockerfile -│ ├── docker-compose.yml -│ ├── README.md -│ └── ... -├── scripts/ -│ └── ... -├── README.md -├── package.json -├── Dockerfile -├── docker-compose.yml -└── docs/ - └── ... -``` - -# Install +# Installation + ## Prerequisites ### **Dependencies** @@ -91,17 +22,18 @@ Before installation, ensure that you have installed the following: Ensure that your system meets the following requirements to run Cortex: - **OS**: - - MacOSX 13.6 or higher. - - Windows 10 or higher. - - Ubuntu 12.04 and later. + - MacOSX 13.6 or higher. + - Windows 10 or higher. + - Ubuntu 12.04 and later. - **RAM (CPU Mode):** - - 8GB for running up to 3B models. - - 16GB for running up to 7B models. - - 32GB for running up to 13B models. + - 8GB for running up to 3B models. + - 16GB for running up to 7B models. + - 32GB for running up to 13B models. - **VRAM (GPU Mode):** - - 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. - - 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. - - 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. + + - 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. + - 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. + - 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. - **Disk**: At least 10GB for app and model download. @@ -152,6 +84,7 @@ cortex init > Nvidia Others (Vulkan) ``` + 3. Select CPU instructions (will be deprecated soon). ```bash @@ -165,22 +98,27 @@ cortex init 2. Once downloaded, Cortex is ready to use! ### Step 4: Pull a model + From HuggingFace + ```bash cortex pull janhq/phi-3-medium-128k-instruct-GGUF ``` From Jan Hub (TBD) + ```bash cortex pull llama3 ``` ### Step 5: Chat + ```bash cortex run janhq/phi-3-medium-128k-instruct-GGUF ``` ## Run as an API server + ```bash cortex serve ``` diff --git a/cortex-js/README.md b/cortex-js/README.md index 62a9ae278..4ec7ed411 100644 --- a/cortex-js/README.md +++ b/cortex-js/README.md @@ -1,4 +1,5 @@ -# Install +# Installation + ## Prerequisites ### **Dependencies** @@ -19,17 +20,18 @@ Before installation, ensure that you have installed the following: Ensure that your system meets the following requirements to run Cortex: - **OS**: - - MacOSX 13.6 or higher. - - Windows 10 or higher. - - Ubuntu 12.04 and later. + - MacOSX 13.6 or higher. + - Windows 10 or higher. + - Ubuntu 12.04 and later. - **RAM (CPU Mode):** - - 8GB for running up to 3B models. - - 16GB for running up to 7B models. - - 32GB for running up to 13B models. + - 8GB for running up to 3B models. + - 16GB for running up to 7B models. + - 32GB for running up to 13B models. - **VRAM (GPU Mode):** - - 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. - - 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. - - 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. + + - 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. + - 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. + - 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. - **Disk**: At least 10GB for app and model download. @@ -80,6 +82,7 @@ cortex init > Nvidia Others (Vulkan) ``` + 3. Select CPU instructions (will be deprecated soon). ```bash @@ -93,22 +96,27 @@ cortex init 2. Once downloaded, Cortex is ready to use! ### Step 4: Pull a model + From HuggingFace + ```bash cortex pull janhq/phi-3-medium-128k-instruct-GGUF ``` From Jan Hub (TBD) + ```bash cortex pull llama3 ``` ### Step 5: Chat + ```bash cortex run janhq/phi-3-medium-128k-instruct-GGUF ``` ## Run as an API server + ```bash cortex serve ```