Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Readme and the URL for the API documentation #779

Merged
merged 76 commits into from
Jun 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
86e6444
API: Added the description
irfanpena Jun 18, 2024
b21ef85
API: Added the description
irfanpena Jun 18, 2024
4ba863e
Update per comment
irfanpena Jun 18, 2024
a71d8e9
Merge branch 'pena-patch' of https://github.com/janhq/cortex into pen…
irfanpena Jun 18, 2024
89d2ee1
Update
irfanpena Jun 18, 2024
37edce8
Merge branch 'dev' of https://github.com/janhq/cortex into pena-patch
irfanpena Jun 19, 2024
b7feed6
Merge branch 'dev' of https://github.com/janhq/cortex into pena-patch
irfanpena Jun 26, 2024
142d58f
Update the readme and the server URL for the API documentation
irfanpena Jun 26, 2024
5bdbd27
Update the description
irfanpena Jun 26, 2024
3b8be74
Merge branch 'dev' of https://github.com/janhq/cortex into pena-patch
irfanpena Jun 26, 2024
fb88a0c
update @janhq/cortex -> cortexso
irfanpena Jun 26, 2024
d52da04
add prerequisites section
irfanpena Jun 26, 2024
546d625
update per feedback
irfanpena Jun 26, 2024
6bb0fdc
add brew installation
irfanpena Jun 26, 2024
fc98be0
Merge branch 'dev' into pena-patch
irfanpena Jun 26, 2024
70e4a41
Separate npm and brew
irfanpena Jun 26, 2024
2ae3988
Merge branch 'pena-patch' of https://github.com/janhq/cortex into pen…
irfanpena Jun 26, 2024
d667aba
nits
irfanpena Jun 26, 2024
0b9f291
edit the URL
irfanpena Jun 26, 2024
8e11b14
update the desc
irfanpena Jun 26, 2024
1135657
Chore: test CLI test with serve --detach (#722)
Van-QA Jun 18, 2024
331ce40
API: Added the description
irfanpena Jun 18, 2024
83769b7
Update
irfanpena Jun 18, 2024
dc187bc
feat: cortex single executable binary (#715)
louis-jan Jun 18, 2024
77615e6
feat: ship ONNX runtime on Windows (#716)
louis-jan Jun 18, 2024
1c0c4b5
feat: `cortex-cpp` with `cortex.tensorrt-llm` (#719)
CameronNg Jun 18, 2024
4d6c1ff
fix: cortex serve attached mode outputs nothing (#727)
louis-jan Jun 18, 2024
409b15c
chore: update env for cuda toolkit dependencies and tensorrt-llm (#728)
louis-jan Jun 19, 2024
e612594
fix: prevent error loop, create crash report file if not exist
marknguyen1302 Jun 19, 2024
bee66fb
Update cortex.llamacpp nightly to version 0.1.17-19.06.24
github-actions[bot] Jun 19, 2024
38c1817
chore: update models to extends from openai
namchuai Jun 17, 2024
ce2cbcc
fix: update message object
namchuai Jun 17, 2024
d6431a9
update model
namchuai Jun 17, 2024
65a47d9
update
namchuai Jun 17, 2024
172cab4
update
namchuai Jun 17, 2024
c851de8
update
namchuai Jun 18, 2024
c0af273
update
namchuai Jun 19, 2024
8ef236f
adding clean thread api
namchuai Jun 20, 2024
6589d59
fix: test cases
namchuai Jun 20, 2024
65208dd
chore: change --host to -- address (#736)
Van-QA Jun 20, 2024
9fb436a
fix: add telemetry timeout, remove debug log
marknguyen1302 Jun 21, 2024
1971c19
chore: build cortex binary (#726)
louis-jan Jun 21, 2024
c0e4c91
refactor: shorten cortex setup - save cortex cpp serve configs (#747)
louis-jan Jun 22, 2024
cbb594e
fix: bun - cortex serve does not work (#751)
louis-jan Jun 24, 2024
3afaa3c
fix: change ci runner to address visual sdk issue (#752)
louis-jan Jun 24, 2024
39f6019
chore: fix cortex bundle relative path lookup (#758)
louis-jan Jun 24, 2024
c94af4a
fix: remove build target from build script (#759)
louis-jan Jun 24, 2024
634fe6c
fix: update cortex package.json version (#760)
louis-jan Jun 24, 2024
5fdd9c2
fix: cortex version on windows (#761)
louis-jan Jun 24, 2024
5ce8827
Correct tag for cortex binary build (#762)
hiento09 Jun 24, 2024
ce95358
Windows enable ccache (#763)
hiento09 Jun 24, 2024
2f12165
docs: Update README.md (#767)
0xSage Jun 25, 2024
cfb9d08
chore: remove python e2e testing (#731)
vansangpfiev Jun 25, 2024
bdcd108
fix: remove build target (#769)
louis-jan Jun 25, 2024
0cd2d53
chore: use customized build for vulkan (#768)
vansangpfiev Jun 25, 2024
8d0f678
feat: add engine pull support for tensorrt-llm (#765)
louis-jan Jun 25, 2024
1b2cb35
fix: remove patch-package dependency (#770)
louis-jan Jun 25, 2024
0be10c5
fix: cortex node suppress warning (#771)
louis-jan Jun 25, 2024
6cbcb49
chore: rename npm package cortexso (#772)
louis-jan Jun 25, 2024
d482c69
chore: Add Swagger UI Log Message and Update `README.md` with API Acc…
l2D Jun 25, 2024
e143638
Update the readme and the server URL for the API documentation
irfanpena Jun 26, 2024
2108c14
Update the description
irfanpena Jun 26, 2024
d5ae28f
chore: add dependency instructions (#773)
louis-jan Jun 26, 2024
8ccc304
update @janhq/cortex -> cortexso
irfanpena Jun 26, 2024
8bb43cd
add prerequisites section
irfanpena Jun 26, 2024
37222fc
update per feedback
irfanpena Jun 26, 2024
f626e42
add brew installation
irfanpena Jun 26, 2024
65a5d39
Separate npm and brew
irfanpena Jun 26, 2024
1dba491
chore: add MacOS pkg installer (#774)
louis-jan Jun 26, 2024
afd79ca
nits
irfanpena Jun 26, 2024
0944227
edit the URL
irfanpena Jun 26, 2024
f939ef5
update the desc
irfanpena Jun 26, 2024
f91fd54
Merge branch 'dev' of https://github.com/janhq/cortex into pena-patch
irfanpena Jun 26, 2024
49061d4
Merge branch 'pena-patch' of https://github.com/janhq/cortex into pen…
irfanpena Jun 26, 2024
124b3e5
added `v1` URL
irfanpena Jun 26, 2024
d5f02e4
Merge branch 'dev' into pena-patch
irfanpena Jun 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 39 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,45 @@ Cortex currently supports 3 inference engines:
- [Docs](https://cortex.jan.ai/docs/)

## Quickstart
### Prerequisites
Ensure that your system meets the following requirements to run Cortex:
- **Dependencies**:
- **Node.js**: version 18 and above is required to run the installation.
- **NPM**: Needed to manage packages.
- **CPU Instruction Sets**: Available for download from the [Cortex GitHub Releases](https://github.com/janhq/cortex/releases) page.
- **OS**:
- MacOSX 13.6 or higher.
- Windows 10 or higher.
- Ubuntu 22.04 and later.

> Visit [Quickstart](https://cortex.jan.ai/docs/quickstart) to get started.

### NPM
``` bash
# Install using NPM
npm i -g cortexso
louis-jan marked this conversation as resolved.
Show resolved Hide resolved
# Install using Brew
brew tap janhq/cortexso
brew install cortexso
# Run model
cortex run llama3
# To uninstall globally using NPM
npm uninstall -g cortexso
```

Visit [Quickstart](https://cortex.jan.ai/docs/quickstart) to get started.

### Homebrew
``` bash
npm i -g @janhq/cortex
# Install using Brew
brew tap janhq/cortexso
brew install cortexso
# Run model
cortex run llama3
# To uninstall using Brew
brew uninstall cortexso
brew untap janhq/cortexso
```
> You can also install Cortex using the Cortex Installer available on [GitHub Releases](https://github.com/janhq/cortex/releases).

To run Cortex as an API server:
```bash
cortex serve
Expand Down Expand Up @@ -90,18 +122,13 @@ See [CLI Reference Docs](https://cortex.jan.ai/docs/cli) for more information.
models start Start a specified model.
models stop Stop a specified model.
models update Update the configuration of a specified model.
```

## Uninstall Cortex

Run the following command to uninstall Cortex globally on your machine:

```
# Uninstall globally using NPM
npm uninstall -g @janhq/cortex
louis-jan marked this conversation as resolved.
Show resolved Hide resolved
benchmark Benchmark and analyze the performance of a specific AI model using your system.
presets Show all the available model presets within Cortex.
telemetry Retrieve telemetry logs for monitoring and analysis.
```

## Contact Support
- For support, please file a GitHub ticket.
- For questions, join our Discord [here](https://discord.gg/FTk2MvZwJH).
- For long-form inquiries, please email [hello@jan.ai](mailto:hello@jan.ai).

119 changes: 47 additions & 72 deletions cortex-js/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Cortex - CLI
# Cortex
<p align="center">
<img alt="cortex-cpplogo" src="https://raw.githubusercontent.com/janhq/cortex/dev/assets/cortex-banner.png">
</p>
Expand All @@ -11,91 +11,71 @@
> ⚠️ **Cortex is currently in Development**: Expect breaking changes and bugs!

## About
Cortex is an openAI-compatible local AI server that developers can use to build LLM apps. It is packaged with a Docker-inspired command-line interface and a Typescript client library. It can be used as a standalone server, or imported as a library.
Cortex is an OpenAI-compatible AI engine that developers can use to build LLM apps. It is packaged with a Docker-inspired command-line interface and client libraries. It can be used as a standalone server or imported as a library.

Cortex currently supports two inference engines:
Cortex currently supports 3 inference engines:

- Llama.cpp
- ONNX Runtime
- TensorRT-LLM

> Read more about Cortex at https://jan.ai/cortex

## Quicklinks
Cortex
- [Website](https://jan.ai/)
- [GitHub](https://github.com/janhq/cortex)
- [User Guides](https://jan.ai/cortex)
- [API reference](https://jan.ai/api-reference)

## Prerequisites

### **Dependencies**

Before installation, ensure that you have installed the following:
- **Node.js**: version 18 and above is required to run the installation.
- **NPM**: Needed to manage packages.
- **CPU Instruction Sets**: Available for download from the [Cortex GitHub Releases](https://github.com/janhq/cortex/releases) page.


>💡 The **CPU instruction sets** are not required for the initial installation of Cortex. This dependency will be automatically installed during the Cortex initialization if they are not already on your system.


### **Hardware**
- [Homepage](https://cortex.jan.ai/)
- [Docs](https://cortex.jan.ai/docs/)

## Quickstart
### Prerequisites
Ensure that your system meets the following requirements to run Cortex:

- **Dependencies**:
- **Node.js**: version 18 and above is required to run the installation.
- **NPM**: Needed to manage packages.
- **CPU Instruction Sets**: Available for download from the [Cortex GitHub Releases](https://github.com/janhq/cortex/releases) page.
- **OS**:
- MacOSX 13.6 or higher.
- Windows 10 or higher.
- Ubuntu 22.04 and later.
- **RAM (CPU Mode):**
- 8GB for running up to 3B models.
- 16GB for running up to 7B models.
- 32GB for running up to 13B models.
- **VRAM (GPU Mode):**

- 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
- 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
- 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
> Visit [Quickstart](https://cortex.jan.ai/docs/quickstart) to get started.

- **Disk**: At least 10GB for app and model download.

## Quickstart
To install Cortex CLI, follow the steps below:
1. Install the Cortex NPM package globally:
### NPM
``` bash
# Install using NPM
npm i -g cortexso
```
> Cortex automatically detects your CPU and GPU, downloading the appropriate CPU instruction sets and required dependencies to optimize GPU performance.

2. Download a GGUF model from Hugging Face:
``` bash
# Pull a model most compatible with your hardware
cortex pull llama3

# Pull a specific variant with `repo_name:branch`
cortex pull llama3:7b

# Pull a model with the HuggingFace `model_id`
cortex pull microsoft/Phi-3-mini-4k-instruct-gguf
```
3. Load the model:
``` bash
cortex models start llama3:7b
# Install using Brew
brew tap janhq/cortexso
brew install cortexso
# Run model
cortex run llama3
# To uninstall globally using NPM
npm uninstall -g cortexso
```

4. Start chatting with the model:
### Homebrew
``` bash
cortex chat tell me a joke
# Install using Brew
brew tap janhq/cortexso
brew install cortexso
# Run model
cortex run llama3
# To uninstall using Brew
brew uninstall cortexso
brew untap janhq/cortexso
```
> You can also install Cortex using the Cortex Installer available on [GitHub Releases](https://github.com/janhq/cortex/releases).


## Run as an API server
To run Cortex as an API server:
```bash
cortex serve

# Output
# Started server at http://localhost:1337
# Swagger UI available at http://localhost:1337/api
```

You can now access the Cortex API server at `http://localhost:1337`,
and the Swagger UI at `http://localhost:1337/api`.

## Build from Source

To install Cortex from the source, follow the steps below:
Expand All @@ -120,9 +100,10 @@ chmod +x '[path-to]/cortex/cortex-js/dist/src/command.js'
npm link
```

## Cortex CLI Command
The following CLI commands are currently available:
> ⚠️ **Cortex is currently in Development**: More commands will be added soon!
## Cortex CLI Commands

The following CLI commands are currently available.
See [CLI Reference Docs](https://cortex.jan.ai/docs/cli) for more information.

```bash

Expand All @@ -141,18 +122,12 @@ The following CLI commands are currently available:
models start Start a specified model.
models stop Stop a specified model.
models update Update the configuration of a specified model.
engines Execute a specified command related to engines.
engines list List all available engines.
benchmark Benchmark and analyze the performance of a specific AI model using your system.
presets Show all the available model presets within Cortex.
telemetry Retrieve telemetry logs for monitoring and analysis.
```
## Uninstall Cortex

Run the following command to uninstall Cortex globally on your machine:

```
# Uninstall globally using NPM
npm uninstall -g cortexso
```
## Contact Support
- For support, please file a GitHub ticket.
- For questions, join our Discord [here](https://discord.gg/FTk2MvZwJH).
- For long-form inquiries, please email [hello@jan.ai](mailto:hello@jan.ai).
- For long-form inquiries, please email [hello@jan.ai](mailto:hello@jan.ai).
2 changes: 1 addition & 1 deletion cortex-js/src/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ export const getApp = async () => {
const config = new DocumentBuilder()
.setTitle('Cortex API')
.setDescription(
'Cortex API provides a command-line interface (CLI) for seamless interaction with large language models (LLMs). Fully compatible with the [OpenAI API](https://platform.openai.com/docs/api-reference), it enables straightforward command execution and management of LLM interactions.',
'Cortex API provides a command-line interface (CLI) for seamless interaction with large language models (LLMs). It is fully compatible with the [OpenAI API](https://platform.openai.com/docs/api-reference) and enables straightforward command execution and management of LLM interactions.',
)
.setVersion('1.0')
.addTag(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ export class StatusController {

@ApiOperation({
summary: "Get health status",
description: "Retrieves the health status of the Cortex's API endpoint server.",
description: "Retrieves the health status of your Cortex's system.",
})
@HttpCode(200)
@ApiResponse({
Expand Down
Loading