Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

bug: cortex chat is a non-interactive request & should stdout to terminal #1313

@freelerobot

Description

@freelerobot

Cortex version

v83

Describe the Bug

Right now cortex chat starts an interactive shell.
Originally, we only intended for run to start the shell.

For consistency, cortex chat should mirror the other cortex subcommands like models, engines: ping the api endpoint, and return a response within the standard terminal.

But cmiiw @dan-homebrew

Steps to Reproduce

  1. cortex start the server
  2. cortex models start ... the model
  3. cortex chat ...
  4. expected: chat request response within terminal

Screenshots / Logs

Instead, we enter an interactive chat shell.

(base) PS C:\Users\n\cortexcpp-nightly\models> cortex-nightly chat mistral -m "hello"
Inorder to exit, type `exit()`
> Hello! How can I assist you today?
>

What is your OS?

  • MacOS
  • Windows
  • Linux

What engine are you running?

  • cortex.llamacpp (default)
  • cortex.tensorrt-llm (Nvidia GPUs)
  • cortex.onnx (NPUs, DirectML)

Metadata

Metadata

Assignees

Labels

P1: importantImportant feature / fixtype: bugSomething isn't working

Type

No type

Projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions