epic: Better Cortex Error Handling and Hardware Compatibility Checks

[freelerobot]

Spec

https://www.notion.so/jan-ai/Better-Error-Handling-and-Hardware-Compatibility-Checks-3d6944c3fb7b41429dee3e44432dc3a8?pvs=4

Update:

In Github issues, search Nitro - it is current the number one root cause of a majority of user experience issues when using Jan.

Let's focus on quality improvements to Nitro for a few sprints.

---

Previous Context

Problem:
Currently, Nitro just fails. The process terminates with no clear reasons.
Debugging entails asking users for 1000 lines of logs, and making a best guess on where we went wrong.

There are 2 failure states:

1. User system setup/dependencies issues, see bug: error loading model : undefined jan#1683
2. User input validation error (To alan's pt, this introduces additional logic and latency, so we may choose to prioritize 2)
3. Esoteric Nitro internal errors from llamacpp, drogon, or any part of its subroutines.

Ideal:

• Clearly defined error states/types in Nitro
• Nitro terminates with the correct signals
• Invocation code gets better errors from Nitro from callback or something...

Tasklist

• Implement Computer Specification Check in Onboarding Process
• Handle Nitro Exit Code to Display Proper Error Message => depending on epic: Rebase Nitro on Llama.cpp (as Jan.cpp) jan#2103

Context:
Also @alan can you queue up an epic/feat for nitro error handling and graceful failures? i.e. people want Nitro to expect common errors, define clear error enums, and fail gracefully, propagating errors to whatever is dependent on it

[tikikun]

I want to raise up a few points.

Nitro did have message, if not then we wouldn't be able to debug from user log in the first place.

Nitro crashing, failing come at advantage since it can be re-opened as a new process. We need to make sure this #367 is properly done so that thing can be handled in a separation of concern manner.

We are highly dependent on a few low-level project (llama cpp whisper and ggml) which has its own way of handling their error. Handle it from a low level standpoint means that we will also need to maintain the dependencies.

[tikikun]

We must solve the correct priority issue first, nitro node itself should be isolated as a process manager for nitro before moving on to fix harder issue.

[tikikun]

Hi @Van-QA since we already resolved the nitro-node (pending @louis-jan confirmation you can check with him) we just have one single issue in the way llama cpp handle error using abort(), we can just use this epic only? @louis-jan is quite familiar with this issue so can provide recommendation, we can directly fix it in jan.cpp