You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Steps to reproduce
Steps to reproduce the behavior:
Download model
Click Use
Chat
It shows model cannot load
Check the log and see the error
ERROR Could not load engine: Could not load library "./engines/cortex.llamacpp/libengine.dylib"
Expected behavior
Model starts and can chat normally
Screenshots
If applicable, add screenshots to help explain your issue.
Environment details
Operating System: MacOS 14.4.1
Jan Version: Jan v0.4.12-435
Processor: Apple M3
RAM: 32GB RAM
Any additional relevant hardware specifics: [e.g., Graphics card, SSD/HDD]
Logs
2024-05-17T03:26:09.120Z [SPECS]::Version: 0.4.12-4352024-05-17T03:26:09.121Z [SPECS]::CPUs: [{"model":"Apple M3 Pro","speed":24,"times":{"user":188596440,"nice":0,"sys":125261940,"idle":889957730,"irq":0}},{"model":"Apple M3 Pro","speed":24,"times":{"user":159653960,"nice":0,"sys":100688510,"idle":949465300,"irq":0}},{"model":"Apple M3 Pro","speed":24,"times":{"user":118884900,"nice":0,"sys":68591080,"idle":1030493130,"irq":0}},{"model":"Apple M3 Pro","speed":24,"times":{"user":82693540,"nice":0,"sys":43202680,"idle":1098255630,"irq":0}},{"model":"Apple M3 Pro","speed":24,"times":{"user":62060280,"nice":0,"sys":29563860,"idle":1136126430,"irq":0}},{"model":"Apple M3 Pro","speed":24,"times":{"user":49663330,"nice":0,"sys":22410700,"idle":1157780260,"irq":0}},{"model":"Apple M3 Pro","speed":24,"times":{"user":41871120,"nice":0,"sys":10771500,"idle":1180387850,"irq":0}},{"model":"Apple M3 Pro","speed":24,"times":{"user":40673060,"nice":0,"sys":10549550,"idle":1182069780,"irq":0}},{"model":"Apple M3 Pro","speed":24,"times":{"user":40293510,"nice":0,"sys":10452250,"idle":1182733880,"irq":0}},{"model":"Apple M3 Pro","speed":24,"times":{"user":40033180,"nice":0,"sys":10435860,"idle":1183089040,"irq":0}},{"model":"Apple M3 Pro","speed":24,"times":{"user":39877320,"nice":0,"sys":10460290,"idle":1183293120,"irq":0}}]
2024-05-17T03:26:09.121Z [SPECS]::Endianness: LE
2024-05-17T03:26:09.121Z [SPECS]::Free Mem: 1661222912
2024-05-17T03:26:09.121Z [SPECS]::Total Mem: 38654705664
2024-05-17T03:26:09.121Z [SPECS]::OS Version: Darwin Kernel Version 23.4.0: Fri Mar 15 00:12:25 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6030
2024-05-17T03:26:09.121Z [SPECS]::OS Platform: darwin
2024-05-17T03:26:09.121Z [SPECS]::Machine: arm64
2024-05-17T03:26:09.121Z [SPECS]::Parallelism: 11
2024-05-17T03:26:09.121Z [CORTEX]::CPU information - 11
2024-05-17T03:26:09.121Z [CORTEX]::Debug: Request to kill cortex
2024-05-17T03:26:09.121Z [SPECS]::OS Release: 23.4.0
2024-05-17T03:26:09.138Z [CORTEX]::Debug: cortex process is terminated
2024-05-17T03:26:09.138Z [CORTEX]::Debug: Spawning cortex subprocess...
2024-05-17T03:26:09.139Z [APP]::/Users/hiro/jan/extensions/@janhq/inference-cortex-extension/dist/bin/mac-arm64
2024-05-17T03:26:09.139Z [CORTEX]::Debug: Spawn cortex at path: /Users/hiro/jan/extensions/@janhq/inference-cortex-extension/dist/bin/mac-arm64/cortex-cpp, and args: 1,127.0.0.1,3928
2024-05-17T03:26:09.559Z [CORTEX]::Debug: cortex is ready
2024-05-17T03:26:09.559Z [CORTEX]::Debug: Loading model with params {"cpu_threads":11,"ctx_len":2048,"prompt_template":"<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{system_message}<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n","llama_model_path":"/Users/hiro/jan/models/llama3-8b-instruct/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf","ngl":32,"system_prompt":"<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n","user_prompt":"<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n","ai_prompt":"<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n","model":"llama3-8b-instruct"}
2024-05-17T03:26:09.563Z [CORTEX]::Debug: 20240517 03:26:09.512154 UTC 18118032 INFO cortex-cpp version: default_version - main.cc:50
20240517 03:26:09.513108 UTC 18118032 INFO cortex.llamacpp version: 0.1.2 - main.cc:55
20240517 03:26:09.513110 UTC 18118032 INFO Server started, listening at: 127.0.0.1:3928 - main.cc:58
20240517 03:26:09.513112 UTC 18118032 INFO Please load your model - main.cc:59
20240517 03:26:09.513120 UTC 18118032 INFO Number of thread is:11 - main.cc:66
20240517 03:26:09.563100 UTC 18118054 ERROR Could not load engine: Could not load library "./engines/cortex.llamacpp/libengine.dylib"
dlopen(./engines/cortex.llamacpp/libengine.dylib, 0x0006): tried: './engines/cortex.llamacpp/libengine.dylib' (relative path not allowed in hardened program), '/System/Volumes/Preboot/Cryptexes/OS./engines/cortex.llamacpp/libengine.dylib' (no such file), '/usr/lib/./engines/cortex.llamacpp/libengine.dylib' (no such file, not in dyld cache), './engines/cortex.llamacpp/libengine.dylib' (relative path not allowed in hardened program) - server.cc:152
2024-05-17T03:26:09.568Z [CORTEX]::Debug: Load model success with response {}
2024-05-17T03:26:09.570Z [CORTEX]::Debug: Validate model state failed with response "Conflict"
2024-05-17T03:26:09.570Z [CORTEX]::Debug: Validate model state with response 409
2024-05-17T03:26:09.570Z [CORTEX]::Error: Validate model status failed
Additional context
Add any other context or information that could be helpful in diagnosing the problem.
The text was updated successfully, but these errors were encountered:
Describe the bug
llama3-8b-instruct
Steps to reproduce
Steps to reproduce the behavior:
Use
Expected behavior
Screenshots
If applicable, add screenshots to help explain your issue.
Environment details
Logs
Additional context
Add any other context or information that could be helpful in diagnosing the problem.
The text was updated successfully, but these errors were encountered: