bug: could not load library ~\cortex/engines/cortex.tensorrt-llm/engine.dll when chatting with model

[freelerobot]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce

1. cortex run llama3:tensorrt-llm-windows-ada (valid model branch this time)
2. check that model dlls , binaries all exist (model downloaded successfully)

√ Model loaded
In order to exit, type 'exit()'.
>> hi
Something went wrong! Please check model status.
500 Request failed with status code 500

4. When checking logs

{"method":"POST","path":"/v1/threads","statusCode":201,"ip":"127.0.0.1","content_length":"483","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
{"method":"GET","path":"/v1/threads/1f597d11-5e1c-41c8-b1ec-23d1ae8339be/messages?limit=10&order=desc","statusCode":200,"ip":"127.0.0.1","content_length":"44","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
Validating app logs. Next attempt in  120000
Validating app logs. Next attempt in  120000
Validating app logs. Next attempt in  120000
{"method":"GET","path":"/v1/models/llama3:tensorrt-llm-windows-ada","statusCode":200,"ip":"127.0.0.1","content_length":"690","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
- Loading model...
20240815 15:53:40.063000 UTC 1424 WARN  Engine is not loaded yet - server.cc:166
20240815 15:53:40.080000 UTC 1424 ERROR Could not load engine: Could not load library "C:\Users\n\cortex/engines/cortex.tensorrt-llm/engine.dll"
The specified module could not be found.

 - server.cc:299
× Model loading failed
{"method":"POST","path":"/v1/threads/1f597d11-5e1c-41c8-b1ec-23d1ae8339be/messages","statusCode":201,"ip":"127.0.0.1","content_length":"376","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
{"method":"POST","path":"/v1/chat/completions","statusCode":500,"ip":"127.0.0.1","content_length":"35","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
- Loading model...
20240815 15:53:43.167000 UTC 1424 WARN  Engine is not loaded yet - server.cc:166
20240815 15:53:43.169000 UTC 1424 ERROR Could not load engine: Could not load library "C:\Users\n\cortex/engines/cortex.tensorrt-llm/engine.dll"
The specified module could not be found.

 - server.cc:299
× Model loading failed
{"method":"POST","path":"/v1/chat/completions","statusCode":500,"ip":"127.0.0.1","content_length":"35","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
- Loading model...
20240815 15:53:44.080000 UTC 1424 WARN  Engine is not loaded yet - server.cc:166
20240815 15:53:44.082000 UTC 1424 ERROR Could not load engine: Could not load library "C:\Users\n\cortex/engines/cortex.tensorrt-llm/engine.dll"
The specified module could not be found.

 - server.cc:299
× Model loading failed
{"method":"POST","path":"/v1/chat/completions","statusCode":500,"ip":"127.0.0.1","content_length":"35","user_agent":"CortexClient/JS 0.1.7","x_correlation_id":""} HTTP
Validating app logs. Next attempt in  120000
Validating app logs. Next attempt in  120000

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

[louis-jan]

The same root cause with Failed to init tensorrt-llm. Fixed - Wrong binary lookup.

Could not load library "C:\Users\n\cortex/engines/cortex.tensorrt-llm/engine.dll"
The specified module could not be found.

[hantran-co]

I saw the PR is merged, it's fixed right? @louis-jan

[louis-jan]

Yes @imtuyethan

[louis-jan]

From now on, cortex-cpp will handle engine stuff. I will pass it over to @vansangpfiev. (Not to take action, just to follow up when release)