Cannot get woriking with Mistral AI

[vanhellthing93]

Hi, spend a lot of time trying to get working with Mistral AI (free models).

I use docker with the following config:

  INFERENCE_CONTEXT_LENGTH: 31000
  INFERENCE_TEXT_MODEL: mistral-small-latest
  INFERENCE_IMAGE_MODEL: pixtral-12b-2409
  OPENAI_BASE_URL: https://api.mistral.ai/v1
  OPENAI_API_KEY: my api key
  INFERENCE_JOB_TIMEOUT_SEC: 120

First I have received timeout errors. After changing job-timeout variable I received the following error:

2025-02-07T07:57:55.908Z error: [inference][128] inference job failed: Error: Connection error.

Error: Connection error.

at OpenAI.makeRequest (/app/apps/workers/node_modules/.pnpm/openai@4.67.1_zod@3.22.4/node_modules/openai/core.js:321:19)

at process.processTicksAndRejections (node:internal/process/task_queues:105:5)

at async OpenAIInferenceClient.inferFromText (/app/apps/workers/node_modules/.pnpm/@hoarder+shared@file+packages+shared_better-sqlite3@11.3.0/node_modules/@hoarder/shared/inference.ts:2:2002)

at async inferTagsFromText (/app/apps/workers/openaiWorker.ts:6:3097)

at async inferTags (/app/apps/workers/openaiWorker.ts:6:3356)

at async Object.runOpenAI [as run] (/app/apps/workers/openaiWorker.ts:6:6814)

at async Runner.runOnce (/app/apps/workers/node_modules/.pnpm/liteque@0.3.2_better-sqlite3@11.3.0/node_modules/liteque/dist/runner.js:2:2656)

I've tested connection to api with the same key, link and model name via python openai library, everything works fine (no problems with connection, or invalid credentials).

Any help will be appreciated

[MohamedBassem]

Getting connection error is very suspicious. Is there any other error logs around those log lines?

[vanhellthing93]

Only deprecation warning, I don't think it is important.

Just in case, here are the logs from the start of the container (just after start I've pushed "Regenerate AI Tags for All Bookmarks" button)

Log

2025-02-10T06:42:13.234448264Z s6-rc: info: service s6rc-oneshot-runner: starting
2025-02-10T06:42:13.267760358Z s6-rc: info: service s6rc-oneshot-runner successfully started
2025-02-10T06:42:13.284753664Z s6-rc: info: service fix-attrs: starting
2025-02-10T06:42:13.289970086Z s6-rc: info: service init-db-migration: starting
2025-02-10T06:42:13.429620039Z s6-rc: info: service fix-attrs successfully started
2025-02-10T06:42:13.446117291Z s6-rc: info: service legacy-cont-init: starting
2025-02-10T06:42:13.495903114Z Running db migration script
2025-02-10T06:42:13.551068438Z s6-rc: info: service legacy-cont-init successfully started
2025-02-10T06:42:15.541633308Z s6-rc: info: service init-db-migration successfully started
2025-02-10T06:42:15.541699203Z s6-rc: info: service svc-workers: starting
2025-02-10T06:42:15.541712175Z s6-rc: info: service svc-web: starting
2025-02-10T06:42:15.610398487Z s6-rc: info: service svc-workers successfully started
2025-02-10T06:42:15.614607473Z s6-rc: info: service svc-web successfully started
2025-02-10T06:42:15.631851073Z s6-rc: info: service legacy-services: starting
2025-02-10T06:42:15.710763070Z s6-rc: info: service legacy-services successfully started
2025-02-10T06:42:19.004803836Z   ▲ Next.js 14.2.21
2025-02-10T06:42:19.005804534Z   - Local:        http://localhost:3000
2025-02-10T06:42:19.010612364Z   - Network:      http://0.0.0.0:3000
2025-02-10T06:42:19.011187046Z 
2025-02-10T06:42:19.011718385Z  ✓ Starting...
2025-02-10T06:42:19.444364244Z  ✓ Ready in 2.9s
2025-02-10T06:42:20.123107044Z 
2025-02-10T06:42:20.123191321Z > @hoarder/workers@0.1.0 start:prod /app/apps/workers
2025-02-10T06:42:20.123227632Z > tsx index.ts
2025-02-10T06:42:20.123241820Z 
2025-02-10T06:42:38.648473183Z 2025-02-10T06:42:38.635Z info: Workers version: 0.22.0
2025-02-10T06:42:38.712312764Z 2025-02-10T06:42:38.711Z info: [crawler] Loading adblocker ...
2025-02-10T06:42:38.714695325Z (node:128) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
2025-02-10T06:42:38.714733913Z (Use `node --trace-deprecation ...` to show where the warning was created)
2025-02-10T06:42:46.499616111Z 2025-02-10T06:42:46.499Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222
2025-02-10T06:42:46.505252180Z 2025-02-10T06:42:46.504Z info: [Crawler] Successfully resolved IP address, new address: http://10.0.22.14:9222/
2025-02-10T06:42:46.937449970Z 2025-02-10T06:42:46.936Z info: Starting crawler worker ...
2025-02-10T06:42:46.938934208Z 2025-02-10T06:42:46.938Z info: Starting inference worker ...
2025-02-10T06:42:46.939608598Z 2025-02-10T06:42:46.939Z info: Starting search indexing worker ...
2025-02-10T06:42:46.940237691Z 2025-02-10T06:42:46.940Z info: Starting tidy assets worker ...
2025-02-10T06:42:46.941119693Z 2025-02-10T06:42:46.940Z info: Starting video worker ...
2025-02-10T06:42:46.942456468Z 2025-02-10T06:42:46.942Z info: Starting feed worker ...
2025-02-10T06:42:46.943203991Z 2025-02-10T06:42:46.943Z info: Starting asset preprocessing worker ...
2025-02-10T06:42:46.943743274Z 2025-02-10T06:42:46.943Z info: Starting webhook worker ...
2025-02-10T06:42:49.690068406Z (node:69) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
2025-02-10T06:42:49.690134603Z (Use `node --trace-deprecation ...` to show where the warning was created)
# Here I've pushed the button to regenerate tags
2025-02-10T06:43:04.216137138Z 2025-02-10T06:43:04.215Z info: [inference][149] Starting an inference job for bookmark with id "co8vxwmhw7xxw5lu8a8zwwl7"
2025-02-10T06:43:50.776929046Z 2025-02-10T06:43:50.776Z error: [inference][149] inference job failed: Error: Connection error.
2025-02-10T06:43:50.777070092Z Error: Connection error.
2025-02-10T06:43:50.777083955Z     at OpenAI.makeRequest (/app/apps/workers/node_modules/.pnpm/openai@4.67.1_zod@3.22.4/node_modules/openai/core.js:321:19)
2025-02-10T06:43:50.777096295Z     at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
2025-02-10T06:43:50.777108000Z     at async OpenAIInferenceClient.inferFromText (/app/apps/workers/node_modules/.pnpm/@hoarder+shared@file+packages+shared_better-sqlite3@11.3.0/node_modules/@hoarder/shared/inference.ts:2:2002)
2025-02-10T06:43:50.777120205Z     at async inferTagsFromText (/app/apps/workers/openaiWorker.ts:6:3097)
2025-02-10T06:43:50.777158085Z     at async inferTags (/app/apps/workers/openaiWorker.ts:6:3356)
2025-02-10T06:43:50.777169593Z     at async Object.runOpenAI [as run] (/app/apps/workers/openaiWorker.ts:6:6814)
2025-02-10T06:43:50.777181168Z     at async Runner.runOnce (/app/apps/workers/node_modules/.pnpm/liteque@0.3.2_better-sqlite3@11.3.0/node_modules/liteque/dist/runner.js:2:2656)
2025-02-10T06:43:50.804063500Z 2025-02-10T06:43:50.803Z info: [inference][149] Starting an inference job for bookmark with id "co8vxwmhw7xxw5lu8a8zwwl7"

[IdeallyGrey]

I was trying to do this and was having a very similar issue (my errors included 401s). Setting the model name to "mistral-small" instead of "mistral-small-latest" resolved it. Hope it helps you.

Update: Upon further testing the free models only work without the "-latest", while the paid models only work when you include it. Funny stuff.

[dchakro]

As of July 2026,

INFERENCE_TEXT_MODEL=mistral-small-latest
INFERENCE_IMAGE_MODEL=mistral-small-latest

OPENAI_BASE_URL=https://api.mistral.ai/v1
OPENAI_API_KEY=MISTRAL_API_KEY

# And ensuring these two are not set or are commented off
# OPENAI_REASONING_EFFORT=low
# INFERENCE_REASONING_EFFORT=low

The above setting works for me.