[Question]: When parsing a document under the maximum token limit preset by the model, an error is reported indicating that the input tokens exceed the model's limit. How should this issue be debugged and resolved?

[biofer]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (Language Policy).
• Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• Please do not modify this template :) and fill in all the required fields.

Describe your problem

To start all services using Docker, image version: infiniflow/ragflow:v0.17.2.
The model being used is SiliconFlow's BAAI/bge-m3, with the maximum token count set to 8192.
BAAI/bge-m3___OpenAI-API@OpenAI-API-Compatible
When parsing a document under the maximum token limit preset by the model, an error is reported indicating that the input tokens exceed the model's limit. How should this issue be debugged and resolved?
The front-end error is as follows:

The log error is as follows:

2025-03-19 19:52:14,987 INFO     27 task_consumer_0 reported heartbeat: {"name": "task_consumer_0", "now": "2025-03-19T19:52:14.985+08:00", "boot_at": "2025-03-19T19:26:31.041+08:00", "pending": 1, "lag": 0, "done": 0, "failed": 0, "current": {"1aa9c6ce04b511f09f760242ac190006": {"id": "1aa9c6ce04b511f09f760242ac190006", "doc_id": "d8786200049211f0a6310242ac180006", "from_page": 100000000, "to_page": 100000000, "retry_count": 0, "kb_id": "c82c222e049211f0922e0242ac180006", "parser_id": "naive", "parser_config": {"auto_keywords": 5, "auto_questions": 0, "chunk_token_num": 512, "delimiter": "\\n!?;\u3002\uff1b\uff01\uff1f", "graphrag": {"entity_types": ["drug", "tumor", "gene", "mutation", "category"], "method": "light", "resolution": "true", "use_graphrag": "true"}, "layout_recognize": "DeepDOC"}, "name": "\u975e\u5c0f\u7ec6\u80de\u80ba\u764c.pdf", "type": "pdf", "location": "\u975e\u5c0f\u7ec6\u80de\u80ba\u764c.pdf", "size": 18991802, "tenant_id": "abaf5490007311f0b5530242ac180006", "language": "Chinese", "embd_id": "BAAI/bge-m3___OpenAI-API@OpenAI-API-Compatible", "pagerank": 0, "kb_parser_config": {"layout_recognize": "DeepDOC", "chunk_token_num": 512, "delimiter": "\\n!?;\u3002\uff1b\uff01\uff1f", "auto_keywords": 5, "auto_questions": 0, "html4excel": false, "raptor": {"use_raptor": false}, "graphrag": {"use_graphrag": "true", "entity_types": ["drug", "tumor", "gene", "mutation", "category"], "method": "light", "resolution": "true"}}, "img2txt_id": "", "asr_id": "", "llm_id": "deepseek-chat___OpenAI-API@OpenAI-API-Compatible", "update_time": 1742383633143, "task_type": "graphrag"}}}
2025-03-19 19:52:21,949 INFO     27 HTTP Request: POST http://192.168.1.56:9002/v1/chat/completions "HTTP/1.1 200 OK"
2025-03-19 19:52:22,342 INFO     27 set_progress(1aa9c6ce04b511f09f760242ac190006), progress: -1, progress_msg: 19:52:22 [ERROR][Exception]: Exceptions from Trio nursery (4 sub-exceptions) -- **ERROR**: Error code: 400 - {'error': {'message': '输入Token超过模型限制 (request id: 20250319195027616899205DVONcrl3)', 'type': 'upstream_error', 'param': '400', 'code': 'bad_response_status_code'}}
2025-03-19 19:52:22,342 ERROR    27 handle_task got exception for task {"id": "1aa9c6ce04b511f09f760242ac190006", "doc_id": "d8786200049211f0a6310242ac180006", "from_page": 100000000, "to_page": 100000000, "retry_count": 0, "kb_id": "c82c222e049211f0922e0242ac180006", "parser_id": "naive", "parser_config": {"auto_keywords": 5, "auto_questions": 0, "chunk_token_num": 512, "delimiter": "\\n!?;\u3002\uff1b\uff01\uff1f", "graphrag": {"entity_types": ["drug", "tumor", "gene", "mutation", "category"], "method": "light", "resolution": "true", "use_graphrag": "true"}, "layout_recognize": "DeepDOC"}, "name": "\u975e\u5c0f\u7ec6\u80de\u80ba\u764c.pdf", "type": "pdf", "location": "\u975e\u5c0f\u7ec6\u80de\u80ba\u764c.pdf", "size": 18991802, "tenant_id": "abaf5490007311f0b5530242ac180006", "language": "Chinese", "embd_id": "BAAI/bge-m3___OpenAI-API@OpenAI-API-Compatible", "pagerank": 0, "kb_parser_config": {"layout_recognize": "DeepDOC", "chunk_token_num": 512, "delimiter": "\\n!?;\u3002\uff1b\uff01\uff1f", "auto_keywords": 5, "auto_questions": 0, "html4excel": false, "raptor": {"use_raptor": false}, "graphrag": {"use_graphrag": "true", "entity_types": ["drug", "tumor", "gene", "mutation", "category"], "method": "light", "resolution": "true"}}, "img2txt_id": "", "asr_id": "", "llm_id": "deepseek-chat___OpenAI-API@OpenAI-API-Compatible", "update_time": 1742383633143, "task_type": "graphrag"}
  + Exception Group Traceback (most recent call last):
  |   File "/ragflow/rag/svr/task_executor.py", line 594, in handle_task
  |     await do_handle_task(task)
  |   File "/ragflow/rag/svr/task_executor.py", line 522, in do_handle_task
  |     await run_graphrag(task, task_language, with_resolution, with_community, chat_model, embedding_model, progress_callback)
  |   File "/ragflow/graphrag/general/index.py", line 94, in run_graphrag
  |     await resolve_entities(
  |   File "/ragflow/graphrag/general/index.py", line 231, in resolve_entities
  |     reso = await er(graph, callback=callback)
  |   File "/ragflow/graphrag/entity_resolution.py", line 101, in __call__
  |     async with trio.open_nursery() as nursery:
  |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_core/_run.py", line 1058, in __aexit__
  |     raise combined_error_from_nursery
  | exceptiongroup.ExceptionGroup: Exceptions from Trio nursery (4 sub-exceptions)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/ragflow/graphrag/entity_resolution.py", line 176, in _resolve_candidate
    |     response = await trio.to_thread.run_sync(lambda: self._chat(text, [{"role": "user", "content": "Output:"}], gen_conf))
    |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 447, in to_thread_run_sync
    |     return msg_from_thread.unwrap()
    |   File "/ragflow/.venv/lib/python3.10/site-packages/outcome/_impl.py", line 213, in unwrap
    |     raise captured_error
    |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 373, in do_release_then_return_result
    |     return result.unwrap()
    |   File "/ragflow/.venv/lib/python3.10/site-packages/outcome/_impl.py", line 213, in unwrap
    |     raise captured_error
    |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 392, in worker_fn
    |     ret = context.run(sync_fn, *args)
    |   File "/ragflow/graphrag/entity_resolution.py", line 176, in <lambda>
    |     response = await trio.to_thread.run_sync(lambda: self._chat(text, [{"role": "user", "content": "Output:"}], gen_conf))
    |   File "/ragflow/graphrag/general/extractor.py", line 66, in _chat
    |     raise Exception(response)
    | Exception: **ERROR**: Error code: 400 - {'error': {'message': '输入Token超过模型限制 (request id: 20250319195027616899205DVONcrl3)', 'type': 'upstream_error', 'param': '400', 'code': 'bad_response_status_code'}}
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    |   File "/ragflow/graphrag/entity_resolution.py", line 176, in _resolve_candidate
    |     response = await trio.to_thread.run_sync(lambda: self._chat(text, [{"role": "user", "content": "Output:"}], gen_conf))
    |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 447, in to_thread_run_sync
    |     return msg_from_thread.unwrap()
    |   File "/ragflow/.venv/lib/python3.10/site-packages/outcome/_impl.py", line 213, in unwrap
    |     raise captured_error
    |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 373, in do_release_then_return_result
    |     return result.unwrap()
    |   File "/ragflow/.venv/lib/python3.10/site-packages/outcome/_impl.py", line 213, in unwrap
    |     raise captured_error
    |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 392, in worker_fn
    |     ret = context.run(sync_fn, *args)
    |   File "/ragflow/graphrag/entity_resolution.py", line 176, in <lambda>
    |     response = await trio.to_thread.run_sync(lambda: self._chat(text, [{"role": "user", "content": "Output:"}], gen_conf))
    |   File "/ragflow/graphrag/general/extractor.py", line 66, in _chat
    |     raise Exception(response)
    | Exception: **ERROR**: Error code: 400 - {'error': {'message': '输入Token超过模型限制 (request id: 20250319195027535236662bw7KpkuU)', 'type': 'upstream_error', 'param': '400', 'code': 'bad_response_status_code'}}
    +---------------- 3 ----------------
    | Traceback (most recent call last):
    |   File "/ragflow/graphrag/entity_resolution.py", line 176, in _resolve_candidate
    |     response = await trio.to_thread.run_sync(lambda: self._chat(text, [{"role": "user", "content": "Output:"}], gen_conf))
    |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 447, in to_thread_run_sync
    |     return msg_from_thread.unwrap()
    |   File "/ragflow/.venv/lib/python3.10/site-packages/outcome/_impl.py", line 213, in unwrap
    |     raise captured_error
    |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 373, in do_release_then_return_result
    |     return result.unwrap()
    |   File "/ragflow/.venv/lib/python3.10/site-packages/outcome/_impl.py", line 213, in unwrap
    |     raise captured_error
    |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 392, in worker_fn
    |     ret = context.run(sync_fn, *args)
    |   File "/ragflow/graphrag/entity_resolution.py", line 176, in <lambda>
    |     response = await trio.to_thread.run_sync(lambda: self._chat(text, [{"role": "user", "content": "Output:"}], gen_conf))
    |   File "/ragflow/graphrag/general/extractor.py", line 66, in _chat
    |     raise Exception(response)
    | Exception: **ERROR**: Error code: 400 - {'error': {'message': '输入Token超过模型限制 (request id: 20250319195027892940097YDLhAIM4)', 'type': 'upstream_error', 'param': '400', 'code': 'bad_response_status_code'}}
    +---------------- 4 ----------------
    | Traceback (most recent call last):
    |   File "/ragflow/graphrag/entity_resolution.py", line 176, in _resolve_candidate
    |     response = await trio.to_thread.run_sync(lambda: self._chat(text, [{"role": "user", "content": "Output:"}], gen_conf))
    |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 447, in to_thread_run_sync
    |     return msg_from_thread.unwrap()
    |   File "/ragflow/.venv/lib/python3.10/site-packages/outcome/_impl.py", line 213, in unwrap
    |     raise captured_error
    |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 373, in do_release_then_return_result
    |     return result.unwrap()
    |   File "/ragflow/.venv/lib/python3.10/site-packages/outcome/_impl.py", line 213, in unwrap
    |     raise captured_error
    |   File "/ragflow/.venv/lib/python3.10/site-packages/trio/_threads.py", line 392, in worker_fn
    |     ret = context.run(sync_fn, *args)
    |   File "/ragflow/graphrag/entity_resolution.py", line 176, in <lambda>
    |     response = await trio.to_thread.run_sync(lambda: self._chat(text, [{"role": "user", "content": "Output:"}], gen_conf))
    |   File "/ragflow/graphrag/general/extractor.py", line 66, in _chat
    |     raise Exception(response)
    | Exception: **ERROR**: Error code: 400 - {'error': {'message': '输入Token超过模型限制 (request id: 20250319195028971046021zbPawvoY)', 'type': 'upstream_error', 'param': '400', 'code': 'bad_response_status_code'}}
    +------------------------------------

[KevinHuSh]

What's the number you setup while adding models?

[biofer]

what number？
After resubmitting the analysis, it succeeded directly.

[biofer]

The parameter is set to MAX_CONCURRENT_CHATS=10.
The document parsing succeeded after three attempts.
The error from the first attempt is as follows:

11:20:00 Task has been received.
11:20:01 Page(1~100000001): Start to parse.
11:20:02 Page(1~100000001): Finish parsing.
11:20:09 Page(1~100000001): Start to generate keywords for every chunk ...
11:20:09 Page(1~100000001): Keywords generation 47 chunks completed in 0.40s
11:20:09 Page(1~100000001): Generate 47 chunks
11:20:12 Page(1~100000001): Embedding chunks (2.78s)
11:20:16 Page(1~100000001): Indexing done (4.19s). Task done (15.87s)
11:20:17 created task raptor
11:20:17 Task has been received.
11:21:04 Cluster one layer: 47 -> 8
11:21:43 Cluster one layer: 8 -> 4
11:22:06 Cluster one layer: 4 -> 3
11:22:40 Cluster one layer: 3 -> 2
11:23:03 Cluster one layer: 2 -> 1
11:23:06 Indexing done (2.13s). Task done (168.83s)
11:27:14 Entities extraction of chunk 15 33/64 done, 11 nodes, 11 edges, 11811 tokens.
...
11:37:22 Entities extraction of chunk 50 63/64 done, 26 nodes, 12 edges, 21528 tokens.
11:37:22 [ERROR][Exception]: Exceptions from Trio nursery (1 sub-exception) -- **ERROR**: Error code: 504 - {'error': {'message': ' (request id: 20250320113153157623793ipzNrq3s)', 'type': 'upstream_error', 'param': '504', 'code': 'bad_response_status_code'}}

No adjustments were made, and the parsing was resubmitted directly.

11:51:54 Page(0~100000000): Reused previous task's chunks.
11:51:57 created task raptor
11:51:57 Task has been received.
11:52:33 Cluster one layer: 47 -> 8
11:53:22 Cluster one layer: 8 -> 3
11:53:52 Cluster one layer: 3 -> 2
11:54:02 Cluster one layer: 2 -> 1
11:54:12 Indexing done (9.42s). Task done (134.37s)
11:56:19 Entities extraction of chunk 38 36/61 done, 24 nodes, 17 edges, 12758 tokens.
...
12:10:43 Entities extraction of chunk 57 61/61 done, 0 nodes, 27 edges, 33511 tokens.
12:10:43 Entities and relationships extraction done, 408 nodes, 818 edges, 960372 tokens, 986.24s.
12:11:40 Entities merging done, 56.71s.
12:15:27 Relationships merging done, 226.97s.
12:15:27 generated subgraph for doc 208b3098053a11f0a73b0242ac190006 in 1270.33 seconds.
12:15:37 merging subgraph for doc 208b3098053a11f0a73b0242ac190006 into the global graph done in 10.08 seconds.
12:15:38 Identified 8180 candidate pairs
12:18:44 [ERROR][Exception]: Exceptions from Trio nursery (3 sub-exceptions) -- **ERROR**: Error code: 400 - {'error': {'message': '输入Token超过模型限制 (request id: 20250320121538748319063bjqjnwtu)', 'type': 'upstream_error', 'param': '400', 'code': 'bad_response_status_code'}}

Encountered the same problem as before, and also resubmitted directly. Then it ran successfully in less than a minute.
Were the errors in both cases caused by issues with the third-party API interface? Is there any way to avoid this?

13:01:20 Page(0~100000000): Reused previous task's chunks.
13:01:21 created task raptor
13:01:21 Task has been received.
13:01:58 Cluster one layer: 47 -> 8
13:02:29 Cluster one layer: 8 -> 3
13:03:04 Cluster one layer: 3 -> 2
13:03:20 Cluster one layer: 2 -> 1
13:03:22 Indexing done (1.60s). Task done (120.78s)
13:03:22 created task graphrag
13:03:22 Task has been received.
13:03:23 Graph already contains 208b3098053a11f0a73b0242ac190006, cancel myself
13:03:23 Knowledge Graph done (0.42s)

[KevinHuSh]

Here. Sorry, image uploading failure.

[biofer]

Embedding model: BAAI/bge-m3 (Silicon Flow); the maximum token is set to 8192.
Chat model: DeepSeek V3 API provided by SophNet; the maximum token is set to 64000.
The error message indicated "Input tokens exceed the model's limit," but it succeeded directly upon resubmission. Theoretically, it should still have thrown an error, right?