-
Notifications
You must be signed in to change notification settings - Fork 624
Open
Description
1️⃣ 描述一下问题
知识库中上传的是剑来小说的txt,报Chunk token length 1414 exceeds chunk_token_size 1200.
2️⃣ 报错日志
请运行以下命令,并提供部分相关日志:
# macOS / Linux
make logs
# Windows
docker logs --tail=100 api-dev
git rev-parse HEADmake logs 的输出:
api-dev | 03-11 19:27:49 WARNING __init__.py:1028: 2026-03-11 19:27:49,441 - lightrag - WARNING - Chunk split_by_character exceeds token limit: len=1414 limit=1200
api-dev | 03-11 19:27:49 ERROR __init__.py:1028: 2026-03-11 19:27:49,449 - lightrag - ERROR - Traceback (most recent call last):
api-dev | File "/usr/local/lib/python3.12/site-packages/lightrag/lightrag.py", line 1848, in process_document
api-dev | chunking_result = self.chunking_func(
api-dev | ^^^^^^^^^^^^^^^^^^^
api-dev | File "/usr/local/lib/python3.12/site-packages/lightrag/operate.py", line 121, in chunking_by_token_size
api-dev | raise ChunkTokenLimitExceededError(
api-dev | lightrag.exceptions.ChunkTokenLimitExceededError: Chunk token length 1414 exceeds chunk_token_size 1200. Preview: '“我曾下山游历三年,知道天时有变,顺带着地利人和,皆有极大变化,天下多出了许多前所未有的神异怪事。”
api-dev | “但是这些年来,我不曾遇到任何一位来自外乡的谪仙人。”
api-dev | 陈'
api-dev |
api-dev | 03-11 19:27:49 ERROR __init__.py:1028: 2026-03-11 19:27:49,450 - lightrag - ERROR - Failed to extract document 1/1: http://localhost:9000/ref-kb-3360ab47af15bc3598c3473c761c0/《剑 来 》(精校版全本)_1773228413234.txt
3️⃣ 相关截图
#️⃣ 其他相关信息
✅ 如果问题与模型调用相关,请尝试切换到其他在线模型
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels