[Question]: Document process blocked at 80%

[shaoyie]

Describe your problem

local docker with latest image, the document process is blocked at 80%. LLM is ChatGLM and the API Key is set in the web ui.
Error log is:

Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
ES create index error ragflow_0196ca84f5a111ee80170242ac150006 ----BadRequestError(400, 'resource_already_exists_exception', 'index [ragflow_0196ca84f5a111ee80170242ac150006/gHToUXxJSNSqdLG9Yo0mNA] already exists')
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '367e6c34f5a111ee9a840242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '367e6c34f5a111ee9a840242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '367e6c34f5a111ee9a840242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '372033f2f5a111eea71b0242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '371b7de4f5a111eea0880242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '367e6c34f5a111ee9a840242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '367e6c34f5a111ee9a840242ac150006'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '367e6c34f5a111ee9a840242ac150006'}}
Fail put 0beab266f5a111eeab0c0242ac150006/附件1：《好烤漆金牌造》销售工具话术.pdf: S3 operation failed; code: NoSuchKey, message: Object does not exist, resource: /0beab266f5a111eeab0c0242ac150006/%E9%99%84%E4%BB%B61%EF%BC%9A%E3%80%8A%E5%A5%BD%E7%83%A4%E6%BC%86%E9%87%91%E7%89%8C%E9%80%A0%E3%80%8B%E9%94%80%E5%94%AE%E5%B7%A5%E5%85%B7%E8%AF%9D%E6%9C%AF.pdf, request_id: 17C44D75219FE3C5, host_id: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8, bucket_name: 0beab266f5a111eeab0c0242ac150006, object_name: 附件1：《好烤漆金牌造》销售工具话术.pdf
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '9b962552f5a211eea4b10242ac150005'}}
Fail put 8568867ef5a411eebc050242ac150005/附件1：《好烤漆金牌造》销售工具话术.pdf: S3 operation failed; code: NoSuchBucket, message: The specified bucket does not exist, resource: /8568867ef5a411eebc050242ac150005, request_id: 17C44E3BAF64631C, host_id: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8, bucket_name: 8568867ef5a411eebc050242ac150005
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
ES updateByQuery deleteByQuery: ApiError(503, 'search_phase_execution_exception')【Q】：{'match': {'doc_id': '97e2aa3cf5a411eeac6d0242ac150005'}}
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING
Can't update token usage for 0196ca84f5a111ee80170242ac150006/EMBEDDING

What might be the error?

[KevinHuSh]

The minio might not be set properly especially for the user name and the key.

And the ES seems a little short of resources and reject connection.