error with very large chunk_size

[seoeaa]

Error if you insert a lot of text, it is better to read the text in the input window and not skip such a request

Traceback (most recent call last):
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\queueing.py", line 427, in call_prediction
output = await route_utils.call_process_api(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\route_utils.py", line 232, in call_process_api
output = await app.get_blocks().process_api(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 1497, in process_api
result = await self.call_function(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 1119, in call_function
prediction = await anyio.to_thread.run_sync(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\utils.py", line 665, in wrapper
response = f(*args, **kwargs)
File "d:\Programm\AI\autollm\app.py", line 54, in greet
return query_engine.query(query).response
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\query\base.py", line 32, in query
return self._query(str_or_query_bundle)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\query_engine\retriever_query_engine.py", line 182, in _query
response = self._response_synthesizer.synthesize(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\response_synthesizers\base.py", line 148, in synthesize
response_str = self.get_response(
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\response_synthesizers\compact_and_refine.py", line 37, in get_response
new_texts = self._make_compact_text_chunks(query_str, text_chunks)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\response_synthesizers\compact_and_refine.py", line 52, in _make_compact_text_chunks
return self._service_context.prompt_helper.repack(max_prompt, text_chunks)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\prompt_helper.py", line 223, in repack
text_splitter = self.get_text_splitter_given_prompt(prompt, padding=padding)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\prompt_helper.py", line 186, in get_text_splitter_given_prompt
chunk_size = self._get_available_chunk_size(prompt, num_chunks, padding=padding)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\prompt_helper.py", line 171, in _get_available_chunk_size
available_context_size = self._get_available_context_size(prompt)
File "C:\Users\Aleksandr\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\prompt_helper.py", line 152, in _get_available_context_size
raise ValueError(
ValueError: Calculated available context size -4574 was not non-negative.

[talhasarit]

Hi @seoeaa, can you please share your code snippet that caused this error?

[fcakyon]

Hello @seoeaa, we appeciate you feedbacks on the autollm 🙏

autollm supports documents with infinite length. Your issue is not related to inserting a lot of text.

Your issue will be fixed once you select lower chunk_size as 512 or 1024. 👍

Check this comment for more context: https://github.com/safevideo/autollm/issues/77#issuecomment-1792349351

Comment:

I changed it otherwise it didn't work service_context_params = { "chunk size": 4096, "embed_model": "local" }

@seoeaa you should be using smaller chunk sizes such as 1024 or 512. chunk_size is a different parameter than context_size.

Context size automatically arranges by autollm for target llm, you don't have to set it manually.