Keep hitting the timeout

[sanxfxteam]

856 tokens in prompt: You are an AI developer who is trying to write a p
383 tokens in prompt: 
    We have broken up the program into per-file g
Task's current input in-asvotoKZmysBBhM8gYwuO7 hit its timeout of 300s
Task's current input in-2faqX2wOstABbDFnaz7bL4 hit its timeout of 300s
Task's current input in-V1wHgq9MirPebunNHpsAbD hit its timeout of 300s
Task's current input in-tgHALPxITM0zESibMZcZMi hit its timeout of 300s
Task's current input in-dFkDAszKCVLVtxaDYPyLYK hit its timeout of 300s
Task's current input in-wb1AOmhAJi6AE42fP69QSZ hit its timeout of 300s
Task's current input in-cM4rYTeziKeLTEvij4Le7Z hit its timeout of 300s
Task's current input in-TyTG7gTy0hahqCcGUqfCMN hit its timeout of 300s
Task's current input in-ppHAaueBWIDGHNcMI76IyA hit its timeout of 300s
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
....
│ /usr/local/lib/python3.10/site-packages/modal/functions.py:437 in          │
│ fetch_output                                                               │
│                                                                            │
│    436 │   │   try:                                                        │
│ ❱  437 │   │   │   output = await _process_result(item.result, client.stub │
│    438 │   │   except Exception as e:                                      │
│                                                                            │
│ /usr/local/lib/python3.10/site-packages/modal/functions.py:109 in          │
│ _process_result                                                            │
│                                                                            │
│    108 │   if result.status == api_pb2.GenericResult.GENERIC_STATUS_TIMEOU │
│ ❱  109 │   │   raise _TimeoutError(result.exception)                       │
│    110 │   elif result.status != api_pb2.GenericResult.GENERIC_STATUS_SUCC │
╰────────────────────────────────────────────────────────────────────────────╯
TimeoutError: Task's current input in-2faqX2wOstABbDFnaz7bL4 hit its timeout 
of 300s

I've change things in main.py:

@stub.function(
    image=openai_image,
    secret=modal.Secret.from_dotenv(),
    retries=modal.Retries(
        max_retries=3,
        backoff_coefficient=2.0,
        initial_delay=1.0,
    ),
    concurrency_limit=2,
    timeout=600,
)
def generate_response(system_prompt, user_prompt, *args):

I use modal.

[swyxio]

thanks for the help! this is very strange. calling in friends from modal

[swyxio]

hey @sanxfxteam - we had a look and think it was a transient network issue on your end or on Modal's end or on OpenAI's end. we reallyhave no way of knowing. @gerred reran everything and couldnt repro. please let reopen this issue if the problem persists and send us your prompt!

[aksh-at]

I suspect it might be this. Would be nice to have a client-side timeout for the chat completion. I can try and add that in a little bit :)

[sanxfxteam]

I was using GPT-4 model, maybe the problem was there too. Will try again later. Thanks for the quick response.

…

On Sun, May 21, 2023, 17:27 Akshat Bubna ***@***.***> wrote: I suspect it might be this <https://community.openai.com/t/completion-endpoint-randomly-freezes/107206/4>. Would be nice to have a client-side timeout for the chat completion. I can try and add that in a little bit :) — Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADNGNZVGCXSUXLOZYX4RVDXHKXHLANCNFSM6AAAAAAYJALIOI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[gerred]

@sanxfxteam If you run into it again and it's repeatable for a given prompt, send it over and I'm happy to investigate and provide some guidance. At least over in local LLM land with very low temperatures I can repro issues where models either get stuck in repeat loops or don't provide parseable tokens to move forward, and these issues can still happen with SOTA models. Don't think this is a smol or Modal issue.

[jmanhype]

We have broken up the program into per-file g

src/templates/ai_recommendation.html
{% extends "base.html" %}

{% block content %}

AI-Powered Prediction & Recommendation

Discover new Pokemon based on your preferences and search history.

Get Recommendations

{% endblock %} 1266 tokens in prompt: You are an AI developer who is trying to write a p 724 tokens in prompt: We have broken up the program into per-file g Task's current input in-YmoNF7Xt5tEGlE9J7QySYG hit its timeout of 300s 1266 tokens in prompt: You are an AI developer who is trying to write a p 728 tokens in prompt: We have broken up the program into per-file g Task's current input in-apZfTpMwXPgWMVrAOXmHeO hit its timeout of 300s Task's current input in-VlbbQh3rv1ASe3SxOYVdVd hit its timeout of 300s Task's current input in-A6q3nngSp8YKVf6jTYHcpM hit its timeout of 300s Task's current input in-uEUxfvK5Gb3wiLPEhQilLv hit its timeout of 300s Task's current input in-QgcLVZlkoWD0VViGSbZLZ4 hit its timeout of 300s Task's current input in-xnE8MxN7SWSyOdadNXohBW hit its timeout of 300s Task's current input in-HCrJAhIwGfgAJeEUxhmoHT hit its timeout of 300s Task's current input in-fVOsjosbgVjLr4PhjFzIBh hit its timeout of 300s Task's current input in-Yy9pGxdzbh30MT66oiQDao hit its timeout of 300s Task's current input in-IsptLUPKGowtIsdxr4oJ5H hit its timeout of 300s Task's current input in-S5l0PkdRRXDM6mHkCTKQXR hit its timeout of 300s Task's current input in-KjU9nGrwr2gaewi7HYvEXJ hit its timeout of 300s Runner terminated, in-progress inputs will be re-scheduled Runner terminated, in-progress inputs will be re-scheduled Task's current input in-F3AkWItdoeC4q5K0cYOelr hit its timeout of 300s Runner terminated, in-progress inputs will be re-scheduled Task's current input in-GaSn05JJqilmwLjOyxuIKY hit its timeout of 300s Task's current input in-BBPolgJKFgz0pRDCaFli83 hit its timeout of 300s Task's current input in-6PEqVdkell9JuPaEttwkW9 hit its timeout of 300s Runner terminated, in-progress inputs will be re-scheduled Runner terminated, in-progress inputs will be re-scheduled ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ :198 in _run_module_as_main │ │ │ │ :88 in _run_code │ │ │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\Scripts\modal.exe\__main__.py:7 in │ │ │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\modal\__main__.py:6 in main │ │ │ │ 5 def main(): │ │ ❱ 6 │ entrypoint_cli() │ │ 7 │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\click\core.py:1130 in __call__ │ │ │ │ 1129 │ │ """Alias for :meth:`main`.""" │ │ ❱ 1130 │ │ return self.main(*args, **kwargs) │ │ 1131 │ │ │ │ C:\Python311\Lib\site-packages\typer\core.py:778 in main │ │ │ │ 777 │ ) -> Any: │ │ ❱ 778 │ │ return _main( │ │ 779 │ │ │ self, │ │ │ │ C:\Python311\Lib\site-packages\typer\core.py:216 in _main │ │ │ │ 215 │ │ │ with self.make_context(prog_name, args, **extra) as ctx: │ │ ❱ 216 │ │ │ │ rv = self.invoke(ctx) │ │ 217 │ │ │ │ if not standalone_mode: │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\click\core.py:1657 in invoke │ │ │ │ 1656 │ │ │ │ with sub_ctx: │ │ ❱ 1657 │ │ │ │ │ return _process_result(sub_ctx.command.invoke(sub_ctx)) │ │ 1658 │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\click\core.py:1657 in invoke │ │ │ │ 1656 │ │ │ │ with sub_ctx: │ │ ❱ 1657 │ │ │ │ │ return _process_result(sub_ctx.command.invoke(sub_ctx)) │ │ 1658 │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\click\core.py:1404 in invoke │ │ │ │ 1403 │ │ if self.callback is not None: │ │ ❱ 1404 │ │ │ return ctx.invoke(self.callback, **ctx.params) │ │ 1405 │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\click\core.py:760 in invoke │ │ │ │ 759 │ │ │ with ctx: │ │ ❱ 760 │ │ │ │ return __callback(*args, **kwargs) │ │ 761 │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\click\decorators.py:26 in new_func │ │ │ │ 25 │ def new_func(*args, **kwargs): # type: ignore │ │ ❱ 26 │ │ return f(get_current_context(), *args, **kwargs) │ │ 27 │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\modal\cli\run.py:116 in f │ │ │ │ 115 │ │ │ else: │ │ ❱ 116 │ │ │ │ func(*args, **kwargs) │ │ 117 │ │ │ if app.function_invocations == 0: │ │ │ │ C:\Users\strau\developer\main.py:174 in main │ │ │ │ 173 │ │ │ # Existing for loop │ │ ❱ 174 │ │ │ for filename, filecode in generate_file.map( │ │ 175 │ │ │ │ list_actual, order_outputs=False, kwargs=dict(filepaths_string=filepaths │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\synchronicity\synchronizer.py:365 │ │ in _run_generator_sync │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\modal\functions.py:660 in map │ │ │ │ 659 │ │ input_stream = stream.zip(*(stream.iterate(it) for it in input_iterators)) │ │ ❱ 660 │ │ async for item in self._map(input_stream, order_outputs, return_exceptions, kwar │ │ 661 │ │ │ yield item │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\modal\functions.py:599 in _map │ │ │ │ 598 │ │ self._track_function_invocation() │ │ ❱ 599 │ │ async for item in _map_invocation( │ │ 600 │ │ │ self._object_id, │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\modal\functions.py:475 in │ │ _map_invocation │ │ │ │ 474 │ async with response_gen.stream() as streamer: │ │ ❱ 475 │ │ async for response in streamer: │ │ 476 │ │ │ if response is not None: │ │ │ │ C:\Python311\Lib\site-packages\aiostream\stream\advanced.py:59 in base_combine │ │ │ │ 58 │ │ │ try: │ │ ❱ 59 │ │ │ │ result = task.result() │ │ 60 │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\modal\functions.py:455 in │ │ poll_outputs │ │ │ │ 454 │ │ async with outputs_fetched.stream() as streamer: │ │ ❱ 455 │ │ │ async for idx, output in streamer: │ │ 456 │ │ │ │ if count_update_callback is not None: │ │ │ │ C:\Python311\Lib\site-packages\aiostream\stream\advanced.py:59 in base_combine │ │ │ │ 58 │ │ │ try: │ │ ❱ 59 │ │ │ │ result = task.result() │ │ 60 │ │ │ │ C:\Python311\Lib\site-packages\aiostream\stream\create.py:85 in call │ │ │ │ 84 │ if asyncio.iscoroutinefunction(func): │ │ ❱ 85 │ │ yield await func(*args, **kwargs) │ │ 86 │ else: │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\modal\functions.py:443 in │ │ fetch_output │ │ │ │ 442 │ │ │ else: │ │ ❱ 443 │ │ │ │ raise e │ │ 444 │ │ return (item.idx, output) │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\modal\functions.py:438 in │ │ fetch_output │ │ │ │ 437 │ │ try: │ │ ❱ 438 │ │ │ output = await _process_result(item.result, client.stub, client) │ │ 439 │ │ except Exception as e: │ │ │ │ C:\Users\strau\AppData\Roaming\Python\Python311\site-packages\modal\functions.py:110 in │ │ _process_result │ │ │ │ 109 │ if result.status == api_pb2.GenericResult.GENERIC_STATUS_TIMEOUT: │ │ ❱ 110 │ │ raise _TimeoutError(result.exception) │ │ 111 │ elif result.status != api_pb2.GenericResult.GENERIC_STATUS_SUCCESS: │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ TimeoutError: Task's current input in-YmoNF7Xt5tEGlE9J7QySYG hit its timeout of 300s

[jmanhype]

same error

[jmanhype]

this is my prompt Title: PokeFinder App

Description: Discover, track and collect detailed information about your favorite Pokemon with the PokeFinder App. Leverage the power of artificial intelligence to find and learn about the Pokemon you love.

Objective: The objective of the PokeFinder App is to help Pokemon enthusiasts explore, discover, and learn about their favorite Pokemon characters. Users can search and track these creatures, collect detailed information, and enjoy an engaging, fun experience as a Pokemon fan.

Key Features:

1. Search Functionality: Quickly and easily search for Pokemon by their names, types, abilities, or other criteria such as generation or region.

2. Pokemon Profiles: Access comprehensive profiles for each Pokemon, including their stats, abilities, evolutions, moves, and other essential information.

3. Custom Tracking & Collection: Keep track of the Pokemon you've found, build your collection, and share it with friends or rival collectors.

4. Augmented Reality (AR) Integration: Discover and interact with Pokemon in real-time using your smartphone camera. Bring your favorite creatures to life and watch them battle in virtual environments.

5. AI-Powered Prediction & Recommendation: With the help of AI technology, get smart recommendations on which Pokemon you might like based on your search patterns, preferences, and already captured ones.

6. Event Notifications & Alerts: Stay informed about special limited-time events, like tournaments, shiny Pokemon appearances, or egg hatching events.

7. Social Connectivity: Chat with other users, join fan clubs, organize friendly battles, or share your findings with the PokeFinder community.

8. Educational Mode: Learn more about Pokemon, their habitats, and origins through an in-app interactive encyclopedia, featuring detailed information and fun facts.

Additional Requirements:

• Responsiveness on various devices (smartphones, tablets, and desktop computers)
• Compatibility with iOS and Android platforms
• Security measures to protect user data and privacy
  The "assets/images/" directory is meant for storing image files, not code. Therefore, there is no code to generate for
  this directory. Instead, you would place image files such as PNG, JPEG, or SVG files in this directory to be used within
  the PokeFinder App.
  we added some usecase specific code in there to skip destroying/regenerating them,
  Success Metrics:
• User engagement rate: number of daily active users, time spent within the app, repeat usage
• Conversion rate: number of casual users turning into dedicated collectors
• Growth rate: increase in new users and app downloads within a specified time frame
• Community impact: interaction between users, content sharing, collaborations, and app ratings
• User satisfaction: positive feedback and reviews from users about the app's usability, features, and reliability

[jmanhype]

I was having the image issue where it would crash beacause it cant tell if it needs to create images,

[jmanhype]

now I am getting 300 sec rate limit error

[knewter]

422 tokens in prompt:
    We have broken up the program into per-file g...
Task's current input in-OUvrOt7AEKM2HJVJ0EXHwK hit its timeout of 300s
Task's current input in-fpmlE8J0PpO5qHGldKdCmM hit its timeout of 300s
Task's current input in-amM44X0IGw3JRgrXl2g960 hit its timeout of 300s
Task's current input in-K4cKc9jAnXOOYHsACQF4N1 hit its timeout of 300s
Task's current input in-RpmCAIbZ8C6ubeMVYXLjoB hit its timeout of 300s
Task's current input in-JQmD2kl0b3qFqV0WywOWX3 hit its timeout of 300s
Task's current input in-1MY5h2VYW1Sr2i8SxCkkFO hit its timeout of 300s
Task's current input in-dnAqAaeO5VIbzl90PpRKaI hit its timeout of 300s
Task's current input in-6EIhtvVlsN5sJlJSaW121O hit its timeout of 300s
Task's current input in-sYTxCL4TzunpNGto7RidWB hit its timeout of 300s
Task's current input in-dy39ueGmqQpHfZjFWcVgA4 hit its timeout of 300s
Task's current input in-WBuBsu3YRZ1CdkfCDx2Bbe hit its timeout of 300s
Task's current input in-wN5Q93cRTSDUGaLHbXzapx hit its timeout of 300s
Task's current input in-wk6Nwk3yGJLJbSW0NBdK53 hit its timeout of 300s
Task's current input in-LUY2w3lmqK1XQgd8mLi0Cg hit its timeout of 300s
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Task's current input in-X5pz2LeQfFIRygfZXUiAbX hit its timeout of 300s
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Task's current input in-DUttWWacPjEH3iCEfWh0e5 hit its timeout of 300s
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Task's current input in-Agub759Qte1bCjtFqPrQDF hit its timeout of 300s
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Task's current input in-8uyz6zSISdztXkLjHnENrR hit its timeout of 300s
Task's current input in-iQ49BRhMJXl4c8VljKldTi hit its timeout of 300s
Task's current input in-1iPfxyhnjceEgu5J7oOHIG hit its timeout of 300s
Task's current input in-VnDQbngSFhPiPlR2DkEXTK hit its timeout of 300s
Task's current input in-yGSQnQ2tDuWPUK39kLbdAT hit its timeout of 300s
Runner terminated, in-progress inputs will be re-scheduled
Task's current input in-ZMullIUJhxeqlDaAugKgg1 hit its timeout of 300s
Task's current input in-uhzaLbc3xAVhNN82sRzXnk hit its timeout of 300s
Task's current input in-V6trTx9X3BVqLwn0Gcwpxk hit its timeout of 300s
Task's current input in-hW4mZr6GGHPmiddFeZe8f0 hit its timeout of 300s
Task's current input in-DIqVeGjAmqj9cYHhHmUg3z hit its timeout of 300s
Task's current input in-imMVvAMTNdM6pTFs18oI4I hit its timeout of 300s
Task's current input in-PzBJ3JWiAWSFm6d8yEo3zo hit its timeout of 300s
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled
Runner terminated, in-progress inputs will be re-scheduled

I also repeatedly run into this when trying to generate an app. I'm using gpt-4.

[JeffreyBoehme]

I had this with a large fairly complicated prompt and I figured this out. you have to update the main.py file and change @stub.function() to @stub.function(timeout=30000) This is for the API call to modal to update the timeout for the full program build length
and update any other timeouts in the four files anthropic.py, main.py, debugger.py, code2prompt.py These are for the API calls to ChatGPT, I set these to
initial_delay=20.0,
),
concurrency_limit=3,
timeout=3000,

and that seems to have done it for my api calls hitting limits.

Hope it helps you guys!

[swyxio]

thanks @JeffreyBoehme !