Problem Running Flash.py

[Tanaz-R]

Hello, I have seen that you already have an open issue regarding how to produce the result of the agents, I think I do not have time until it fixed, so I am spending money try at least run specific problems by Flash but most of the time it stucks at this point and never proceed or if proceed after 20 runs for port-misconfig I have 0% accuracy of it. Could you help me in this regards, or check what is worng there, please?

[Tanaz-R]

I tried scale to Zero, it exceed the valid token limit for GPT-4o, I cannot see if it can submit and success or not, I am done with my project and just need results to compare, I cannot use your table result in the paper as I cannot run my agent 48 problems so much money...

but I am wasting time and money and earning nothing. I am so frustrated I would appreciate any help.

[HacksonClark]

@Tanaz-R Don't send screeneshots of code, that is bad practice.

The rate limit has nothing to do with the benchmark, sadly it seems like you just do not have the right usage tier. That's an issue between you and the LLM provider and doesn't have anything to do with AIOpsLab.

Closing.

[Tanaz-R]

@HacksonClark, I know the second problem has nothing to do with the benchmark but the first one is, Flash stuck in port-forwarding state and never proceed I wait 5 min nothing will happen, run again, same issue has happened.

[HacksonClark]

@Tanaz-R the port-forwarding seems to be established. Is there any other debug messages? Could you print some? I don't really have much to go on from your output.