Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cancel requests which are not possible #69

Closed
leszekhanusz opened this issue Oct 29, 2022 · 7 comments
Closed

Cancel requests which are not possible #69

leszekhanusz opened this issue Oct 29, 2022 · 7 comments
Labels
enhancement New feature or request

Comments

@leszekhanusz
Copy link

Currently, if you request an image using a model,
and this model becomes unavailable (last worker proposing it disconnected),
then you will stay at an estimated time varying between ~5 or 10 seconds,
without any indication that you're waiting for nothing.

Feature requests:

  • Directly refuse any request with a model which is not available
  • Cancel all the requests with a model if that model become unavailable
@db0 db0 added the enhancement New feature or request label Oct 29, 2022
@db0
Copy link
Member

db0 commented Oct 29, 2022

There is always a possibility that the model will come online though, in face I have a plan in mind to make workers load new models as demand changes in the waiting queue.

@leszekhanusz
Copy link
Author

ok, but currently you can request in the API any model name, even if it does not exist and it will go through.

@leszekhanusz
Copy link
Author

Some other cases to consider:

  • providing an invalid worker id should instantly return an error instead of going through
  • if you ctrl-c your worker during a generation, the client will wait indefinitely instead of receiving an error or sending back the request to the horde, even if you restart your worker right away

@leszekhanusz
Copy link
Author

And it is also the case if you try to request an image on your own worker with a width/height too big for your config

@leszekhanusz
Copy link
Author

Basically we should follow the fail-fast principle.

@db0
Copy link
Member

db0 commented Oct 30, 2022

I don't disagree. All these stuff is in my plan :D

@db0
Copy link
Member

db0 commented Nov 3, 2022

  • Request will be aborted if a bad worker_id is given
  • Jobs whose workers never returned, eventually expire and restart
  • models can still be specified freeform, but /check now returns a field is_possible, which is False when there's no available workers to fulfil the request (so you can notify the user to cancel)

@db0 db0 closed this as completed Nov 3, 2022
db0 added a commit that referenced this issue Mar 3, 2023
All-Ki pushed a commit to All-Ki/AI-Horde that referenced this issue Mar 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants