You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[for the record: this was proposed by @TimDettmers ]
Currently, hivemind-server treats all requests on a first come first served basis.
If we want to reward active participants with faster inference/training, we could change that into an auction.
Here's how client-server interaction looks like:
server gives client its stats, the current-highest bid, and maybe some metadata for bidding, e.g. the lowest serviced bids over last T seconds
in TaskPool.priority, server will now set pool's priority based on highest bid in the pool, instead of wait time
As suggested by @GreenFatGuy , we need to think through how to deal with situations when low bids on high-demand servers won't ever be processed, and will hence take up memory on both client and server. First order solution: add absolute expiration time to each request, drop requests that hit expiration time.
The text was updated successfully, but these errors were encountered:
[for the record: this was proposed by @TimDettmers ]
Currently, hivemind-server treats all requests on a first come first served basis.
If we want to reward active participants with faster inference/training, we could change that into an auction.
Here's how client-server interaction looks like:
As suggested by @GreenFatGuy , we need to think through how to deal with situations when low bids on high-demand servers won't ever be processed, and will hence take up memory on both client and server. First order solution: add absolute expiration time to each request, drop requests that hit expiration time.
The text was updated successfully, but these errors were encountered: