[feat] Top-level Guard Timeout #1233
Description
Kudos to XW for the original recommendation: https://discord.com/channels/1085077079697150023/1085077080653434922/1342022562032521268
Description
Currently, Guardrails provides a way for people to timeout when connecting to and fetching remote guards, however, when performing validation calls with guards there isn't a consistent or obvious way to signal failure if they take longer than a certain amount of time.
Why is this needed
For latency-sensitive applications, being able to assume an automatic failure after a set amount of time is an important guarantee. While it might not be possible to enforce a timeout for some synchronous validators (or those which depend on a local hardware-inference), it would be useful to try and support the async case if possible.
Implementation details
[If known, describe how this change should be implemented in the codebase]
End result
from guardrails import AsyncGuard
from guardrails.hub import RemoteValidatorOfSomeKind
guard = AsyncGuard()
remote = RemoteValidatorOfSomeKind(on_fail="fix")
guard.use(remote)
guard.timeout = 0.1 # 100ms
guard.validate("abc") # pass
guard.validate("abc"*1000000000) # Takes over 100ms -> raise exception.