You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There was a fantastic idea from a Redditor about making EnergeticAI run well in Cloudflare Workers, so you can have super fast inference at the edge, without the need to distribute your model weights:
This task is to add support for Cloudflare Workers to EnergeticAI.
Approach
Given that Cloudflare Workers have even more restrictive bundle limits than AWS Lambda, I suspect the way to do this would be to distribute sharded model weights in Cloudflare KV, and then fetch from that in parallel on function invocation. On paper at least KV values should be colocated with the functions enough that this should be fast.
The text was updated successfully, but these errors were encountered:
Background
There was a fantastic idea from a Redditor about making EnergeticAI run well in Cloudflare Workers, so you can have super fast inference at the edge, without the need to distribute your model weights:
https://www.reddit.com/r/tensorflow/comments/1493uoq/comment/jo6axc9/?utm_source=reddit&utm_medium=web2x&context=3
Goal
This task is to add support for Cloudflare Workers to EnergeticAI.
Approach
Given that Cloudflare Workers have even more restrictive bundle limits than AWS Lambda, I suspect the way to do this would be to distribute sharded model weights in Cloudflare KV, and then fetch from that in parallel on function invocation. On paper at least KV values should be colocated with the functions enough that this should be fast.
The text was updated successfully, but these errors were encountered: