feat: gateway worker #195
feat: gateway worker #195
Conversation
scott-dn
force-pushed
the
feat/gateway-worker
branch
from
56b936c
to
4cd8b9e
Compare
| cid: String::from(k), | ||
| sender: tx, | ||
| })?; | ||
| let data = rx.await??; |
There was a problem hiding this comment.
@scott-dn I don't think this model [send request to node] -> [read response and cache it] -> [send respond to client] is going to work with stream piping the IO (client <-> node). Do you have some ideas?
There was a problem hiding this comment.
Yeah, with cache, we don't have to deal with stream data at the moment (we might want to stream from cache).
But for the case of files larger than 1GB, we don't want to cache them; we should stream all the way instead.
There was a problem hiding this comment.
How would we know the size before requesting content from the cache nodes or are you thinking that we will use content-lenght?
There was a problem hiding this comment.
block info will do.
There was a problem hiding this comment.
@theBeardA so what does the "block info" rpc method return?
There was a problem hiding this comment.
isn't it another round trip to the indexer?
I don't think so. Aren't we currently calling the global indexer, given a cid? That returns the metadata. So instead of adding another rpc method or calling ursa_get_cid to get the file size, we could add the file size in the metadata. Thoughts?
sounds good to me.
There was a problem hiding this comment.
That would be a lot easier then messing around with HTTP headers to make proxying decisions 😅
There was a problem hiding this comment.
Agree 💯 we should have an issue ticket for this. 🙏
There was a problem hiding this comment.
Here #206
Left some questions, notes, links from research.
see #122