-
Notifications
You must be signed in to change notification settings - Fork 403
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a batched
API to ZClient
#3008
Conversation
@kyri-petrou then you would add explicit streaming methods? |
@987Nabil You mean adding add methods for |
Yes. Make the |
@987Nabil I ended up going down a little different path in the end. Instead of re-implementing all of the methods to have variants that don't require a client.simple.get("/foo")
client.simple(Request.get(???))
client.simple.request(Request.get(???)) |
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #3008 +/- ##
==========================================
- Coverage 65.65% 65.62% -0.03%
==========================================
Files 155 155
Lines 10134 10142 +8
Branches 1899 1909 +10
==========================================
+ Hits 6653 6656 +3
- Misses 3481 3486 +5 ☔ View full report in Codecov by Sentry. |
@kyri-petrou I thought this is not working for streaming bodies, when we don't handover a callback or? |
@987Nabil it is working for streaming bodies. What happens with streaming bodies is that the request body will be materialized in memory. Basically these two are equivalent when we have a streaming body: client.simple.get("/foo").flatMap(_.body.asString)
ZIO.scoped(client.get("/foo").flatMap(_.body.asString)) The only usage where something is different is this: ZStream.unwrap(client.simple.get("/foo").map(_.body.asStream))
ZStream.unwrapScoped(client.get("/foo").map(_.body.asStream)) In this case, the resulting stream from the |
@kyri-petrou idk if this is not a problem. You don't necessarily know, if the server returns a chunked stream. Or if it will fit in memory. I see the bug report already coming 🙈 |
But this would still be a problem if they don't explicitly stream the response, even with the current API. And it's only a problem if a user decides to use the The only way to make this can be made more safe is for the I documented the caveat of the |
@kyri-petrou I am not sure about simple as a name I'll give it some thought |
@kyri-petrou Here's an idea I just want to float for purposes of discussion -- I am not 100% convinced, but it has some promising elements. Let's say Then a streaming client must clearly be What have we achieved in this design?
|
One more comment, only partially related to this PR, and mostly just for purposes of future PRs: essentially, for purposes of best-practice architecture, I think all functionality should be directly exposed on trait/class. This means no functionality can be implemented only in terms of |
# Conflicts: # zio-http-example/src/main/scala/example/ServerSentEventAsJsonEndpoint.scala # zio-http-example/src/main/scala/example/ServerSentEventEndpoint.scala # zio-http-example/src/main/scala/example/endpoint/CliExamples.scala # zio-http/shared/src/main/scala-2/zio/http/UrlInterpolator.scala
@jdegoes I reimplemented the client so that now there is an In line with your last comment, I'm considering is to remove the |
@kyri-petrou I am not sure, if we should remove env. Right now, it is buried deep in the lib and average joe would never touch it. But I could imagine, that it could enable some future features. Since the user facing API is only the returned env that by default is always Any, I'd tend to leaving it in. |
|
||
final case class ZClient[-Env, -In, +Err, +Out]( | ||
final case class ZClient[-Env, S >: Scope, -In, +Err, +Out]( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- In general, I am not a fan of constraining type parameters at the declaration site, unless absolutely strictly required. The constraints add a lot of work for no real benefit and also interfere with evolution.
- Because this
S
is added to each request response (covariant), and there it appears in contravariant position, we should be able to make the parameter contravariant, unless I'm missing something.
This would change the signature like so:
final case class ZClient[-Env, S >: Scope, -In, +Err, +Out]( | |
final case class ZClient[-Env, -S, -In, +Err, +Out]( |
In which case I might rename:
final case class ZClient[-Env, S >: Scope, -In, +Err, +Out]( | |
final case class ZClient[-Env, -ReqEnv, -In, +Err, +Out]( |
"The env added to requests."
Or something similar (AddEnv
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not a fan of constraining type parameters at the declaration site
Main reason that I went with the constraining type is that removing it doesn't affect binary compatibility, so I thought perhaps it's better to be more strict and remove it if needed. But I agree I don't think that it adds much benefit so I'll remove it
we should be able to make the parameter contravariant, unless I'm missing something
@jdegoes the problem with making it contravariant is that then this becomes possible:
val batchedClient: ZClient[Any, Any, Body, Throwable, Response] = ???
val streamingClient: ZClient[Any, Scope, Body, Throwable, Response] = batchedClient
This is a problem because once we've eliminated the Scope
the client cannot stream thereafter. The only way to enforce it AFAICT is via an invariant type param
docs/reference/client.md
Outdated
|
||
```scala | ||
object Client { | ||
def batched(request: Request): ZIO[Client, Throwable, Response] = ??? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thinking more about this (good) naming convention: could it make sense to rename Client.request
to Client.streaming
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this is a widely used method, I ended up keeping Client.request
and annotated it deprecated so that we can point users to the new methods without breaking their code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure about the deprecated. Then we have to keep this method in until 4.x
I think we should remove it and write migration docs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a couple comments, but looks great! Beautiful API.
# Conflicts: # zio-http-example/src/main/scala/example/endpoint/CliExamples.scala
@jdegoes I applied all your recommendations minus making the I also updated the docs to first introduce the "streaming" client (i.e., default) followed up by the "batched" client/methods. I hope this will be clearer for users to follow |
/fixes #2956
/fixes #2957
/claim #2956
/claim #2957
In a nutshell, this PR adds the following methods and updates docs/examples/tests to change most usages of
Client.request
to these new methods. The first 3 will materialize the body in memory, whereas the latter will embed the Scope into the ZStream, which will be closed when the stream ends.@987Nabil @jdegoes, I'm REALLY tempted to also change the
get
,post
,put
,patch
,delete
andhead
methods onClient
to also materialize the body in memory. The issue with this though is that if there are users using these methods to stream responses, then we'll be messing things up for them.Wdut? Should we change those methods as well and document it very clearly in the release notes, or leave them as is and hope for the best that users are handling the
Scope
properly?