readString returns Nothing, but pipe shows data

[kobble-git]

I'm having a curious problem when I use Node.HTTP.Client. When I try to use readString on the response, I get Nothing but if I pipe the responseStream to stdout (for instance), I can see the data. Specifically, I am using Node.HTTP.Client to get an access_code from a github oauth2 flow.

I'm posting this here because it looks like a stream issue, and to look for any guidance as to what I could look for.

Here is the code:

affRequest :: forall eff. Options RequestOptions -> (Request -> Eff (err :: EXCEPTION, console :: CONSOLE, http :: HTTP | eff) Unit) -> Aff (err :: EXCEPTION, console :: CONSOLE, http :: HTTP | eff) (Maybe String)
affRequest opts f = makeAff \_ cb -> do
    req <- request opts \res -> do
          let responseStream = responseAsStream res
          mc <- readString responseStream Nothing UTF8
          pipe responseStream stdout
          cb mc

mc is Nothing, but I can see the correct data (the access_code) on stdout.

Cheers.

[kobble-git]

Never mind, I had to use onDataString. I guess readString is synchronous, and there was no data yet.

[hdgarrood]

Yep, exactly. I would also add that neither of onDataString or readString will make any guarantees about whether you've received the whole response, so you usually need to collect all the chunks and wait for the stream to tell you that it's finished before continuing. There's a node package called concat-stream that does this, which I used in Pulp: https://github.com/bodil/pulp/blob/df822a37a68c17727471a91c1948e167d953d582/src/Pulp/System/Stream.js but I think it would be fairly straightforward to write the same code in PureScript too (if you felt like it).

[kobble-git]

Thanks for the note about concat-stream, I'll take a look. Do you know of
any simpler server side http client libraries? Something like fetch in the
browser?

Cheers!

Bill

On Tue, Nov 15, 2016 at 12:48 PM, Harry Garrood notifications@github.com
wrote:

Yep, exactly. I would also add that neither of onDataString or readString
will make any guarantees about whether you've received the whole response,
so you usually need to collect all the chunks and wait for the stream to
tell you that it's finished before continuing. There's a node package
called concat-stream that does this, which I used in Pulp:
https://github.com/bodil/pulp/blob/df822a37a68c17727471a91c1948e1
67d953d582/src/Pulp/System/Stream.js but I think it would be fairly
straightforward to write the same code in PureScript too (if you felt like
it).

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#12 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ARoo1BiSKDV1WIRbquomkEsBl1qXWoyIks5q-PNBgaJpZM4Kx8Bi
.

[hdgarrood]

https://github.com/slamdata/purescript-affjax is primarily designed for browsers but works in node.js too. You might also consider wrapping an existing node.js HTTP client library, like request or got.

[kobble-git]

affjax looks good. I can use the same thing from both the client and
server. Cheers.

On Tue, Nov 15, 2016 at 3:48 PM, Harry Garrood notifications@github.com
wrote:

https://github.com/slamdata/purescript-affjax is primarily designed for
browsers but works in node.js too. You might also consider wrapping an
existing node.js HTTP client library, like request or got.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#12 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ARoo1I4vWt5ITcmYj2cGs3mPPBIM54qIks5q-R1ggaJpZM4Kx8Bi
.

[jamesdbrock]

Asynchronously read all the data from a stream: https://pursuit.purescript.org/packages/purescript-node-streams-aff/1.1.0/docs/Node.Stream.Aff#v:readAll