Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Until and while may cause an error in the unpiped stream. #23

Closed
MichalCz opened this issue Nov 29, 2018 · 2 comments
Closed

Until and while may cause an error in the unpiped stream. #23

MichalCz opened this issue Nov 29, 2018 · 2 comments

Comments

@MichalCz
Copy link
Sponsor Member

Consider this more a use case than a problem (as it is outside of scramjet).

Let's consider code:

StringStream.from(request(someUrl))
    .lines()
    .parse(JSON.parse)
    .until(x => x.date < Date.now())

What we do is unpipe the response, but that is not closed by request so how do we handle this?

Mention: openaq-fetch#547

@MichalCz MichalCz added this to the v4.20 milestone Dec 9, 2018
@MichalCz
Copy link
Sponsor Member Author

MichalCz commented Dec 14, 2018

The current solution has been deployed to OpenAQ adapters.

  const requestObject = request.get(stationUrl);
  return DataStream	  return DataStream
    .from(() => request.get(stationUrl).pipe(JSONStream('*')))
    .pipeline(
      requestObject,
      JSONStream('*')
    )
    .catch(e => {
      requestObject.abort();
      e.stream.end();
      throw e;
    })

Should it and can it be any cleaner - the abort operation is specific to request module...

@MichalCz MichalCz removed this from the v4.20 milestone Jan 6, 2019
@MichalCz
Copy link
Sponsor Member Author

MichalCz commented Jan 6, 2019

After giving this some thought it seems that the current solution is good enough.

@MichalCz MichalCz closed this as completed Jan 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant