Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

got.stream() not compatible with continue-stream #108

Closed
techniq opened this issue Sep 22, 2015 · 2 comments
Closed

got.stream() not compatible with continue-stream #108

techniq opened this issue Sep 22, 2015 · 2 comments

Comments

@techniq
Copy link

techniq commented Sep 22, 2015

I attempted to use got.stream() with continue-stream to stream multiple pages of API requests into a single stream but it appears to not work. The first page will pull down, but additional calls will not work. Here's is the snippet I'm using for testing (pulled from their README). It works fine with the request library's stream that the example uses.

var continueStream = require('continue-stream')
var request = require('request')
var got = require('got')
var pumpify = require('pumpify')
var JSONStream = require('JSONStream')

var page = 1

function next(callback, previousStream) {
  if (page >= 4) return callback()

  /*
  var req = request({
    url: 'https://api.github.com/repos/joyent/node/events?page=' + (page++),
    headers: {'user-agent': 'pug'}
  })
  */
  var req = got.stream('https://api.github.com/repos/joyent/node/events?page=' + (page++), {
    headers: {'user-agent': 'pug'}
  })

  var stream = pumpify.obj(req, JSONStream.parse('*'))

  callback(null, stream)
}

continueStream.obj(next)
  .on('data', function(data) {
    console.log(data);
  })
@floatdrop
Copy link
Contributor

@techniq seems like problem on the combine-stream side. I can recommend multistream as replacement:

var MultiStream = require('multistream')
var got = require('got')

var count = 0;
function factory (cb) {
  if (count > 3) return cb(null, null)
  count++
  setTimeout(function () {
    cb(null, got.stream('https://www.random.org/integers/?num='+count+'&min=1&max=6&col=1&base=10&format=plain&rnd=new'))
  }, 100)
}

MultiStream(factory).pipe(process.stdout);

@techniq
Copy link
Author

techniq commented Sep 22, 2015

@floatdrop Perfect, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants