Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot get all Subscribers for a channel #172

Closed
BarryCarlyon opened this issue Jul 21, 2020 · 2 comments
Closed

Cannot get all Subscribers for a channel #172

BarryCarlyon opened this issue Jul 21, 2020 · 2 comments
Labels
product: api API endpoints in the "helix" namespace

Comments

@BarryCarlyon
Copy link
Contributor

BarryCarlyon commented Jul 21, 2020

Brief description

Using code similar to #18 but for the subscribers endpoint

I get a HTTP 500 error after getting a number of pages into downloading all subscribers for a channel

How to reproduce

Use code in #18 but for the subscribers endpoint
Download all the subscribers
Observe a 500 error when attempting to get the last page. Meaning the call to get all subs "fails" and you can't get all subscribers of a channel.

Expected behavior

Get all pages of subscribers from the helix endpoint

Additional context or questions

Original report on Discord

https://discordapp.com/channels/504015559252377601/523675960797691915/734923497398206566

Replicated by myself and on the forums

https://discuss.dev.twitch.tv/t/helix-subscriptions-api-returning-error-after-reading-subscriber-800-900-of-a-streamer/27248

@BarryCarlyon BarryCarlyon added the product: api API endpoints in the "helix" namespace label Jul 21, 2020
@BarryCarlyon
Copy link
Contributor Author

BarryCarlyon commented Jul 21, 2020

Test code.

Got is version ^11.2.0

const fs = require('fs');
const path = require('path');

const got = require('got');

var users = {};

let page = 0;
fetchPage = function(c) {
    console.log('Go', c);
    got({
        url: 'https://api.twitch.tv/helix/subscriptions',
        searchParams: {
            broadcaster_id: 'CASTERID',
            after: c,
            first: 100
        },
        headers: {
            authorization: 'Bearer ATOKEN'
        },
        responseType: 'json'
    })
    .then(resp => {
        page++;

        for (var x=0;x<resp.body.data.length;x++) {
            users[resp.body.data[x].user_id] = 1;
        }

        console.log('Page',page,'has',resp.body.data.length,'users',Object.keys(users).length, c);

        fs.appendFileSync(path.join(
            __dirname,
            'pages',
            page + '.json'
        ), JSON.stringify(resp.body,null,4));
        if (resp.body.pagination && resp.body.pagination.cursor) {
            return fetchPage(resp.body.pagination.cursor);
        }
    })
    .catch(err => {
        console.log(err, err.response.headers);
    });
    return;
}
fetchPage('');

@jbulava
Copy link
Member

jbulava commented Jul 21, 2020

A fix for pagination of subscriptions has been deployed to fix this issue. The pagination from #18 is a separate issue to address.

@jbulava jbulava closed this as completed Jul 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
product: api API endpoints in the "helix" namespace
Projects
None yet
Development

No branches or pull requests

2 participants