Missing clips when using an expanded date range

[BarryCarlyon]

Brief description

Larger date ranges omit clips from the response when using Get Clips

https://dev.twitch.tv/docs/api/reference#get-clips

How to reproduce

https://dev.twitch.tv/docs/api/reference#get-clips

Calling

https://api.twitch.tv/helix/clips?broadcaster_id=66016174&started_at=2020-03-01T12:00:00Z&ended_at=2020-03-30T12:00:00Z&first=100

Fails to return the clip

https://api.twitch.tv/helix/clips?id=TrappedColdSashimiTBTacoLeft

But

https://api.twitch.tv/helix/clips?broadcaster_id=66016174&started_at=2020-03-04T00:00:00Z&ended_at=2020-03-05T12:00:00Z&first=100

Returns it.

The second date range is inside the first so the clip in question should be returned in the first lookup. One assumes other clips maybe missing from "large" date range lookups.

Expected behavior

Return all clips created inside the date range

Additional context or questions

Report on discord starts here

https://discordapp.com/channels/504015559252377601/523675960797691915/689052568780931117

[mauerbac]

Thanks. HLX-1322

[BarryCarlyon]

Dupe of #48

So 48 has other context/testing

[Brandin]

@mauerbac Is there an update on this issue? I've verified these longer ranges continue to produce this problem. We're having to hit the API more (for example, 4x more if we use weekly vs monthly) which is only straining things to get data out. This is becoming to be an important problem given the decent uptick in DMCAs towards Clips. Thank you for your time!

[mauerbac]

pinging @jbulava

[lleadbet]

@brandinarsenault - Quick question for you- are the missing clips lower in view count than the last returned clip, or is it supposed to be somewhere in the middle?

For example, does the last clip have 5 views, but the missing clip have 100?

[Brandin]

Hi @lleadbet! I just ran a few queries to see what I could pull out, but I don't have an example as concrete as Barry's. I did, however, observe a very weird behavior, that where my limit is set to 100 in all of my queries, but often times its returning less than 100.

I performed 11 queries to the clips API:

• The first without a cursor (NULL)
• All with 100 limit
• StartDate of 2020-01-19T00:00:00-05:00, EndDate of now

The return:

• All but 2 of the 11 hits returned less than 100
• Based on my analysis of the Twitch Clips Page for the Broadcaster, the clips line up (the last of the previous is just before the first of the next, based on view count)
• The 11th call returned no cursor
• The total returned clips were 1056
• The last clip in the returned data (1056) has 2 views

My observations:

• The Clips API may return more, equal to, or less than the 1k limit that is documented
• The Clips API doesn't adhere to the limit parameter provided strictly as it should, it pushes the default but doesn't keep it strict
• I wasn't able to quickly find any missing clips in the middle when I did my own check
• The missing clips seem to be lower view counts than the last, at least in my case, which would make sense, based on the limit presented in the documentation