Large volumes of tweets silently crashes tweet provider

[ALazyMeme]

So this issue really only happens if you're following donald trump - and as a result, I can't test with any other user on twitter.
It appears that due to trump receiving such a large volume of tweets, tweet provider crashes or disconnects and stops working. If someone can find a similar twitter user, I can do more testing, but when I unfollowed trump on fabzeef's bot, the crashing stopped.

[ALazyMeme]

fyi I'm testing this by following a large amount of users on twitter:
https://i.alazymeme.com/LiquidGreatImage.png

https://en.wikipedia.org/wiki/List_of_most-followed_Twitter_accounts - I'm up to no. 32

[ALazyMeme]

Up to 37 now, have added a few extras on the side: https://i.alazymeme.com/AcuteElderlyImage.png

[ldesgoui]

Could you give me IDs of those users in text form in order to test?

[ALazyMeme]

Could you give me IDs of those users in text form in order to test?

I'm so confused...I can't even reproduce with this many users. It only seems to happen with trump? Which makes no fucking sense.

[ALazyMeme]

Need to do more testing on this. It didn't crash overnight last night, but I followed trump again in the morning and it crashed throughout the day. Could it be possible that it's something specifically to do with his account?

[ldesgoui]

Please try #15 via this build https://github.com/pajbot/tweet-provider/actions/runs/147358717

[ldesgoui]

This should also be addressed, it is most definitely the cause of the issue here: twitter task panics, my initial expectation was that the panic would propagate to the rest of the program - it does not - my second expectation was that any task panicking would cause every other task to panic as channels will close partly, it is apparent that this is not the case, as the listener task is still accepting connections while the twitter task is down.
I may look at it later today

tweet-provider/src/main.rs

Line 69 in 92c0ef6

// TODO: replace tokio::spawn with a select, allowing us to return errors