failure to successfully connect to a slack with 30,000 users

[danhimalplanet]

slack-libpurple crashes when use it to connect to the Gophers slack. Gophers slack has thousands of users.

backtrace:
https://gist.github.com/5913b633afd6f67408b39d75649e5583

dylex/slack-libpurple commit id ee0f2bd

[dylex]

Can you get a debug log up to the crash? It looks like some http request is failing, and that error is not being handled well.

[danhimalplanet]

https://gist.github.com/danhimalplanet/6a6168eba295197d56b7eeab8eee7e78

[dylex]

Strange -- it crashes after getting a ratelimited response, and then disconnecting. I'm not sure why it's disconnecting. It has been loading users for 2 minutes by that point -- maybe something higher up timed out? The ratelimiting callback definitely fails to deal with disconnection, so would crash if something external disconnected in that time. Things we should try:

1. Figure out if anything (in bitlbee or libpurple?) has some connection overall connection timeout that we can lengthen or otherwise what's closing the connection
2. Tweaking the ratelimit_delay option (maybe lowering it a bit)
3. Finally adding the header parsing necessary for using the correct ratelimit delay
4. Fixing the crash itself, though this won't fix the underlying failure
5. Longer term, lazy-loading of users. I think the only thing the user list is used for now is searching @ references but if we limit those to channel members maybe that's easier (though I imagine #general has all the users, so maybe that just creates another problem)

[krisfremen]

I'm running into the same issue if I have a few of the slacks connecting at the same time.

[dylex]

I've been working on a lazy loading option on the "lazy" branch. The very basics are working at this point, if anyone wants to try it on a large slack, but it's definitely still in progress. Make sure you enable the "lazy loading" account option.

[renatolfc]

@dylex Just tested the lazy loading option (from the history branch, commit id b7f7d7c) to load a ~9k user Slack Workspace and it worked just fine. Without it, I'm unable to log into that workspace.