Query results are limited and I can't figure out why

[alboss]

I've been trying to figure this out for a few months now in my spare time, and whatever's going on isn't something I'm seeing my way through.

The command

in:a11y sort:desc limit:750

should search the #a11y channel in reverse chronological order and return the most recent 750 messages. As you can see from the attached, it hasn’t even returned 100.

I’ve checked the database and run the exact same query, and I know there are quite a few more results, dating back for years.

I ran the same query but in ascending order and with a ridiculously high number:

in:a11y limit:50000

and received back slightly more results—about 124.

I have to admit, I’m puzzled!

I’ve attached PDFs of what happened with the two queries above.

mr-atoz-descending-dates.pdf
mr-atoz-ascending-dates.pdf

I originally assumed I had a setting wrong with the DigitalOcean app itself, but their support staff didn't seem to think so:

---

From: support@digitalocean.com

From what we see, in general there are no hard limits that we impose on your App. The data you shared doesn't seem to be related to any limitation in-app platform. There are no limitations from our side on the requests. The only limitations are based on tier which includes the outbound data transfer.

After reviewing your app, we did notice that you are using a basic tier. There are few features that are provided in basic tier. Refer to below link for more insights:

https://docs.digitalocean.com/products/app-platform/#feature-comparison-by-tier

We did check your app's hardware metrics and all seems to be well under the control without any issues!

---

The database is updating just fine. Everything goes in, but not everything comes back out.

I thought maybe it was cutting off at a number of characters, a number of lines, a number of words, or a number of bytes, but none of that seems to be the case.

I am truly, genuinely stumped. Has anyone else encountered this kind of odd behavior, or (better yet) do you see what painfully obvious thing I'm either missing or doing wrong?

Perplexedly,

Al

[docmarionum1]

Hey! Pretty sure you're hitting slack's message character limit: https://api.slack.com/changelog/2018-04-truncating-really-long-messages

What's your use case here? Do you actually want to be able to get everything from a channel? If that's the case, I wonder if exporting it to some other format would be a better solution than wading through an entire channel history.

To support this within slack, probably paging is the only option.

[alboss]

Jeremy, I knew you'd see the obvious thing I'm missing! Thanks for that. In hindsight, that was obvious. I'm glad it wasn't something I'd done wrong.

History: My workgroup had been using Slack quietly (because it's not a Microsoft product and therefore not officially sanctioned) for years with no issue. When we switched to telework, lots of people suddenly became enlightened as to why my little group eschewed the approved tools (Skype for Business at first, then Microsoft Teams), and lots more people piled onto our Slack instance, and it started getting pretty heavy use. Slack gave us the three-months-for-free option and then suddenly the limits kicked in.

My original use case for the archive bot were pretty much the normal one: be able to retrieve things that we'd discussed in the past. This worked fine as long as I knew exactly what I was searching for, but it got tricky when trying to find mentions around a nebulous concept where I don't remember an exact term used in the conversation. Still, we could mostly get by.

Then came my current issue, wherein I needed to build a record of everything that happened in a channel, so we could refer to previous discussion threads, or find and organize links. In hindsight, I should have spun up a different tool for these kind of things, and in fact I'm in the process of doing that for accessibility conversations going forward--the free version of Slack is great for ephemeral conversations but web accessibility is a long conversation and would be better served by something like a blogging tool with comments turned on. It's just hard to get the larger group to opt into Yet Another Tool.

I'm sure the employer would want us to use SwearPoint or Teams for this, but since my plan is to have much of the conversation open beyond just within the company, I don't want it tied to their tools.

And you're right--I've pulled the data from the database, transformed it into HTML, and am going to put it into whatever tool I wind up using.

Best,

Al

[docmarionum1]

Great; let me know if I can be of help in extracting the data from the DB. I'll close this for now.