[BUG] Scraper not scrapping data?

[aivct]

Behaviour:

Expected Behaviour:

Steps to Reproduce:
No idea.
These bugs are most prevalent when there is an error on the main chat page itself, but that's just anecdotal for now.
Wait a long time (ie, go take a nap) and see what happens in the morning, because that usually causes weirdness.

Notes
For some reason, the scraper has kind of bugged out. I regret not saving any logs initially, but I just noticed it. The following are logs from when I opened it again. And now, it is still refusing to save the old logs.

There are probably at least two bugs here.

1. Scarper isn't properly saving new chats.
2. Scraper is being weird with recognizing and saving over old chats.

PS: Oh, and whitespace for humans are still wrong. Tabs and indents are not displayed correctly, ugh.

[aivct]

This may or may not be related. OpenAI has also triggered its own error at the same time.

[benf2004]

Something strange is going on right now. It's not working for me.

[aivct]

@benf2004 are you saying you can't reproduce the bug or that you are also reproducing the bug?

[benf2004]

The scraper is not really working for me right now.

[aivct]

I think I see one source of the issue.

The ID is unified, but the thread doesn't know that, so it does weird things like this:

If you look in the last index of t[], it'll give you the thread... with the wrong id, and the unified id is nowhere to be found.

Last time I checked, array[null] is invalid (actually any invalid values seem to just be ignored, gotta love JS failing silently)

Anyways, checking that the index isn't null and inserting it afterwards is a duct tape fix that I'll try. Better to save twice than to overwrite.

This STILL might not fix every cause, but let's just re-open the issue or get a new one in that case.