Status Quo of Threads and Forum Channels/Posts

[CanePlayz]

What's possible as of now?

Currently, there is no proper integration of threads or forum channels/their posts. However, you can still export threads/posts by passing their channel ID to the CLI.

Things to be discussed

Should threads be rendered inside their parent channels or as separate files?

There has been a lot of discussion on that topic in the early comments of #663. Instead of listing pros and cons again, I recommend you read through the discussion over there, it evaluates a ton of important points. Personally, I'm in favor of just exporting them as separate channels. Firstly for simplicity's sake and secondly because that's pretty much how Discord treats them as well.

How will date ranges and partitioning be affected?

Well, that depends on the solution we go for. There's been some discussion with possible solutions being laid out in the early comments of the aforementioned issue.

Integration in the parent channel/forum channel in HTML exports

1. Threads: When a new thread is started, a message that looks like this appears in the parent channel. I think integrating that element is not too unrealistic, but not having it shouldn't be a deal breaker either.
2. Forum channels: I think this isn't as much of a question as with threads, since having to replicate something like this just wouldn't be worth it in my opinion.

Look of the start of threads/posts in HTML exports

Currently, exported threads have a start that looks like this, while in Discord, the start of threads look like this

It gets a bit more complicated with posts, since they look like this.

Naming of exported threads and posts

Two ideas came to my mind:

1. Name the files like this: [[Name of parent/forum channel]] – [[Name of thread/post]]
2. Put all threads of the same parent channel or all posts of the same forum channel into one folder. Then we could even leave out the name of the parent channel in the file name.

Problems yet to solve

- As far as I know, there is no way of getting a list of archived threads from the API. This limits exportguild to only exporting channels and active threads.

It's been pointed out to me that endpoints for this do exist.

Related issues

#663 Main discussion on rendering threads
#738 Make threads show up on the channel command
#792 Shows threads in the GUI channel list

If any other problems that I've missed come up in this discussion, I might add them to this initial discussion post for the sake of completeness.

[Blacklands]

As far as I know, there is no way of getting a list of archived threads from the API. This limits exportguild to only exporting channels and active threads.

There is, as far as I am aware. See https://discord.com/developers/docs/topics/threads#enumerating-threads
I've used this in the past when working on a bot (though I didn't call the API directly, I used a higher-level library), and it seemed to work fine? Although my test channel had only 5 archived threads or so, maybe you hit a limit at some point when you try to go too far back in time; I can't say.

The only thing I didn't try was whether you can actually access the contents of these threads without "resurrecting" them. But I've got told in the #663 discussion that this is possible, too.

[CanePlayz]

Oh, good to know 👍

[Thirtybird]

Separate files all the way. It's a "side-discussion" not to interfere with the main channel topic.

[CanePlayz]

Yeah, I agree.

[Blacklands]

I agree too.

Also, as far as the API is concerned, a Thread basically is a Channel. Just a special kind that shows up differently in the UI and follows some unique rules. It's connected to its "parent Channel", but it's not some sort of "piece" of a Channel.
So having it be its own file, just like "regular" Channels are, makes more sense to me.

[kalomaze]

What's the best way to export a bunch of threads separately right now? There's over 1,000 listed threads with downloads attached that I'm trying to save. One by one is not realistic.

[Blacklands]

I think one by one is the only way right now, unfortunately. Someone still needs to get around to implementing saving all threads on a server. The puzzle pieces are mostly there, someone just needs to put them together.

[EDLLT]

For now, I am using postman. Created a test script and some variables. Namely, offset, threads, and url. The offset's increased by 25 everytime since that is the amount of threads returned per request and then the request is repeated with the updated offset. The amount of times it needs to be run is going to be TOTAL_THREADS/25. The TOTAL_THREADS could be found by looking at the same GET request which requests for threads. Of course, I could've scripted a python script to do this but I was too tired and lazy lol and postman did the cookie handling and whatever perfectly. I then take all these thread ids and throw them into the cli

[MichaelPrinceps]

For now, I am using postman. Created a test script and some variables. Namely, offset, threads, and url. The offset's increased by 25 everytime since that is the amount of threads returned per request and then the request is repeated with the updated offset. The amount of times it needs to be run is going to be TOTAL_THREADS/25. The TOTAL_THREADS could be found by looking at the same GET request which requests for threads. Of course, I could've scripted a python script to do this but I was too tired and lazy lol and postman did the cookie handling and whatever perfectly. I then take all these thread ids and throw them into the cli

[MichaelPrinceps]

Can you please share your code with us ?

[UnforeseenOcean]

A PowerShell script I made for archiving threads that you get from the commands:

$global:token = Get-Content ./bait.txt
$global:last = Get-Content ./last.txt

$time = Get-Date -Format "yyyy-MM-dd HH:mm"
$date = Get-Date -Format "yyyy-MM-dd"

$global:threads = "./threads2.txt"
$n = 0
foreach($line in Get-Content $threads){
	dotnet DiscordChatExporter.Cli.dll export -c $line -t `"$global:token`" -f Json --after `"$global:last`" --media -o `"./backups/My Lovely Server/threads/$global:date/`"
	$n = $n+1
}
Set-Content -Path ./last.txt -Value $time

1. Put your token in bait.txt (it is named as such so blanket search doesn't work)
2. On last.txt, put something like 2015-01-01 00:00 (it will automatically get updated)
3. And finally on threads2.txt you put all the channel IDs of threads you want to archive.
4. If you don't want to archive after certain date and instead want to archive everything, remove --after "$global:last". You can further modify the script to do what you want.

The script will automatically iterate through each line (each line must contain exactly one ID and one ID only -- 123456789012345678 is invalid, and so is 123456789012345678 | The Thread)
To run the script, go to the current directory where the script file lives and run ./scriptfilename.ps1 .

[1011025m]

Here's a script to pull a list of threads inside a forum channel. It'll ask you to put your auth token and target forum channel ID.
It'll generate a txt file in the working directory, you can then pipe the thread list to the exporter however you like.
Wrote it last week, think it'd be helpful.

https://gist.github.com/1011025m/6c2109cdb0178082aea1db4ddab585e9