I'd like to thank everyone who helped me in the project. You're all awesome. Please be on lookout for the publication of the archives. The repo is now archived because it has served its purpose.
Hello!
If you're here, you must know that things aren't looking good. The Boards is
shutting down in a few days and Riot is planning to wipe the archives. To at
least preserve some of this great (and terrible) site, I wrote this program
which can be used to basically save everything from the forum.
## How does it work?
When you start up CHARON, it will ask you a few simple questions to know which
version of the Boards you wish to save. Then it will create a folder where the
executable is named "backup_{REGION}_{LANGUAGE}". After that, it will fetch the
posts and comments of whoever you named and saves them in this folder.
Meanwhile, it also collects every single name it encounters. Then it takes one
of those names and repeats this whole process. As you can probably imagine,
this will eventually exhaust every single topic and comment.
## How quick is it?
Both very quick and quite slow at the same time. The program itself uses
optimized libraries and Rust's already excellent speed. However, since you're
downloading from the internet, bandwidth and the inherent slowness of requests
makes the program run much slower than it could.
To give you a few concrete numbers:
The Hungarian Boards contains 21,000 topics from about 13,000 users.
Downloading this took about 2 and a quarter hours on my Ryzen 5 3600 processor
and 600 Mbps internet connection.
## How will I be able to access this data?
I already have a working version of the Hungarian dataset, which can be reached
here. If you click on the second "OK" button,
you can see that over 20 thousand posts load almost immediately.
The project can be reached
here, it shouldn't
take much to modify it to be able to use any dataset.
However, either way, once we have the content saved up, we can take as much
time as wish to release it again, the first priority should be to back it up.
## How do I run the project?
### Download
Just download the newest binary from the Releases Tab and run it.
### Run from source
You need to install Rust, navigate into
the folder where you downloaded the code and issue
cargo run --release
which will download the dependencies and run the program.
## How can you help?
While the program can handle the EUNE/HU region well, I haven't tested it
anywhere else, so if you can just do that, that's already very helpful.
Also, in the unlikely case you need to restart the application, it sadly has no
idea how much has it already processed, so it'll start again from scratch. This
is obviously undesirable, but I wanted to quickly whip up a working version. If
you can manage to patch this in, please open a pull request.
This is now added, but could be improved upon.
And, to be frank, any kind of improvement is very much appreciated. The Boards
might die, but its content doesn't have to. It all comes down to us.