Adding Multithreading

[Emersont1]

hey all

I know this is gaining some traction at the moment, so I thought I'd add some multithreading to try and make it run faster / more efficiently

I'm not going to merge this in until I've had several people test this, so I'd appreciate feedback in the comments

Installing:

# clone the repo
git clone https://github.com/Emersont1/itchio

# checkout to multithreading
cd itchio
git checkout multithreading

# create and load a venv
python -m venv venv
source venv/bin/activate

# install
pip install .

# profit?
python -m itchiodl.downloader -k INSERT_API_KEY_HERE

[mkolassa]

This ended up working great for me. Downloaded 768GB at 100MB/s with the default 4 threads.

[Emersont1]

I'm assuming that is faster than in single-thread mode?
I don't think it's worth adding unless there's a measurable speed boost (as the progress bars are removes)

[mkolassa]

I was only getting about 30MB/s with the single thread version.

[lem6ns]

Having too much jobs set causes ratelimiting (8 works fine, 10 starts giving errors)

[Emersont1]

I'll add that into the documentation