Failing to download

[xcorvis]

I created a fresh virtualenv with python 3.10 on Ubuntu 22.04. The command appears to run but only the .json file is downloaded, neither of the two expected artifacts are downloaded. I ran pip install itchiodl and activated the virtualenv.

$ python -m itchiodl.downloader --api-key="REDACTED" --download-game "https://binary-star-games.itch.io/apocalypse-frame"
Downloading APOCALYPSE FRAME
Downloaded APOCALYPSE FRAME (1 of 1)

$ tree binary-star-games/
binary-star-games/
├── apocalypse-frame
└── apocalypse-frame.json

1 directory, 1 file

The game is owned (I can download it manually), there are no errors. There should be a pdf and an epub file.

$ pip list
Package            Version
------------------ -----------
args               0.1.0
beautifulsoup4     4.11.1
certifi            2022.12.7
charset-normalizer 3.0.1
clint              0.5.1
idna               3.4
itchiodl           2.1.2
pip                22.0.2
requests           2.28.2
setuptools         59.6.0
soupsieve          2.3.2.post1
urllib3            1.26.14
wheel              0.37.1

I tried two api keys, both had the same results. I also tried invoking with python -m and also with itch-download. No errors.txt file was generated. I also tried with -j 4 and -p all flags, same result.

Thanks!

[xcorvis]

I just tried downloading a freely available project, and it did download all 3 PDFs ( https://failbetter-games.itch.io/skyfarer ), so why one and not the other?

[xcorvis]

Pretty sure this is just a continuation of #57.

[Emersont1]

I think this is an issue with how the API was reverse engineered.

I'm currently trying to run the app through a proxy. I think the issue is that if the item is paid-for it requires an download_key_id or uploads comes back empty. there doesn't seem to be a way around this.

The only way i can see is to add the purchased item api to get the keys and then do the downloads.

[Emersont1]

are you able to test the code in #66 to see if the fix works for you?

[xcorvis]

Thanks for the quick turnaround! I tried it out on my purchased game (same command as in the original post) but instead of downloading the specified game, it started downloading every game. I had to kill it. I have a lot of games from a couple of big bundles, so I can't let it run through all of them to confirm. But there were some purchased games in the first few that downloaded successfully.

[Emersont1]

doh! i made a mistake with the order of trimming the list / exiting the loop, the new one should work.

it was only going to download the current loaded page (50 items)

[xcorvis]

That worked! It had to check 50 pages of purchases first, but I got the files.

Would it be possible to cache that info so you'd only have to run an "update purchases" job occasionally? My ideal scenario is to create a list of games I want to keep updated, and then I'd script jobs to grab updates. Pulling the same 50 pages of purchase info for each game seems onerous.

[Emersont1]

That does seem like a good idea on the whole. Tbh this system was designed more of a one and done "download the entire library onto a nas" tool instead of downloading an arbitrary game. Are you okay to spin this off into a new issue, because i think this will need a bit more of a redesign for it?

[xcorvis]

Yes, I can create a feature request for it. Thanks!