Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Games that can't be downloaded / have persistent non-fatal warnings though Gogrepoc but can be downloaded though the GOG website #60

Open
Kalanyr opened this issue Mar 18, 2022 · 24 comments
Open

Games that can't be downloaded / have persistent non-fatal warnings though Gogrepoc but can be downloaded though the GOG website #60

Kalanyr opened this issue Mar 18, 2022 · 24 comments

Comments

@Kalanyr
Copy link
Owner

Kalanyr commented Mar 18, 2022

Please use this thread to report games / items, that have problems downloading through gogrepoc but do download through the website (please include the website link you used to verify this (ie the one with /en0game1 or /18102 at the end, not the specific download URI )
Please separate out reports of items that do not download vs those that have persistent non-fatal errors.
@Kalanyr Kalanyr mentioned this issue Mar 18, 2022
Closed
@theg721
Copy link

theg721 commented Mar 26, 2022

I'm not 100% sure if this is the same issue as I'm not trying to download anything, just run the update command, but the following games in my library all have XML parsing errors + they can all be downloaded successfully via the website:

Hope this is the link format you were after—if not let me know, happy to provide any further details as necessary.

Full output is here: https://pastebin.com/JDZjczNL (also includes some request failed errors?)

@Ygg01
Copy link

Ygg01 commented Apr 2, 2022 •
edited
Loading

AI war 2 had a fail during update
https://gist.github.com/Ygg01/9405cfb46dd214edad82e5422bb29d4b

@Jaysyn904
Copy link

Cyberpunk 2077

gogrepo.log

@alzhur
Copy link

alzhur commented Apr 9, 2022

No man's sky (maybe error on gog side?):

15:31:52 | (1 / 1) fetching game details for no_mans_sky...
15:32:11 | xml parsing error occurred trying to get md5 data for setup_no_mans_sky_3.84_sentinel_84011_(54108)-1.bin
15:32:11 | The handled exception was:
15:32:11 | 
Traceback (most recent call last):
  File "./gogrepoc.py", line 692, in fetch_file_info
    shelf_etree = xml.etree.ElementTree.fromstring(md5_response.content)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1312, in XML
    return parser.close()
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1665, in close
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror
    raise err
ParseError: no element found: line 1, column 0
15:32:11 | End exception report.
15:32:27 | xml parsing error occurred trying to get md5 data for setup_no_mans_sky_3.84_sentinel_84011_(54108)-3.bin
15:32:27 | The handled exception was:
15:32:27 | 
Traceback (most recent call last):
  File "./gogrepoc.py", line 692, in fetch_file_info
    shelf_etree = xml.etree.ElementTree.fromstring(md5_response.content)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1312, in XML
    return parser.close()
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1665, in close
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror
    raise err
ParseError: no element found: line 1, column 0
15:32:27 | End exception report.

@pasbec
Copy link

pasbec commented Oct 18, 2022

I was able to solve all md5 issues for files that can manually be downloaded from GOG by one small modification in the function append_xml_extension_to_url_path:

diff --git a/gogrepoc.py b/gogrepoc.py
index 98cd8a1..968d92b 100755
--- a/gogrepoc.py
+++ b/gogrepoc.py
@@ -898,7 +898,7 @@ def is_numeric_id(s):
 
 def append_xml_extension_to_url_path(url):
     parsed = urlparse(url)
-    return urlunparse(parsed._replace(path = parsed.path + ".xml"))
+    return urlunparse(parsed._replace(path = parsed.path + ".xml")).replace('%28','(').replace('%29',')')
 
 def process_argv(argv):
     p1 = argparse.ArgumentParser(description='%s (%s)' % (__appname__, __url__), add_help=False)

Seems like GOG does not like the encoding of braces when requesting xml files in some cases.

@Kalanyr
Copy link
Owner Author

Kalanyr commented Oct 18, 2022 via email

@Sargon-Aelther
Copy link

Ahh. Thank you. I'm guessing what it doesn't like is the brackets that are actually part of the file name being replaced.

Sorry if it's a dumb question, but do you plan to implement pasbec's discovery?

@Kalanyr
Copy link
Owner Author

Kalanyr commented Oct 19, 2022 via email

@Wilstorm
Copy link

Wilstorm commented Oct 19, 2022 •
edited
Loading

I don't know if this error belongs in this thread or if it's enough information, just let me know if it's incorrect as I am new to the script. I have 733 games to download but it fails on game

(11 / 733) fetching game details for alien_isolation...

I tried multiple times and the error is always the same and it always the same game. The error

request failed: HTTPSonnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (3 retries left) -- will retry in 5s....

The first 10 games take a few seconds and everything seems to be working fine, the 11 just hangs for a bit, then it's just screens of error after error so I quit the script.

On a side note, I have 746 games in Galaxy but when I start downloading it only shows 733 to download.

Also, what a neat script and thank you for writing it! :)

@Kalanyr
Copy link
Owner Author

Kalanyr commented Oct 20, 2022 via email

@pasbec
Copy link

pasbec commented Oct 20, 2022 •
edited
Loading

Hi @Kalanyr, may I suggest to add some --verbose switch (true/false would do) to the main argument parser and use this to enabled/disable the detailed expection logging like this:

if args.verbose:
    warn("The handled exception was:")
    log_exception('')
    warn("End exception report.")

I repeatedly see that users are terminating the script or think there is some unhandeled error due to the detailed excpetion log.
The optional verbose option should be fairly simple to implement and would still allow you or developers to debug the code or tell other users to use it if they need help.

What do you think?

@Kalanyr
Copy link
Owner Author

Kalanyr commented Oct 20, 2022 via email

@pasbec
Copy link

pasbec commented Oct 20, 2022

I don't know if this error belongs in this thread or if it's enough information, just let me know if it's incorrect as I am new to the script. I have 733 games to download but it fails on game

(11 / 733) fetching game details for alien_isolation...

I tried multiple times and the error is always the same and it always the same game. The error

request failed: HTTPSonnectionPool(host='gog-cdn-lumen.secure2.footprint.net', port=443): Read timed out. (3 retries left) -- will retry in 5s....

The first 10 games take a few seconds and everything seems to be working fine, the 11 just hangs for a bit, then it's just screens of error after error so I quit the script.

On a side note, I have 746 games in Galaxy but when I start downloading it only shows 733 to download.

Also, what a neat script and thank you for writing it! :)

There seem to be still many games on GOG, where md5/xml information is missing. If you always end up with timeouts for one specific game, you'll probably hitting such an issue. @Kalanyr has already created a seperate issue to collect a list of affected games but there's probably not much that can be done from our side. The only solution that I can see is to contact GOG, let them know and hope they get it fixed.

@Wilstorm
Copy link

@Kalanyr @pasbec - Thank you for the reply and help. Just a quick question. If I do contact GOG what would I say? I am guessing they don't support any script issues. If that's a completely wrong I apologize for the assumption. I would gladly reach out to them and mention any issues that might help but I'm not quite sure what to point out as the issue. Gogrepoc download issues for the specific game or missing MD5/XML information? Thanks again guys for the quick replies and help.

@Kalanyr
Copy link
Owner Author

Kalanyr commented Nov 8, 2022

I was able to solve all md5 issues for files that can manually be downloaded from GOG by one small modification in the function append_xml_extension_to_url_path:

diff --git a/gogrepoc.py b/gogrepoc.py
index 98cd8a1..968d92b 100755
--- a/gogrepoc.py
+++ b/gogrepoc.py
@@ -898,7 +898,7 @@ def is_numeric_id(s):
 
 def append_xml_extension_to_url_path(url):
     parsed = urlparse(url)
-    return urlunparse(parsed._replace(path = parsed.path + ".xml"))
+    return urlunparse(parsed._replace(path = parsed.path + ".xml")).replace('%28','(').replace('%29',')')
 
 def process_argv(argv):
     p1 = argparse.ArgumentParser(description='%s (%s)' % (__appname__, __url__), add_help=False)

Seems like GOG does not like the encoding of braces when requesting xml files in some cases.

I have implemented this locally and am currently testing it. So far it seems like it helps a lot (no connection errors so far from 56 games + fewer content decoding errors ), but I've had one content decoding error so need to drill down and see if that's a different problem.

That's still a substantial enough improvement to justify integrating it, so thank you.

@Sargon-Aelther
Copy link

I assume this is still being tested and is not part of the recent update?

@Kalanyr
Copy link
Owner Author

Kalanyr commented Nov 17, 2022 via email

@pasbec
Copy link

pasbec commented Nov 17, 2022

Thank you very much for keeping this nice tool running and updated @Kalanyr!

@bassclarinetl2 bassclarinetl2 mentioned this issue Dec 18, 2022
Open
@BileDemon
Copy link

This is the complete list of games in my library that gogrepoc fails to fetch/download:

a_plague_tale_innocence
absolute_drift
alien_isolation
alien_isolation_-_the_trigger
batman_arkham_origins
battlevoid_harbinger
blade_runner
brigador_up_armored_edition
call_of_juarez
chicago_1930_the_prohibition
crosscode_ninja_skin
deadlock_planetary_conquest
dream_pinball_3d
dying_light_the_following_enhanced_edition
fallout_new_vegas
heroes_of_might_and_magic_v_-_tribes_of_the_east
imperium_galactica
ironclad_tactics
mafia_definitive_edition
mafia_iii_definitive_edition
metro_exodus
nikopol_secrets_of_the_immortals
openttd_12.2
parkan_the_imperial_chronicles
post_mortem_1.0
project_warlock
raiden
resonance
return_to_mysterious_island
riven_-_the_sequel_to_myst
sacred_2_gold
silent_storm_gold
space_empires_4_deluxe
starbound_1.4.2
stronghold_hd
sudden_strike_resource_war
sudden_strike_3
technobabylon_2_5a
terminator_resistance
the_last_blade_2_cm
the_mystery_of_the_druids
witcher3_de_goty
patch_the_witcher_3_wild_hunt_-_game_of_the_year_edition
thronebreaker_1.1
time_gate_knight_s_chase_de_gog
tomb_raider_1_german
trine_3_the_artifacts_of_power
unavowed_
the_undergames_v2.1
warhammer_skulls_2022_digital_goodie_pack
wasteland_2_directors_cut_digital_deluxe_edition_upgrade
wolfenstein_the_new_order
wolfenstein_the_old_blood
x3_reunion
zafehouse_diaries

All these games fail with either one of these error messages:

xml parsing error occurred trying to get md5 data for ...
unexpected connection error fetching md5 data for ...
unexpected content decoding error fetching md5 data for ...
request failed: 403 Client Error: Forbidden for url: ...
request failed: 404 Client Error: Not Found for url: ...

The errors are quite numerous. I didn't want to download all 55 missing games through the GOG website, so I can't verify if this works yet. I guess it should work that way.
At least I hope that the XML Parsing/Decoding problems can be resolved in a future version.

@Kalanyr
Copy link
Owner Author

Kalanyr commented Jan 23, 2023 via email

@BileDemon
Copy link

Oh ... you are absolutely right. I totally misunderstood what these messages meant. I was under the impression every error represented a single file that could not be downloaded.

Thank you for the clarification.

@idkicarus
Copy link

The script gives me a 404 error when trying to fetch the base game and patch files for Imperator: Rome. The extras and DLC are added to the manifest and downloaded without issue.

16:41:19 | (1 / 1) fetching game details for imperator_rome...
16:41:25 | request failed: 404 Client Error: Not Found for url: https://www.gog.com/downloads/imperator_rome/en1installer0.  will not retry.
16:41:25 | failed to fetch https://www.gog.com/downloads/imperator_rome/en1installer0
16:41:26 | request failed: 404 Client Error: Not Found for url: https://www.gog.com/downloads/imperator_rome/en1installer1.  will not retry.
16:41:26 | failed to fetch https://www.gog.com/downloads/imperator_rome/en1installer1
16:41:26 | request failed: 404 Client Error: Not Found for url: https://www.gog.com/downloads/imperator_rome/en1patch0.  will not retry.
16:41:26 | failed to fetch https://www.gog.com/downloads/imperator_rome/en1patch0
16:41:26 | request failed: 404 Client Error: Not Found for url: https://www.gog.com/downloads/imperator_rome/en1patch1.  will not retry.
16:41:26 | failed to fetch https://www.gog.com/downloads/imperator_rome/en1patch1
16:41:26 | request failed: 404 Client Error: Not Found for url: https://www.gog.com/downloads/imperator_rome/en1patch2.  will not retry.
16:41:26 | failed to fetch https://www.gog.com/downloads/imperator_rome/en1patch2
16:41:27 | request failed: 404 Client Error: Not Found for url: https://www.gog.com/downloads/imperator_rome/en1patch3.  will not retry.
16:41:27 | failed to fetch https://www.gog.com/downloads/imperator_rome/en1patch3

I can download these files from GOG's website without any problems.

The current base game files are:

  • setup_imperator_rome_2.0.3_rc2_(45615).exe
  • setup_imperator_rome_2.0.3_rc2_(45615)-1.bin

The current patch files are:

  • patch_imperator_rome_1.5.1_(40458)to_1.5.2(40614).exe
  • patch_imperator_rome_1.5.2_(40614)to_1.5.3(40956).exe
  • patch_imperator_rome_2.0.1_RC2_(44994)to_2.02_RC1(45377).exe
  • patch_imperator_rome_2.02_RC1_(45377)to_2.0.3_RC2(45615).exe

@Kalanyr
Copy link
Owner Author

Kalanyr commented Jul 6, 2023 via email

@idkicarus
Copy link

Sure enough, that worked. I tried again this morning and retrieved the files. Thanks for the quick response!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@Ygg01 @theg721 @Kalanyr @idkicarus @Wilstorm @pasbec @alzhur @Jaysyn904 @BileDemon @Sargon-Aelther