Version 1.0.4 Alpha

[ABaumher]

As of this moment, we're in alpha. I'm not announcing it, but i'm also not making it private. It is now public.

At this point, the code works, but it doesn't work well. There are things that need fixing, most notably when you do something wrong. So that's what we really need you to do. Provide bad passwords. Use illegal or nonstandard characters (diacritics, emojis, etc). Provide usernames that don't exist. Forget your password*. Wait an hour on the steam guard code so it expires. That sort of thing. The mobile steamguard page is probably the thing that needs the most testing, because i don't have steam guard and haven't checked it. One of our devs who does use it had managed to trigger an unexpected error and we could use help replicating it.

If you are on MacOS, you are doubly valuable because most of us are on Windows and we need it to work everywhere

The instructions are in the readme. We've simplified this process as much as possible, but we're not providing a one-click solution because there isn't a good one-click solution - you may have multiple instances of python (or not), and frankly checking that is beyond our scope here. You may have set up the virtual environment wrong, etc. Or, we may have told you the wrong thing to do, lol.

Regardless, the instructions are on the readme. See: https://github.com/ABaumher/galaxy-integration-steam/blob/master/README.md

Good Luck, and happy bug-finding!

*I guarentee you this is bugged. We're going to revert the start page to be username and password, which should indirectly fix this by forcing us to update the RSA key.

[ABaumher]

@OldManLoki @flamusiptheranji @Argalt @HiddenAnomaly @nakkamarra

[ABaumher]

If you get stuck during setup, i can walk you through it. I should respond within the hour if i am awake, but it may be a little longer if i'm eating or AFK, etc. If you prefer to speak another language over English, do so. I can use a translator.

[ABaumher]

Just a heads-up, i'm on EDT or UTC-4 so i might be asleep when you are awake, unfortunately. if there's some overlap time that works best for you just for the initial setup, let me know.

[ABaumher]

Known bug: Your stats may not sync (play time, achievements). We don't send those messages right now. it's on our to-do list. It's trivial, but the old way was a huge hacky mess and we're going to do it better

[ABaumher]

@woj1993 what step are you at in the instructions?

[ABaumher]

yeah, i see that. Sorry, was working on a merge to cleanup my crappy proto file system.

run py.exe -3.7 -m venv .venv
EDIT: then .\.venv\Scripts\activate.ps1
then pip install pip==22.0.4
it might complain you are downgrading or upgrading pip, but as long as it works you can ignore it

[woj1993]

Well I prepared install script in meantime that automates installing on Windows. Will fork and ask for merge.

[ABaumher]

Cool! LMK when you got it and i'll take a look.

[ABaumher]

When the code is stable it'll be available through GOG and no one will have to do this but it'll be really useful for developers
'

[woj1993]

@ABaumher For you. ABaumher#2
I'm going to sleep. See you in the morning.

[ABaumher]

When you hit a bug, please explain what you were doing, the exact steps you took to get there, and then attach your log.

[ABaumher]

Found it. Holy crap, we are dumb. Give me a second to fix it.

Edit: we aren't dumb, Python is. any other language would have detected the types didn't match, but Python happily converted it and we never noticed. I guess we just got lucky with email code being the same value.

[ABaumher]

i'll push the change, but if you don't want to bother pulling the latest, i'll post the protocol_client as a text file here. Rename to .py and replace the file in the plugin.
protocol_client.txt

[ABaumher]

For context, we got lucky af on email code:
k_EAuthSessionGuardType_EmailCode = 2; and EmailCode = 2
meanwhile, mobile:
k_EAuthSessionGuardType_DeviceCode = 3; and PhoneCode = 1
Steam thought we were giving them a k_EAuthSessionGuardType_None, hence the InvalidState

[Amarinth]

Ahhh, yes that would explain why it would sit there doing nothing for quite a while, before failing to connect.

As you predicted, it works as it's supposed to now! Signs in almost instantly and seems to get the right login token, so the connection persists through a restart of Galaxy. Thanks for the quick replies btw!

Now that I've established it works, I guess it's time to see if I can do anything to break it on purpose...

[ABaumher]

you got lucky i wasn't busy, otherwise it might've been a half hour or more.

Glad it works now. Get destructive!

[ABaumher]

Well, we're succeeding in finding bugs, but you can't run it yet, lol. Feelsbad. When we finished removing the public profiles, apparently a dependency they use was also indirectly used in steam network. and not noted anywhere. whoops! I'll have a patch up in a minute

[woj1993]

I have a question. Where this integration stores logs?

[ABaumher]

%systemdrive%\ProgramData\GOG.com\Galaxy\logs

[ABaumher]

Update the repo to patch out the bugs. BeautifulSoup is now part of app.txt in requirements

[ABaumher]

Possible Issue: The first time you close GOG Galaxy after logging in from a fresh install, the client may hang. I think this is an issue with the job list. That thing was the biggest ball of hacks and we just left it there because it "worked"

[engiefox]

Success! Had to click retry once to get it to sync, but I think it was because I clicked Continue before accepting the Steam Guard request. It asked for a Steam guard code after, but then just showed "Offline". Clicked retry and it asked for another Guard request, made sure to accept before clicking Continue and it worked perfect.
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

Lmao i have a TODO right above where you errored.
in %localappdata%\GOG.com\Galaxy\plugins\installed\steam_ca27391f-2675-49b1-92c0-896d43afa4f8\steam_network\protocol_client.py
Replace

        #TODO: This will likely error if the code is bad. Figure out what to do here. 
        else:
            raise translate_error(result)

With

        #TODO: This will likely error if the code is bad. Figure out what to do here. 
        else:
            logger.error("result: " + str(result))
            raise translate_error(result)

the function this is in is async def check_auth_status(self, client_id:int, request_id:bytes, auth_lost_handler:Callable) -> Tuple[UserActionRequired, Optional[int]]:
you can use Ctrl-F on check_auth_status to find it.

Then do not accept the mobile confirmation (ignore it), click "Continue" on the Mobile Confirm GOG page, and send me the log

EDIT: if you pull/download the latest version, this is done automatically.

[engiefox]

OK, here you go. Steam Guard requires you to either approve or deny a Guard request before it'll show you a Guard code, so I clicked Continue and then approved the request so I could get a code to advance through the UI. It did the same as before, tried syncing for a min, then showed "Offline".
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

It gives us a FileNotFound. why in the hell? Makes no sense, but ok then, that's the mobile confirm error, I can handle it. I'll have a patch up in a minute.

[ABaumher]

Pushed a change to the repo. It should fix that bug. It's not a single file, so you'll have to get it there. If you clone the repo all you need to do in powershell is activate the virtual environment (./.venv/<whatever>/activate.ps1) and run inv install. if you download a zip, copy all the files in the zip and replace wherever you have the code now with the new files. then, activate environment in powershell and install. Or you can do the whole install process again from this new version, whichever you prefer.

Lmk if it still errors

[ABaumher]

Edit: Never pushed that to the repo. my bad, lol.

[ABaumher]

Anyone testing as of Wed May 17 19:02:21 2023 UTC (the time this is posted), i pushed a minor change to the code that should make debugging much easier. Basically, whenever we raise a GOG error, log the Steam Error first. A lot of the bugs we're encountering raise UnknownError which means nothing to us. Please grab the newest version. ty.

[Amarinth]

Newest version looking good so far. No issues with the combined user/password page, and the show password function seems to work.

Lacking a time machine, I can't really test what happens when the login naturally expires, so I'm currently playing around and seeing what happens when I revoke it manually at random intervals. So far it seems to cause no issues on the user end. It does immediately lead to an authorisation error being logged, but the plugin seems to stay functionally connected while Galaxy is still running. And when I restart Galaxy the plugin automatically comes up with a prompt to sign back in.

I'll need to revoke access again and leave it for a bit longer, but apparently Galaxy just pushed a small update. So I'll test that first.

[engiefox]

Ok, plugin crashed after logging in with the new update.
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

Missed a few lines in my last bugfix. idk how. patched. you can replace the file if you don't want to pull/download. i've attached it here. rename to .py and replace the existing one. You also get a weird double recv call, which should never happen. Afaik there's only one spot in our code that calls recv and it's not re-entrant.
backend_steam_network.txt

I'm going to get some sleep. Should be already, but then i went for another shrine in Zelda. and then another. you get the point. So please don't have any errors for the next 8 hours lol (just kidding, if you hit one, post it ASAP. I'll check it when i'm awake).

[engiefox]

Not sure if there's a way to notify user when Steam Guard auth request has been missed, but it still shows "Offline" when a guard request is ignored. At the moment, it'll just show dialog to enter a Steam Guard code after clicking continue without approving a request, but entering a code still doesn't allow syncing.
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

found it. really dumb, really minor bug.

in %localappdata%\GOG.com\Galaxy\plugins\installed\steam_ca27391f-2675-49b1-92c0-896d43afa4f8\steam_network\protocol_client.py

find function async def check_auth_status(self, client_id:int, request_id:bytes, two_factor_method: TwoFactorMethod, auth_lost_handler:Callable) -> Tuple[UserActionRequired, Optional[int]]: (CTRL-F check_auth_status)

replace:

            if (two_factor_method == TwoFactorMethod.PhoneConfirm):
                return (UserActionRequired.TwoFactorRequired, data.new_client_id)

with

            if (two_factor_method == TwoFactorMethod.PhoneConfirm):
                return (UserActionRequired.NoActionConfirmLogin, data.new_client_id)

[ABaumher]

html errors are broken. NodeList doesn't support for...of, thought it did. so now it's a for loop. now they're fixed. in theory i could use NodeList.forEach, but idk if gog's browser is that modern. I'll push the changes here and in protocol client, but you can fix this yourself without too much hassle.
in %localappdata%\GOG.com\Galaxy\plugins\installed\steam_ca27391f-2675-49b1-92c0-896d43afa4f8\steam_network\custom_login\index.html
CTRL-F "un-hide" (it's a comment and it's right above what you're replacing. the method is too large to scroll through).
replace

			if (err)
			{
				for (errorField of err)
				{
					errorField.classList.remove("no-error");
				}
			}

with:

			if (err)
			{
				for (let k = 0; k < err.length; k++)
				{
					err[k].classList.remove("no-error");
				}
			}

EDIT: And replace let errored = urlParams.get["errored"] == 'true'; with let errored = urlParams.get("errored") == 'true'; you can CTRL-F urlParams.get, it'll be the second one.

Edit 2: The formatting on the error messages is terrible, but that's the case for most of the webpage. but it should be there

[engiefox]

Now it asks repeatedly for a Guard code, but won't accept any code given if the Guard request if ignored.
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

Yeah, i was afraid of that. The changes i've made have been a little hacky, but i hoped they would work. they haven't, so I'm gonna have to rework that whole logic flow. Excuse me while i swear like a sailor and then get to it.

[ABaumher]

I've made a bunch of changes under the hood. I probably missed something, so it will probably error. Checkout this branch or download the zip and redo the setup process. In the meantime i'm going to search the code for any references i didn't update.
https://github.com/ABaumher/galaxy-integration-steam/tree/mobile_confirm_refactor

[Amarinth]

Ok, I've found something that could be a problem.

Deauthorising all logins through Steam causes the plugin to silently disconnect. The plugin continues to claim it's "connected" in the Galaxy interface and running "sync integrations" looks like it succeeds, but doesn't actually import any new games or game stats. In this state the user has no way of knowing the plugin isn't working because it doesn't have a working connection to Steam until they restart Galaxy.

alpha1.0.4_deauth.log

Sorry for the length of the log, I only wiped it this morning, so it's been logging the entire day.

[ABaumher]

I may have a way to get it to propegate that error back to the main thread so we can cause gog to say "offline, retry." Unfortunately, it was part of the public profile code that i removed and now i have to go searching through the history to get it back.

[ABaumher]

I'm putting off this bugfix to version 2.0 when i refactor all the code. The way the code is written, this is an unrecoverable error. It closes down the socket we communicate with steam on, kills all the messages being sent, and then shuts down the plugin. In theory, we can send GOG an AuthenticationRequired error, which will tell it to redo all the authenticate stuff. But since we closed everything down, gog has no way to do all the authentication stuff. Closing down everything is good, because the client logged off and we won't get anything anyway, but i'd need to restart everything and that requires all the refactoring i don't want to do yet, lol

[ABaumher]

in the meantime, maybe it works? i caught that EResult we get when you manually disconnect the client, so GOG should now throw the proper error. It was throwing "unknown", which doesn't mean a thing to the plugin so it just silently went about its business. It should at least display something in the synced integration settings now. i have no idea what, but it'll display something.

[Amarinth]

LoggedInElsewhere surprised me as well. I figured since you borrowed some code from SteamKit, maybe it's because they're assuming the reason the connection would terminate like this is that the user is switching between devices where they can only be signed into one at the same time?

Anyway I've loaded up the new branch (still installing from zip for now, since I'm new to Github and just flying by the seat of my pants), and the bad news is whatever you've changed isn't clueing Galaxy in that the plugin has disconnected as you hoped. It's still behaving as before in the client - connection status doesn't change when I revoke the connection, nor when I force Galaxy to run a sync. I don't think there's anything new in the log either: plugin-steam.log

I had a quick look at GalaxyClient.log as well, and thought I'd spotted a clue when I saw it throw up Illegal 'steam' PluginState transition: connected -> connected after syncing, but apparently it does that for every other plugin as well. <_<
To my amateurish eyes it looks like Galaxy is still oblivious to the connection loss, and is simply assuming that no response = nothing new to report.

Anyway that's definitely a problem better hammered out once everything else works. And I am sorry for putting you through all of this!

I'll get back to seeing if I can break the plugin with bad login details. Previously fudging the username/password or SteamGuard code was working as intended (i.e. it would say they're incorrect and ask me to reenter them).
Edit: And I'll see what happens if I just AFK between user/pass and 2FA. I'm pretty good at walking away mid-login and completely forgetting about it for hours, so that's not such a weird test scenario.

[ABaumher]

It turns out, the old code does have a way to recover. I found that due to your TryAnotherCM bug. So, i properly handled the Authentication Required in Websocket Client. It now restarts the authentication process there. But i don't believe i restart it in the backend network. I'll need to pass the error back there and i don't know how (yet). This code is some nonsense lol. If i can find that, gog should hopefully take the reins and ask for the login page again.

[ABaumher]

Okay, the mobile confirm full refactor should now be up. it'll redirect you to the login page for your fallback of choice. this should always be mobile code, but whatever. or, it's horribly broken, idk. let me know

https://github.com/ABaumher/galaxy-integration-steam/tree/mobile_confirm_refactor

[ABaumher]

It's broken. give me a minute. I am gradually losing my sanity

[ABaumher]

my python-fu is terrible. tried to one-line a list of tuples to a dict (which is supposed to be easy in python) and it's a set instead. my brain hurts.

[ABaumher]

i might've been fixing something that already worked because i misread the error message. lmao that's terrible. i'll have a version up that passes email code in a moment. then you all can test the mobile confirm and mobile code

[ABaumher]

patched.

[engiefox]

Another crash. This time it asked for a Guard code, though it shouldn't need it since I approved the sign-in request on mobile. It accepted and showed as connected, but then showed "Not Responding" before crashing. Restarted GOG and it synced, weirdly.

plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

God, I hope mobile confirm works now. if it works when you confirm, and when you choose to ignore it and do the code login instead, that would be the last thing we need to get it to work. I guess there's also the case where you explicitely deny a mobile confirm, we could maybe catch that and say like "disconnected. Retry." I think that might be our "Fail" result (which we don't handle, it'd likely error), but i have no idea.

[engiefox]

GOG itself locks up(though it does that for pretty much all my integrations), but it recovered and it just gave a "Plugin has crashed" status for Steam. CPU goes to 100% when it's trying to sync Steam.

[engiefox]

Now no "Continue" page for Steam Guard shows at all. Approving the request does nothing and it just goes to "Offline"

plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

Another one-line fix.
in %localappdata%\GOG.com\Galaxy\plugins\installed\steam_ca27391f-2675-49b1-92c0-896d43afa4f8\steam_network\enums.py
in function def GetStartUri(self, errored : bool = False, **kwargs:str) -> str:
replace for key, value in kwargs: with for key, value in kwargs.items():

Python!

You're slowly running out of things that can go wrong, lol. At the point you errored, it was preparing the url for the internal browser so the fallback method you allow (mobile code, in all likelihood) can be selected if you choose to do so. I've tested the index.html for mobile confirm, which i can do because everything is encoded in the url parameter. if the url parameters are set correctly in the python code, it will switch to your fallback method if you hit that link. But knowing the python code i wrote, you might find another bug.

There is one potential snag i can think of: because you can select a fallback method in mobile confirm, i'm not sure if the mobile confirm page will ever close if you choose to use a mobile code instead. It should, but the regex we give gog to tell them when the page should close and they give us the results is untested. I'm significantly more experienced with regex than python, so it shouldn't error, but everyone has their own specific format for regular expressions and it's annoying af. i think it uses Posix format over Perl, but i have no friggin clue. If that happens, i'll see it in the log

[ABaumher]

an aside, i really hope i removed my debug msg sent as a url param. It's a good Archer reference, but it's also an Archer reference, so it's not exactly PG.

[ABaumher]

Ok, the last thing i can think of that needs testing is waiting for the mobile code, mobile confirm, and email codes to timeout. I also have no idea if the "resend it" button does anything. That, and the mobile confirm actually working, finally. holy toledo, that has been annoying

[Amarinth]

Looks like nothing too bad happens if you only enter username & password, and wait too long before entering the mobile code.

When doing so the login window still accepts the SteamGuard code as normal and closes, but then the status of the plugin changes from "Connecting" to "Failed, retry". Pressing retry brings up the login screen again from the start, which works fine if you then log in correctly.

Looking at the log, it seems the 2FA request timed out about a minute after I failed to enter something, which is followed by Error Received: TryAnotherCM and it kept repeating this every minute, until I entered a valid mobile code, which was then followed by some InvalidAuthData messages. And shortly thereafter you can see the plugin closing and restarting when I hit "retry" and logged back in properly.

So I think that's mostly working as intended?

[ABaumher]

That's a bug. TryAnotherCM randomly pops up in some cases. I have no idea why. It apparently was a thing in the old code as well. The only way to recover from it is to restart the login process. fortunately, the old code tried to do this, but it's missing a few steps. Our new auth flow takes like 6 steps to get there, the old code was just 1. so unless we cache all that info, we'd have to request it again. we also need to notify GOG that we lost the CM, to start over. I can actually do that by updating an enum but we'll need code changes in a bunch of spots

[ABaumher]

to top it off, whenever we get TryAnotherCM (enginefox found that in another set of errors), we also do a double recv call and i don't know how that's possible. I guess my timeout of 10 seconds is too high? but i don't want to thrash the system with the old 0.1 second value. I'm going to see if i can cancel a recv call

[engiefox]

Crashing again. This time the Steam Guard dialogs worked properly, tho so that's good. Swear I reported it last night, but I don't see it here.
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[engiefox]

Forcing using a Guard code after a failed Guard request might work, instead.

[ABaumher]

Ok, denying the request and trying to use a code instead doesn't work

If the plugin immediately says essentially login failed, retry, that's an expected behavior. If you deny the request, we have to respect that and the login will fail. We probably should mention that on the webpage so it's more clear.

If i may, the text for using a Guard code instead should be a different color

I think the other links in the UI have some css applied to them, if i can apply it here, it'll change color to match. You're right, i've just been trying to get it to work but that's probably an easy fix.

Hate to pile on, but is there a way to resend a Guard request? Clicking continue without accepting gives the proper feedback, but then clicking continue again just leads to the plugin going "Offline".

There should be a way to resend a mobile and email code, but it's from the old code and i have not tested it. i have no idea if we can request a mobile confirm refresh. If there's a way to redo a mobile confirm request, we could recover from that first issue and basically say "you denied the mobile confirmation. As a result, the login failed. Request a new mobile confirmation?"

[ABaumher]

You should be able to repeatedly click continue without accepting mobile confirm and it'll simply loop that page. I'll need to check your log to see what happened there

[engiefox]

yeah, clicking continue, approving the request and clicking continue again just leads to "Offline Retry". Then the login dialog reappears asking for username/password again.
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

Found that mobile confirm bug after one call. It's a few lines of code to fix. I'll roll it into a larger patch.

[ABaumher]

I've had this happen myself, but for me it just froze and i had to kill it with task manager. I'll check the log but i suspect it's because the legacy code is shit and it's being told to close but too much is going on and we don't have a way to tell it to stop. So it's still trying to sync like 1000 games but the client is trying to either shutdown or disconnect it.

…

On Fri, May 19, 2023, 10:41 PM engiefox ***@***.***> wrote: Ok, got it to happen. It crashes on trying to disconnect right after it finishes syncing. plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log <https://github.com/FriendsOfGalaxy/galaxy-integration-steam/files/11521412/plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log> — Reply to this email directly, view it on GitHub <#169 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASX2DPA62XBWY7SJRX3BASLXHAVM5ANCNFSM6AAAAAAYCP7KUY> . You are receiving this because you were mentioned.Message ID: <FriendsOfGalaxy/galaxy-integration-steam/repo-discussions/169/comments/5953515 @github.com>

[ABaumher]

Ty. Quick question - is mobile confirm working now? Judging by how far you're getting i assume yes but i never actually got confirmation lol. Maybe i missed it

…

On Fri, May 19, 2023, 10:44 PM engiefox ***@***.***> wrote: not sure if this'll have it GalaxyClient.log <https://github.com/FriendsOfGalaxy/galaxy-integration-steam/files/11521418/GalaxyClient.log> — Reply to this email directly, view it on GitHub <#169 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASX2DPG5YNFUZ4PHZSFCSRTXHAVYPANCNFSM6AAAAAAYCP7KUY> . You are receiving this because you were mentioned.Message ID: <FriendsOfGalaxy/galaxy-integration-steam/repo-discussions/169/comments/5953527 @github.com>

[engiefox]

Yeah, both methods are working

[ABaumher]

Woooooo!

And hitting continue multiple times without confirming in the app correctly displays the confirm page instead of the mobile code?

[engiefox]

Had an instance where game import stopped after just a few games.
edit: Finished on restarting GOG.

plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

Same as before, gog hits an end of file (EOF) and closes the connection between us and gog. I have no idea why. We get a lot of redundant information in these calls, maybe there's a way to not send as much info. It's possible that when steam changed the auth flow they started sending other messages less/ more often and were fighting that, idk.

…

On Sat, May 20, 2023, 12:36 AM engiefox ***@***.***> wrote: Had an instance where game import stopped after just a few games. plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log <https://github.com/FriendsOfGalaxy/galaxy-integration-steam/files/11521557/plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log> — Reply to this email directly, view it on GitHub <#169 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASX2DPDPSVGO2Z3XFMOWGADXHBC3TANCNFSM6AAAAAAYCP7KUY> . You are receiving this because you were mentioned.Message ID: <FriendsOfGalaxy/galaxy-integration-steam/repo-discussions/169/comments/5953780 @github.com>

[woj1993]

@ABaumher Hi. Are we having support for steam archivments? Because I unlocked one and it didn't get synchronized to Galaxy even through galaxy was running while I got it.

I got archivment 19-05-2023 in 09:34 PM
Steam didn't synch archivments_2.zip

[Amarinth]

Hmm, you're right, it's updated my achievements now as well.

Looks like it may only import them during a sync and since I have a lot of games that seems to take a while to complete, so it just wasn't finished importing when I checked earlier.

[woj1993]

It dos'nt work for me, but I have many games. Meybe it times out?

[ABaumher]

Sorry i'm late, i needed a break lol. I am looking into the achievement issue, but i don't really have enough information. However, i'd like to put the finishing touch on the 2FA stuff so i can handle timeout and request new code and move to beta.
That log may help, i'll take a look. Also, it seems to be sporadic - it works sometimes, and others it doesn't. There's a possible location for me to pull achievements - GOG periodically calls a function called "tick" which i could co-opt and use. Speaking of, @Amarinth i update that very same function so it now checks to see if the authorization was revoked. you may not need to "sync integrations" to get connection lost anymore. I don't know how often the tick function is called though so i have no idea how long you'd need to wait. Not really something that needs to be tested (current system works well enough), but if you want to, you can.

EDIT: I haven't pushed the patch yet. I'm going to attempt the "new code, retry" and then push both in the patch.

[Amarinth]

Whatever you're doing to call the tick function is working btw. If I revoke the connection Galaxy now brings up the login prompt automatically whenever it would next check for game stats, and that happens regularly enough that it should notice within an hour.

Hopefully that'll come in handy as well when the refresh token expires naturally.

And I guess if there's anything else you need to run periodically, you now know you can trigger it off the tick function.

[woj1993]

@ABaumher Are you still checking this?

[ABaumher]

Ok, the final patch (excluding bugfixes, which are probably required) is up at https://github.com/ABaumher/galaxy-integration-steam/tree/mobile_confirm_refactor . If there aren't any bugs and everything works, i'll merge it into master, zip this shit and post it on the main thread as an open beta.

I've looked at the code for achievements. get_unlocked_achievements is a function we need to implement so achievements are obtained. GOG makes a call on that function in order to get all the achievements you have. Unfortunately, I can't control when they call it, so they may not call it all that often. This might be a GOG issue, not sure.

EDIT: before we can call that, we need to prepare them. This is in the function prepare_achievements_context. I've seen in the logs when we're preparing this because i've seen the message "Game stats import already in progress". We dump your achievements to a cache so we don't need to load them all the time. If you have a lot of games (and these games have a lot of achievements), this proces can take several minutes.

Edit 2: It's bugged. Let me fix it, give me a minute

[woj1993]

Ironically in logs I see that it is trying to import archivments but it seems to think I didn't get any?

[ABaumher]

2023-05-21 18:00:23,664 - backend_steam_network - INFO - Asked for achievs for 1097150
2023-05-21 18:00:23,665 - galaxy.api.jsonrpc - INFO - Sending notification: method=game_achievements_import_success, params={'game_id': '1097150', 'unlocked_achievements': []}

(game id obtained via the url https://store.steampowered.com/app/1097150/Fall_Guys/)

They don't get any achievements for fall guys

[ABaumher]

Might not be written to the cache properly, let me check that

[don-de-marco]

I think I found how, when, and why the "file not found" error occurs during mobile confirm: if I click deny on my phone, the log says Received a file not found but were not using mobile confirm. This is unexpected, but maybe ok? no idea. after I click continue on the confirm page in the Galaxy wizard. Maybe the file not found error indicates that a user has denied the login attempt. See the attached log excerpt.

[ABaumher]

If you deny on mobile confirm, it should flat-out fail, as that's what you told us to do. If you are on master, not the mobile confirm refactor branch (see https://github.com/ABaumher/galaxy-integration-steam/tree/mobile_confirm_refactor), then the message in the UI has not been updated to mention this. The only reason you should see that in the log in the latest version is if you are using mobile code or email code. For whatever reason, steam sends a FileNotFound when you do email or mobile code and it has expired. I don't know why it doesn't use Expired like it should, but i can't make Steam send the logical message

An aside, if i can get this last issue with timeouts fixed, i'm merging that branch back into master and then moving to beta. i originally broke it out because i needed to get the multiple login methods and fallbacks working, and it was different enough from your code that i had previously merged in that i knew i'd break a bunch of shit. I didn't want to break the stuff people were testing that i hoped worked well.

[ABaumher]

Found it. I'll have a patched patch up shortly

[ABaumher]

Well, steam no longer resends the code if you repeat the login attempts. I have no idea lol

[woj1993]

?

[ABaumher]

steam gives you this link instead of offering you the option to resend a code. The only option i really have is to do the same. https://help.steampowered.com/en/wizard/HelpWithSteamGuardCode Plus side, the code is simpler. I'll just implement the timeout as kicking you back to login. We really should fix the error code there before going to beta but i really don't want to lol

[ABaumher]

Ok, patched final patch (hopefully) up. There's a known issue: When the login 2FA times out, it kicks you back to the login page, but there error message is incorrect. It says you failed to login the username/password correctly, it should mention that your 2FA expired and you need to redo the login. Steam changed how they do all the login stuff, including removing the resend code option and replacing it with a help wizard in your browser. So, we do the same. It's significantly easier, and as a result literally everything i spent the last few hours on has gone down the trash. (╯°□°）╯︵ ┻━┻

As always, if i can get testers to try these changes. That'd be great. I'm going to try it here with email code, but if @Amarinth could try for mobile code that'd be peachy. same for @engiefox or @pippo-san with mobile confirm. I'm the least confident that will work there, but it might.

[engiefox]

Seems to be working, timeout kicks me back to login, but yeah wrong message is displayed.
edit: Entering username/password again doesn't bring up a new auth request, just disappears and shows "Offline (user) Retry"
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

I updated the message there in a new patch. It'll say something like "steam guard expired". it's a really hacky html fix but might work if everything in the python from the previous patch works. i'll debug the html in a bit. I'll check the login, give me a minute. Checking your log now.

[Amarinth]

Looks to be working with the mobile app code. It doesn't automatically update when the authentication request expires, but it kicks me back to the user & password field with the error when I enter a code after the request has expired. If I then reenter everything as expected it works.

[ABaumher]

Waiting to test email now, but if it works in mobile, it really should work here. again, same code, different enum value (one is 2, other is 3).

I really, really hate mobile confirm. I can't see any error in that log. But i might have figured out why gog shuts down our connections. GOG pings us on their connection, and i guess if we don't respond, it just closes the plugin? idk how we'd respond, i assume it's in some legacy code. I have no idea if it's broken, either. More searching i guess

[ABaumher]

Works in email. Doesn't send a new code, oddly enough, but the old code still works. I've added to the message we display with email login so the users know to try this old code first.

[ABaumher]

@engiefox That mobile confirm fallback issue - does it immediately fail or does it take a minute?

[engiefox]

plugin crashed, here are all the log files.
GOG_logs.zip

[ABaumher]

your log shows you logged in properly. it also shows you switched to mobile code as your fallback. Succeeded at roughly 16:39, i think your time.

[engiefox]

I've been trying multiple times to replicate the problem, but it's been difficult to get it to happen again.

[ABaumher]

That all but confirms it then. This is an issue when we lose a connection while waiting for a request. At the moment we don't really have a way to recover from it. I'm going to tentatively call it a known issue and put it off to 2.0 when all the code is cleaned up. If a lot of people in the open beta have an issue with it, i'll obviously address it ASAP. It's possible to potentially recover from if i know we lost a connection and are waiting on a message, to cancel that, catch all the errors that causes (it's a lot), then resend the message after we reconnect, but it's huge amount of bandaids on a bullet wound. The only solution is that 2.0 rewrite.

[engiefox]

here's another batch of logs if it helps. this time clicking Continue on the Steam Guard dialog multiple time caused a failure with "Offline Retry" status.
GOG_logs.zip

[engiefox]

Crash on the new beta release. Login appeared to go fine, but crashed while trying to sync.
gog_logs.zip

[engiefox]

Another EOF message, Seemed like it would sync some games, but then give up. I would expect it to take awhile to sync the ~750
games in my library, but it didn't seem to stop midway before.
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

That said, we might be crashing during the loads if we get overloaded, and your sync basically says "try again" so it adds more and more until it's done. I don't think this is an issue with our plugin, i think it's the fact that we have to send a lot of information and it's done over a socket (a port on your local host), and there's a fixed throughput there. we might be bottlenecking on that socket, lol. or we're bottlenecking on writing to a database (db) file, i'm not sure.

[ABaumher]

Unfortunately, the code that gets the games hasn't changed, so idk why it's having a fit. I really need to debug why we hit that EOF, but it's always hit GOG-side, not ours. and i can't seem to replicate it here.

[ABaumher]

in just pauses, 750 games should take 75 seconds. The actual pull from steam, judging by your logs, is surprisingly quick (fractions of a second for decent chunks of info), but idk. It also pulls all your friends, their statuses, etc. So even though it's quick, it does add up. For every game, it also needs to make an individual request to get achievements, then cache those results. i really wish there was a call that got all the achievements for all your owned games, but there's not.

[ABaumher]

i'm not trying to make excuses or anything, i legitimately don't know what is happening behind the scenes. that's why i wanted you to add that one logger thing in the jsonrpc file, to see if we hit an error there and that's why gog shut down.

When you hit sync integrations, it calls the tick function. Which forces the game library to update. That's why you get more games when you do that.

[engiefox]

Aside from the issues we discussed, the beta seems to be working pretty well so far. It's stable over long periods and it seems to handle expired Guard requests properly. One thing is it doesn't fallback to the login page if the request is denied. It stays on the Guard dialog and the only way to try again is to enter a Guard code, but that won't actually log you in it just kicks you back to the login page.

[pippo-san]

as a side note, is there a reason the link color was changed back to blue?

It might be because you've already visited the link, but I don't remember if you can disable that or not.

edit: sure you can, just use css :visited to set the same color as if a page hasn't been ever visited

[engiefox]

Does it save any sort of history like that? No other links in GOG/other plugins have that behavior.

[pippo-san]

Does it save any sort of history like that? No other links in GOG/other plugins have that behavior.

it's just a metter of css style, I've updated the post

[engiefox]

I looked at the file history and compared with what's in the zip release. The lines changing the color for that link are gone in the zip release.

[ABaumher]

i accidentally reverted too far when i undid an experiment i was working on to try to get the "resend code" to work. I have since patched it but it's not part of the release. If i get time this weekend to redo the error messages we display and add informational messages that steam gives us like "we sent an email to <your email domain>" i'll make a new beta release and it'll have this patched. I could update the release now but i don't really feel the need to.

[Mr-Spock1]

Hi guys,
encouraged by the announcement of the open beta, I tried it myself.
Unfortunately, as soon as I pressed the "Connect" button, the plugin crashed.
I hope my log helps.
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

You got a memory error, and not even in our code, which is doubly weird. That typically occurs when there isn't enough RAM for whatever python is trying to do. Were you running a lot of other things at the same time? I honestly can't think of a reason the code would produce that. Are you on 32bit or 64bit os?

[ABaumher]

You can also try deleting the database cache file, in case it was buggered from the previous stuff and ate up all your ram. It's at %systemdrive%\ProgramData\GOG.com\Galaxy\storage\installed\steam<gibberish>.db

[Mr-Spock1]

Ok, the available RAM can't be an issue; my iMac (intel, latest OS) has 32 GB and there were always more than 10 GB free. So I deleted GOG Galaxy and reinstalled it: Same result. Then I used a tool to uninstall Galaxy with all related files and folders to reinstall it again with the latest installer from the website: Same result, but somehow many of my settings were still intact. To make it as clean as possible without formatting my SSD, I uninstalled it again and searched every corner of my FS for bytes that might belong to Galaxy, which took me a few hours. This time GOG Galaxy finally forgot everything and I tried again - alas, the same result: I click on "Connect" and the plugin crashes. I've attached the log in case anything has changed. Btw, Epic integration works perfectly and I own more games on Epic (~180) than on Steam (~110). But as prompt as the crash happens, the amount of data can't matter yet, there must be something else hanging. Any idea what to try next?
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

This is why i wanted a Mac tester lol. We haven't tested anything on a Mac yet. Unfortunately that makes you my test dummy. Additionally, 32 bit programs have a hard cap for ram, even if your system has more. Normally i'd assume that's not an issue because Macs have been 64 bit for forever but i never know with python what it's doing behind the scenes. That said, i might be overthinking it - maybe i just need to compile a Mac version. I assumed the python was platform agnostic because it's a runtime language but maybe not. What i would suggest trying (i know it's not ideal) is building the plugin yourself on Mac and seeing if that makes a difference.If it does, i can have you run a single command which will zip it properly, then add it to the releases as the Mac version. There are instructions on my repo but I've never tested it on Mac so I'd probably have to help you troubleshoot. It's only about 3 bash commands, but you'd need python 3.7.9.

[Amarinth]

Just in case you haven't seen this, looks like someone else on Mac has done some of the legwork for you already: ABaumher#7

[Mr-Spock1]

Had to get a few hours of sleep (GMT+2, but live more like GMT-5 lol).
I'd be glad to assist you if I can. I've always wanted to learn about that stuff anyway, but I haven't had the time or the courage yet.
Btw my python version is 3.9.6.

[bgebhardt]

HI. I'd be willing to be a mac tester, can make builds, and maybe even fix a bug or two.

@Mr-Spock1 how big is your steam library? Could that be an issue? Maybe try creating a new empty steam account and connect that to see if you get the same error.

I also posted my build of the plugin in this issue. Download and try it to see if it works.
ABaumher#7 (comment)

[Mr-Spock1]

@bgebhardt There are about 110 titles in my steam library.
Next try. This time with steam_v1.0.4_mac_bgebhardt.zip:
No immediate crash. Instead a lot of popups (>10 on my first try). Each time I try there came fewer but those last ones won't go away. When I now click "Connect" I get a MacOS warning:

macOS cannot verify the developer of "_inlines.cpython-37m-darwin.so". Do you really want to open the app?

with the buttons "Öffnen" (Open), "In den Papierkorb legen" (Put in trash) and "Abbrechen" (Cancel)
I select "Öffnen" and the second popup appears which reads:

"_AppKit.cpython-37m-darwin.so" cannot be opened because the developer cannot be verified.

with only the buttons "In den Papierkorb legen" and "Abbrechen".
I select "Abbrechen" and swich to "Systempreferences" where under "Security" the option

The opening of "_AppKit.cpython-37m-darwin.so" has been blocked because the app does not come from a verified developer.

with the button "Trotzdem erlauben" (Allow anyway) which I do.
The next popup… wait … no way!! … this time it worked!!!

Forget everything I wrote. I tried a last time to report the exact reading and this time I'm in.
Restarted Galaxy… Still connected!

THANK YOU EVERYONE WHO CONTRIBUTED TO MAKE THIS WORKING.

If there is still something you want me to test. Let me know.

[ABaumher]

Kein Problem! Bitte Schön!

My German is very rusty ("Ich spreche nicht gut Deutsch", as my "lehrner" once joked) but i did recognize it almost immediately, lol. It might be worth looking into signing our application so that does not appear, but i'm not sure how to do so

[ABaumher]

I did not. Ty for the update

…

On Tue, May 23, 2023, 9:35 AM Amarinth ***@***.***> wrote: Just in case you haven't seen this, looks like someone else on Mac has done some of the legwork for you already: ABaumher#7 <ABaumher#7> — Reply to this email directly, view it on GitHub <#169 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASX2DPDQZFMJCVJL66O3MKTXHS4J3ANCNFSM6AAAAAAYCP7KUY> . You are receiving this because you were mentioned.Message ID: <FriendsOfGalaxy/galaxy-integration-steam/repo-discussions/169/comments/5978476 @github.com>

[engiefox]

still getting the occasional crash on trying to sync for the first time
GalaxyClient.log
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[ABaumher]

I may have just found out why, let me check your log for confirmation. Does it soft crash ("offline, retry" after a minute or something similar) or is it a hard, immediate one with the angry red text?

[ABaumher]

no, it's the opposite here. weird. nvm

[engiefox]

I'm getting crashes pretty regularly now. The plugin will run for awhile, but eventually it'll crash with the angry red text. Logs just cut off at the end.
plugin-steam-ca27391f-2675-49b1-92c0-896d43afa4f8.log

[engiefox]

would it help to start a new issue under the beta repo?