Tracing sent post request body and comparing it with one made by requests lib

[SanctusAnimus]

So, i'm currently working on own auth module for Steam, it requires me to send a post request with form-data in it.
Issue is, form-data sent with aiohttp is not accepted without any explanation. Here's the few code parts of what's going on:

Dictionary with original data, everything here is either string or bytes for password.
Worth mentioning that requests lib is able to process integers as they are and converted to a string equally, while aiohttp forces me to convert those to string.

data = {
            'username': username,
            "password": b64encode(pkcs1v15_encrypt(self.key, password.encode('ascii'))),
            "emailauth": email_code,
            "emailsteamid": '76561198160729942' if email_code else ' ',
            "twofactorcode": twofactor_code,
            "captchagid": str(self.captcha_gid),
            "captcha_text": captcha,
            "loginfriendlyname": "async-steam webauth",
            "rsatimestamp": self.timestamp,
            "remember_login": 'true',
            "donotcache": str(int(time() * 100000))
        }

Here's the pretty-printed version of what's inside it (with some data being cut, since it's login credentials lol):

{'captcha_text': '',
 'captchagid': '-1',
 'donotcache': '160641367300963',
 'emailauth': 'DGP4X',
 'emailsteamid': '76562298160729942',
 'loginfriendlyname': 'async-steam webauth',
 'password': b'Twe/NBCK7GSHqob4qUuWZdsVrgt6Lh746GJWa17gwsnAFO8CJ3UP7nT80jpOkJiT'
             b'zkfJiLKg/HZcpnfx0InIXg==',
 'remember_login': 'true',
 'rsatimestamp': '151278700000',
 'twofactorcode': '',
 'username': 'NNN'}

I'm sending the data with following call, immediately after defining that data:

resp = await self.session.post(f'https://steamcommunity.com/login/dologin/', data=data, timeout=15)

Session object here is a ClientSession, which contains a header for User-Agent and a few cookies from getting an RSA key from Steam. Changing session here to a new one doesn't change anything.
I checked content-type of sent request and it does encode it as multipart/form-data; boundary... correctly.

But the response i'm getting from Steam contains only {'success': False}

I've tried to do the same via requests lib, and got a successful response, here's how it was sent

resp = requests.post('https://steamcommunity.com/login/dologin/', data=data, timeout=15)

I've replace aiohttp call with requests call directly, data is exactly the same, and it does trigger two-factor / email login confirmation etc.

So here's my question - can i somehow compare the encoded bodies of sent data and understand what's going wrong?
I've tried adding requests tracing but it doesn't yeild much of a useful info.
I feel like i can't really submit this as a bug into Issues since replication is a bit long and the issue itself is unconfirmed, so maybe i'm missing something on aiohttp side?

The purpose of the module is to utilize asyncio, so using synchronous requests is not an option (at least until i give up on fixing it).

[SanctusAnimus]

UPDATE:

So, i tried requesting my local server with same data, and turns out that password field in getting sent as a file stream in aiohttp, while requests is sending them as bytes (as they should be). Is there a way to omit this behavior? I've tried adding content-type of application/octet-stream which were suggested on Mulitpart wiki page, but that haven't change anything

[webknjaz]

what do you mean by a file stream? Looks like you called b64encode() but forgot to convert it to str so the underlying code is probably attempting to apply str() to it and that results in repr leaking into the string representation. Basically str(b'') will emit something like 'b""' breaking your expectations. I don't think it's a good idea for the framework to invoke .decode() because the encoding used to construct some arbitrary byte string could be anything random, it shouldn't assume something like ascii or utf8 just because it may be true for some cases.

[SanctusAnimus]

Well, i'm kinda new to this, and by file stream i mean, that local web server i used (FastAPI in this case) recognizes password sent as bytes correctly (displaying it as b'...'), but for aiohttp i'm getting automatic conversion into UploadFile object, which means that aiohttp (as i've seen from lib code) assumed that bytes passed were a file. Here's the snippet from formdata.py

        elif isinstance(value, (bytes, bytearray, memoryview)):
            if filename is None and content_transfer_encoding is None:
                filename = name

And if you saying that framework should not assume encoding, but it is assuming file being passed? It's not like i'm against it or anything, just a bit unintuitive for me.

I've tried encoding password as ascii or utf8 and Steam at least replies something, but says that resulting password is incorrect. It's kinda hard to work with since Steam doesn't provide any specs for this API, so i'm not sure what exactly is required on the other end.

On the other hand, i haven't tried assigning content_transfer_encoding to binary (or should it be base64?), so that's what i'll do next probably

[webknjaz]

You may try capturing the network traffic and exploring what is transmitted on the HTTP level.
Also, you probably need to set Content-Type to something like application/json since the payload you've shown looks like JSON. OTOH I don't know how Steam operates so I won't be able to help you there.

P.S. Have you considered exploring prior art? Quick googling returns https://github.com/patryk4815/PySteamWeb and https://github.com/Zwork101/steam-trade/blob/master/pytrade/login.py. You could look at their auth mechanisms and reimplement for yourself if you really need to do this: if there's something that already solves the problem, there's almost no reasons not to use those.

[SanctusAnimus]

Fair enough, i'll look into those repositories, marking as solved, thanks for the answers