urequests different behaviour to requests

[weasdown]

I am using MicroPython version v1.19.1 on a Raspberry Pi Pico W. I am writing some code that uses the urequests library to query the Gigya API used to interact with Renault cars. The first step in talking to the API is to log in to Gigya using my email address, password and API key. I have written a function, gigya_login, to do this:

def gigya_login():
    """login to Gigya"""
    gigya_login_url = f"{gigyaRootUrl}/accounts.login"
    post_data = ujson.dumps({
        'ApiKey': 'secret API key',
        'loginID': 'secret email address',
        'password': 'secret password'
        })
    print(f'POST data type: {type(post_data)}')
    print(f'POST data: {post_data}')
    print("")
    login_response = urequests.post(gigya_login_url, data=post_data)
    login_json = login_response.json()
    login_response.close()
    print(login_json)
    return login_json['sessionInfo']['cookieValue']  # returns login_token

This is not working, as the API is not recognising the API key:

{ "callId": "171c67427d5b48b78041bfe16dad2005", "errorCode": 400093, "errorDetails": "Missing required parameter: ApiKey", "errorMessage": "Invalid ApiKey parameter", "apiVersion": 2, "statusCode": 400, "statusReason": "Bad Request", "time": "2022-09-14T06:53:28.542Z" }

However, I have previously written some Python code to do the same thing and that works fine:

def gigya_login():
    """login to Gigya"""
    gigya_login_url = f"{gigya_root_url}/accounts.login"
    data = {
        "ApiKey": gigya_apikey,
        "loginID": login_id,
        "password": login_password,
    }
    print(f'Gigya login data: {data}')
    login_response = requests.post(gigya_login_url, data=data)
    login_json = login_response.json()
    return login_json['sessionInfo']['cookieValue']  # returns login_token

It successfully returns a response with a 200 statusCode, a statusReason of OK, and lots of information about my account (which I have omitted for brevity and security):
{'callId': '61af4a61ab0b4b6e94b48a237a0bad72', 'errorCode': 0, 'apiVersion': 2, 'statusCode': 200, 'statusReason': 'OK', 'time': '2022-09-14T06:55:38.070Z', ... etc}

I have verified that the same data dictionary is being given to each .post command, so I'm really at a loss as to why requests is working but urequests isn't.

[weasdown]

It seems from this forum post that the issue lies in urequests not handling URL encoding in the same way as requests. I'll try implementing their fix later today.

[weasdown]

That forum post did indeed solve the problem. For this particular case, I now have the login working using the following code:

def gigya_login():
    """login to Gigya"""
    gigya_login_url = f"{gigyaRootUrl}/accounts.login"
    
    post_data = {
        'ApiKey': 'secret API key',
        'loginID': 'secret email address',
        'password': 'secret password'
        }
    
    def url_escape(s): # consider making POST and GET functions that include this and are called by the Gigya/Kamereon functions
        return ''.join(c if c.isalpha() or c.isdigit() else '%%%02x' % ord(c) for c in s)
        
    encodedKey = url_escape(post_data['ApiKey'])
    encodedEmail = url_escape(post_data['loginID'])
    encodedPassword = url_escape(post_data['password'])
    
    encodedData=f'ApiKey={encodedKey}&loginID={encodedEmail}&password={encodedPassword}'  
    login_response = urequests.post(gigya_login_url, data=encodedData, headers={'Content-Type': 'application/x-www-form-urlencoded'})

    print('')
    print(login_response.text)
    login_json = login_response.json()
    login_response.close()
    return login_json['sessionInfo']['cookieValue']  # returns login_token

[jimmo]

FWIW, this comes up often enough as a source of confusion that we may want to consider adding it to urequests.

Specifically that if the data argument is a dictionary, then it should be implicitly converted to URL encoding and the content-type header set. i.e. this exact use case from the requests docs https://requests.readthedocs.io/en/latest/user/quickstart/#more-complicated-post-requests

[weasdown]

From my point of view as a new user of urequests, changing urequests to be in line with requests in this sense would certainly be useful, or the documentation at least needs to explicitly mention this difference and how to handle it. jimmo, it was only by stumbling across your forum post linked above that I managed to solve this, so thank you very much for the post but I think more guidance for new users would be beneficial.