OAuth2 refresh token issue

[skchronicles]

Hello there,

• I have checked that the SDK documentation doesn't solve my issue.
• I have checked that the API documentation doesn't solve my issue.
• I have searched the Box Developer Forums and my issue isn't already reported (or if it has been reported, I have attached a link to it, for reference).
• I have searched Issues in this repo and my issue isn't already reported.

Description of the Issue

I am running into some issues related authentication using OAuth2. It appears my refresh token is not getting refreshed, like the store_tokens callback is doing nothing or the .refresh() method does not do anything.

Steps to Reproduce

Here is an example script, auth.py:

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-

# Python standard library
from __future__ import print_function
import configparser, os, sys 

# 3rd party imports
from boxsdk import Client, OAuth2


def err(*message, **kwargs):
    """Prints any provided args to standard error.
    kwargs can be provided to modify print functions 
    behavior.
    @param message <any>:
        Values printed to standard error
    @params kwargs <print()>
        Key words to modify print function behavior
    """
    print(*message, file=sys.stderr, **kwargs)


def fatal(*message, **kwargs):
    """Prints any provided args to standard error
    and exits with an exit code of 1.
    @param message <any>:
        Values printed to standard error
    @params kwargs <print()>
        Key words to modify print function behavior
    """
    err(*message, **kwargs)
    sys.exit(1)


def parsed(config_file = None, required = [
        'client_id', 'client_secret', 
        'access_token', 'refresh_token'
        ]
    ):
    """Parses config file in TOML format. This file should contain
    keys for client_id, client_secret, access_token, refresh_token.
    The `auth` function below will update the values of access_token
    and refresh_token to keep the users token alive. This is needed 
    because the developer tokens have a short one hour life-span.
    @Example `config_file`:
    [secrets]
    client_id = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    client_secret = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    access_token = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    refresh_token = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    @param config_file <str>:
        Path to config file, default location: ~/.config/bx/bx.toml 
    @return [client_id, client_secret, access_token, refresh_token]:
        Returns a list of <str> with authenication information:
            [0] client_id
            [1] client_secret
            [2] access_token
            [3] refresh_token
    """
    # Information to parse from config file
    secrets, missing = [], []
    if not config_file:
        # Set to default location
        # ~/.config/bx/bx.toml
        home = os.path.expanduser("~")
        # TODO: add ENV variable to
        # override this default PATH
        config_file = os.path.join(home, ".config", "bx", "bx.toml")  

    # Read and parse in config file 
    config = configparser.ConfigParser()
    config.read(config_file)
    # Get authentication information,
    # Collect missing required info
    # to pass to user later
    for k in required:
        try:
            v = config['secrets'][k]
            secrets.append(v)
        except KeyError as e:
            missing.append(k)

    if missing:
        # User is missing required 
        # Authentication information
        fatal(
            'Fatal: bx config {0} is missing these required fields:\n\t{1}'.format(
                config_file,
                missing
            )
        )

    return secrets


def update(access_token, refresh_token, config_file=None):
    """Callback to update the authentication tokens. This function is 
    passed to the `boxsdk OAuth2` constructor to save new `access_token` 
    and `refresh_token`. The boxsdk will automatically refresh your tokens
    if they are less than 60 days old and they have not already been re-
    freshed. This callback ensures that when a token is refreshed, we can
    save it and use it later.
    """
    if not config_file:
        # Set to default location
        # ~/.config/bx/bx.toml
        home = os.path.expanduser("~")
        # TODO: add ENV variable to
        # override this default PATH
        config_file = os.path.join(home, ".config", "bx", "bx.toml")
    # Read and parse in config file 
    config = configparser.ConfigParser()
    config.read(config_file)
    # Save the new `access_token` 
    # and `refresh_token`
    config['secrets']['access_token'] = access_token
    config['secrets']['refresh_token'] = refresh_token
    with open(config_file, 'w') as ofh:
        # Method for writing is weird, but
        # this is what the docs say todo
        config.write(ofh)


def authenticate(client_id, client_secret, access_token, refresh_token):
    """Authenticates a user with their client id, client secret, and tokens.
    By default, authentication information is stored in "~/.config/bx/bx.toml".
    Here is an example of bx.toml file:
    [secrets]
    client_id = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    client_secret = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    access_token = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    refresh_token = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    NOTE: This operation needs to be performed prior to any Box API calls. A 
    Box developer token has a short life-span of only an hour. This function will
    automatically refresh a token, to extend its life, when called. A token that
    has an expiration date past 60 days CANNOT be refreshed. In this scenario, a 
    user will need to create a new token prior to running the tool. A new token 
    can be create by creating an new 0auth2 app here: http://developers.box.com/
    """
    auth = OAuth2(
        client_id=client_id,
        client_secret=client_secret,
        refresh_token=refresh_token,
        store_tokens = update
    )

    try:
        access_token, refresh_token = auth.refresh(None)
    except Exception as e:
        # User may not have refreshed 
        # their token in 60 or more days
        err(e)
        err("\nFatal: Authentication token has expired!")
        fatal(" - Create a new token at: https://developer.box.com/")
    
    return access_token, refresh_token



if __name__ == '__main__':
    try:
        # Use user provided config
        test_file = sys.argv[1]
    except IndexError:
        # Test auth config file parser
        home = os.path.expanduser("~")
        test_file = os.path.join(home, ".config", "bx-dev", "bx.toml")

    client_id, client_secret, access_token, refresh_token = parsed(
        config_file = test_file
    )

    # Test token refresh 
    new_access_token, new_refresh_token = authenticate(
         client_id = client_id, 
         client_secret = client_secret,
         access_token = access_token, 
         refresh_token = refresh_token
    )

    # Manually update tokens
    update(
        access_token = new_access_token, 
        refresh_token = new_refresh_token, 
        config_file = test_file
    )

    # Test authentication
    auth = OAuth2(
        client_id = client_id,
        client_secret = client_secret,
        access_token = new_access_token,
        refresh_token = new_refresh_token,
        store_tokens = update
    )

    # Get user info
    client = Client(auth)
    user = client.user().get()
    print("The current user ID is {0}".format(user.id))

Here is an example config file, config.toml:

[secrets]
client_id = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
client_secret = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
access_token = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
refresh_token = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Here is example usage of the script:

python3 auth.py config.toml

Expected Behavior

I was hoping the token would get refreshed and saved in the config file. I am not sure of the best way to get a refresh token. From http://developers.box.com/, I only see a Developer token option. I have been using an access_token and refresh_token that I generated for connecting Box to Rclone. With that being said, I got this script to work a few times by running Rclone and then copying the new refresh_token and access_token Rclone updated in its config file. That worked a few times, and then it randomly stopped working. The tokens provided to the python-sdk are the same tokens that Rclone is using. Rclone works fine with those tokens.

Is there a way to refresh a developer token in a headless manner (working on a remote server)? Similar to how this OAuth2 refresh is supposed to behave? I just need a token to programmatically interact with Box, that I can refresh without a browser getting involved. It seems like Rclone has figured this out. If I checkout the config file it uses, it automatically gets updated when the token expires. Also, it is possible to create a token with a longer life span, say like a year or so. I will be using this token to programmatically interact with files I own on Box.

Error Message, Including Stack Trace

Here is the error:

$ python3 auth.py config.toml
"POST https://api.box.com/oauth2/token" 400 69
{'Date': 'Wed, 14 Dec 2022 20:38:56 GMT', 'Content-Type': 'application/json', 'Transfer-Encoding': 'chunked', 'Strict-Transport-Security': 'max-age=31536000', 'Set-Cookie': 'box_visitor_id=639a3460a13270.50706908; expires=Thu, 14-Dec-2023 20:38:56 GMT; Max-Age=31536000; path=/; domain=.box.com; secure, bv=OPS-45763; expires=Wed, 21-Dec-2022 20:38:56 GMT; Max-Age=604800; path=/; domain=.app.box.com; secure, cn=9; expires=Thu, 14-Dec-2023 20:38:56 GMT; Max-Age=31536000; path=/; domain=.app.box.com; secure, site_preference=desktop; path=/; domain=.box.com; secure', 'Cache-Control': 'no-store', 'Via': '1.1 google', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"'}
{'error': 'invalid_grant', 'error_description': 'Invalid refresh token'}


Message: Invalid refresh token
Status: 400
URL: https://api.box.com/oauth2/token
Method: POST
Headers: {'Date': 'Wed, 14 Dec 2022 20:38:56 GMT', 'Content-Type': 'application/json', 'Transfer-Encoding': 'chunked', 'Strict-Transport-Security': 'max-age=31536000', 'Set-Cookie': 'box_visitor_id=639a3460a13270.50706908; expires=Thu, 14-Dec-2023 20:38:56 GMT; Max-Age=31536000; path=/; domain=.box.com; secure, bv=OPS-45763; expires=Wed, 21-Dec-2022 20:38:56 GMT; Max-Age=604800; path=/; domain=.app.box.com; secure, cn=9; expires=Thu, 14-Dec-2023 20:38:56 GMT; Max-Age=31536000; path=/; domain=.app.box.com; secure, site_preference=desktop; path=/; domain=.box.com; secure', 'Cache-Control': 'no-store', 'Via': '1.1 google', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"'}

Versions Used

Python SDK: 3.5.1
Python: 3.8

[skchronicles]

@antusus @mhagmajer @lukaszsocha2 @mwwoda
Any thoughts on this issue?

[lukaszsocha2]

Hi @skchronicles,
sorry for the late response. I've tested the code snipped you provided and I was able to refresh token and save it to a configuration file without any error. So your code should work fine. The error you are receiving indicates that your refresh token is invalid. Are you sure thet you read it properly from the configuration file? Does it have any extra characters or quotes added there by accident? Values must not be surrounded by any special characters.
refresh_token = 1234567qwerty
If not, can you try to generate a new access and refresh token manually and them paste them directly into your code to be sure that reading from configuration file is not a problem?

I just need a token to programmatically interact with Box, that I can refresh without a browser getting involved There are two other authentication methods, which don't require browser involvement. Here are links to the guides how to set the up: Clinet Credential Grant and JWT.

Let me know if you managed to resolve this issue.
Best,
@lukaszsocha2

[skchronicles]

Hey @lukaszsocha2,

I hope you're having a great start to the new year. I will check out the documentation for the Client Credential Grant, and I will let you know if I run into any problems. Thank you for your help!

Best Regards,
@skchronicles

[skchronicles]

Hey @lukaszsocha2,

Is it possible to refresh a developer token? They have a relatively short lifespan.

[arjankowski]

Hi @skchronicles,

First, let's clarify what a developer token is. Please take a look at this link: https://developer.box.com/guides/authentication/tokens/developer-tokens/. As you can see, a developer token should be used during development and testing, not in production environments. When it expires, you should generate a new one on the developer console. There is no programmatic way to refresh it.

Instead of using a developer token, you can use either OAuth, CCG, or JWT authentication.
Here you can find more info about those auth methods.

The simplest one is CCG. Please take a look at our documentation here: https://github.com/box/box-python-sdk/blob/main/docs/usage/authentication.md#obtaining-user-token.

Please let me know if that helped

Regards,
Artur

[stale]

This issue has been automatically marked as stale because it has not been updated in the last 30 days. It will be closed if no further activity occurs within the next 7 days. Feel free to reach out or mention Box SDK team member for further help and resources if they are needed.

[stale]

This issue has been automatically closed due to maximum period of being stale. Thank you for your contribution to Box Python SDK and feel free to open another PR/issue at any time.