Synapse login not the same with authToken and apiKey

[sujaypatil96]

Bug Report

Operating system

MacOS Big Sur

Client version

Versions 2.2.2 and 2.3.1.

Description of the problem

When using the .synapseConfig file (with the apiKey attribute) as in (for example) synapseclient==2.2.2) the synapseclient.Synapse.login() method works perfectly. However, when using the .synapseConfig (with the authToken attribute) as in (for example), synapseclient==2.3.1), the login method doesn't work as expected.

A minimal reproducible example:

• Install version 2.2.2 of the synapseclient

$ pip install synapseclient==2.2.2

$ python
>>> import synapseclient
>>> syn = synapseclient.Synapse(configPath='/Users/spatil/Desktop/schematic/.synapseConfig')
>>> syn.login(silent=True)

• Repeat the above with version 2.3.1
• Observe the differences in behaviour

Note: Make sure to you the right versions of the .synapseConfig file too.

Expected behavior

User should be logged in successfully.

Actual behavior

No output to console when testing with synapseclient==2.3.1 and using .synapseConfig file with authToken.

[jkiang13]

@sujaypatil96 I'm having some trouble reproducing this issue.

Could you show the [authentication] sections of the synapseConfig file you are using with 2.2.2 and 2.3.1, respectively (with secrets redacted of course).

One potential issue I am aware of is that in 2.3.1 if you are using an email address (rather than the username) as your synapseConfig username this works with an apiKey but not with an authToken. This is fixed in the upcoming 2.4 version (username is not needed at all with an authToken, but if provided it is checked and this check was not working properly with an email/authToken pair).

The above case aside, here are my own steps to attempt to reproduce this:

jordank@003 /tmp % source gh869/bin/activate

(gh869) jordank@003 /tmp % pip install synapseclient==2.2.2
Collecting synapseclient==2.2.2
  Using cached synapseclient-2.2.2.tar.gz (164 kB)
Collecting requests<3.0,>=2.22.0
  Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB)
Collecting keyring==12.0.2
  Using cached keyring-12.0.2-py2.py3-none-any.whl (32 kB)
Collecting deprecated<2.0,>=1.2.4
  Using cached Deprecated-1.2.12-py2.py3-none-any.whl (9.5 kB)
Collecting entrypoints
  Using cached entrypoints-0.3-py2.py3-none-any.whl (11 kB)
Collecting wrapt<2,>=1.10
  Using cached wrapt-1.12.1-cp39-cp39-macosx_10_15_x86_64.whl
Collecting idna<3,>=2.5
  Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting chardet<5,>=3.0.2
  Using cached chardet-4.0.0-py2.py3-none-any.whl (178 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.5-py2.py3-none-any.whl (138 kB)
Using legacy 'setup.py install' for synapseclient, since package 'wheel' is not installed.
Installing collected packages: wrapt, urllib3, idna, entrypoints, chardet, certifi, requests, keyring, deprecated, synapseclient
    Running setup.py install for synapseclient ... done
Successfully installed certifi-2021.5.30 chardet-4.0.0 deprecated-1.2.12 entrypoints-0.3 idna-2.10 keyring-12.0.2 requests-2.25.1 synapseclient-2.2.2 urllib3-1.26.5 wrapt-1.12.1
WARNING: You are using pip version 21.0.1; however, version 21.1.2 is available.
You should consider upgrading via the '/private/tmp/gh869/bin/python3.9 -m pip install --upgrade pip' command.
(gh869) jordank@003 /tmp % cat ~/.synapseConfig
[authentication]
username=jordank
apiKey=<redacted>

(gh869) jordank@003 /tmp % synapse --version
Synapse Client 2.2.2
(gh869) jordank@003 /tmp % python
Python 3.9.4 (default, Apr  5 2021, 01:49:30) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import synapseclient; syn = synapseclient.Synapse(configPath='/Users/jordank/.synapseConfig')
>>> syn = synapseclient.login(silent=True)
UPGRADE AVAILABLE

A more recent version of the Synapse Client (2.3.1) is available. Your version (2.2.2) can be upgraded by typing:
    pip install --upgrade synapseclient

Python Synapse Client version 2.3.1 release notes

https://python-docs.synapse.org/build/html/news.html

>>> print(syn.credentials)
SynapseCredentials(username='jordank', api_key_string='<redacted>')

>>> quit()
(gh869) jordank@003 /tmp % pip install synapseclient==2.3.1
Collecting synapseclient==2.3.1
  Using cached synapseclient-2.3.1-py3-none-any.whl (205 kB)
Requirement already satisfied: deprecated<2.0,>=1.2.4 in ./gh869/lib/python3.9/site-packages (from synapseclient==2.3.1) (1.2.12)
Requirement already satisfied: requests<3.0,>=2.22.0 in ./gh869/lib/python3.9/site-packages (from synapseclient==2.3.1) (2.25.1)
Requirement already satisfied: keyring==12.0.2 in ./gh869/lib/python3.9/site-packages (from synapseclient==2.3.1) (12.0.2)
Requirement already satisfied: entrypoints in ./gh869/lib/python3.9/site-packages (from keyring==12.0.2->synapseclient==2.3.1) (0.3)
Requirement already satisfied: wrapt<2,>=1.10 in ./gh869/lib/python3.9/site-packages (from deprecated<2.0,>=1.2.4->synapseclient==2.3.1) (1.12.1)
Requirement already satisfied: certifi>=2017.4.17 in ./gh869/lib/python3.9/site-packages (from requests<3.0,>=2.22.0->synapseclient==2.3.1) (2021.5.30)
Requirement already satisfied: idna<3,>=2.5 in ./gh869/lib/python3.9/site-packages (from requests<3.0,>=2.22.0->synapseclient==2.3.1) (2.10)
Requirement already satisfied: chardet<5,>=3.0.2 in ./gh869/lib/python3.9/site-packages (from requests<3.0,>=2.22.0->synapseclient==2.3.1) (4.0.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./gh869/lib/python3.9/site-packages (from requests<3.0,>=2.22.0->synapseclient==2.3.1) (1.26.5)
Installing collected packages: synapseclient
  Attempting uninstall: synapseclient
    Found existing installation: synapseclient 2.2.2
    Uninstalling synapseclient-2.2.2:
      Successfully uninstalled synapseclient-2.2.2
Successfully installed synapseclient-2.3.1
WARNING: You are using pip version 21.0.1; however, version 21.1.2 is available.
You should consider upgrading via the '/private/tmp/gh869/bin/python3.9 -m pip install --upgrade pip' command.
(gh869) jordank@003 /tmp % synapse --version
Synapse Client 2.3.1
(gh869) jordank@003 /tmp % cat ~/.synapseConfig
[authentication]
authToken=<redacted>
(gh869) jordank@003 /tmp % python
Python 3.9.4 (default, Apr  5 2021, 01:49:30) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import synapseclient; syn = synapseclient.Synapse(configPath='/Users/jordank/.synapseConfig')
>>> syn = synapseclient.login(silent=True)
>>> print(syn.credentials)
SynapseAuthTokenCredentials(username='jordank', token='<redacted>')
>>> 

[sujaypatil96]

@jkiang13: Yes, absolutely. So I was testing with just version 2.3.1. My .synapseConfig file's [authentication] section looks like this:

[authentication]
username = "sujaypatil"
authtoken = "<redacted>"
#apikey = "<redacted>"

I commented out authToken to ensure using apiKey worked correctly, and vice versa. When I commented out the apiKey part and used the authToken it seemed to just not output anything to the screen. I even created a new Personal Access Token to make sure that I wasn't using an expired one.

[sujaypatil96]

Oh, I think you should be doing syn.login()? I see you're doing synapseclient.login()?

[jkiang13]

@sujaypatil96 is it just the lack of output or is the login not functioning (i.e. are you able to perform authenticated activity)?

If it's the former, what output are you anticipating? In my output above in 2.2.2, with silent=True, upon login I get the warning about the availability of the newer version, but no other output. When using the latest version there is no output, but that is expected with silent=True (arguably the new version warning should also be suppressed with silent=True). Without silent=True there is a console message about the successful login.

[sujaypatil96]

That's interesting. With the new version, I'm seeing this on my console:

Would you mind changing the statement in your snippet to syn.login() and testing again?

[jkiang13]

You're right, I did mean to do syn.login, but in this case it didn't matter (because I happened to be using the default configPath anyway)

(gh869) jordank@003 /tmp % synapse --version
Synapse Client 2.3.1
(gh869) jordank@003 /tmp % cat ~/.synapseConfig
[authentication]
authToken=<redacted>
(gh869) jordank@003 /tmp % python                          
Python 3.9.4 (default, Apr  5 2021, 01:49:30) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import synapseclient; syn = synapseclient.Synapse(configPath='/Users/jordank/.synapseConfig')
>>> syn.login(silent=True)
>>> print(syn.credentials)
SynapseAuthTokenCredentials(username='jordank', token='<redacted>')
>>> 

Can you run the following after the output-less login?

print(syn.credentials)

[sujaypatil96]

>>> print(syn.credentials)
None

This is what I'm seeing @jkiang13.

[jkiang13]

@sujaypatil96 Hmm, can you verify that the auth token is exactly one generated from your settings at synapse.org (for example no characters accidentally omitted when pasting it into your config file).

One way to confirm that it is a valid token is to run the following from the command line:

curl --location --request GET 'https://repo-prod.prod.sagebase.org/repo/v1/userProfile' --header 'Authorization: Bearer <insert token here>'

Replacing with your auth token. If a valid token it should return some JSON describing your user profile. If an invalid token, should return some JSON with an error message.

While debugging this I noticed some cases where an invalid token causes the backend to return a 500 error (rather than a 401 status code) which causes the client to retry (and appear to hang). I wonder if that is what is happening in your case.

[sujaypatil96]

Thanks for sending the above command to me @jkiang13. I ran it, and here is the output:

{"ownerId":"3410110","etag":"a6498af3-b45f-486a-9e68-4fecb151855c","firstName":"Sujay","lastName":"Patil","emails":["sujaysan@usc.edu"],"openIds":[],"userName":"sujaypatil","notificationSettings":{"sendEmailNotifications":true},"createdOn":"2020-05-18T14:57:43.000Z"}%   

It is a valid token, since it's returning a valid JSON with my user credentials. So the token doesn't seem to be the problem. Let me know if you feel like a call will help debug this better?

[sujaypatil96]

Just updating this issue with the real reason as to why I was encountering this problem. Turns out it has to do with inconsistency in specifying the .synapseConfig file between versions of synapseclient. In versions >= 2.3.0, we cannot wrap the username and authtoken attributes in double quotes. The client doesn't parse them correctly. However in previous versions <2.3.0, we could wrap the username and apikey attributes in double quotes and the client would parse them successfully.

[jkiang13]

I've created the following JIRA issue to track whether to support quoted config values: https://sagebionetworks.jira.com/browse/SYNPY-1162 . It turns out that most config values do NOT currently support quoted strings, it just so happens that both username and apiKey do because of side effects (see the JIRA issue for details) rather than as intentional decisions, so there is some behavioral inconsistency.

With the JIRA issue open, I'll close this GitHub issue.

Debugging this also revealed https://sagebionetworks.jira.com/browse/PLFM-6803