Claim/Scope-Tests do not support Distributed Claims

[marsangr]

Tests involving queries to the Userinfo-Endpoint and evaluating return values crash with the message

status=ERROR, message='NoneType' object is not callable

if confronted with distributed claims in the answer (s. https://openid.net/specs/openid-connect-core-1_0.html#AggregatedDistributedClaims). For instance:

https://op.certification.openid.net:60381/test_info/OP-scope-All

[rohe]

I'll have a look at this tomorrow.

[rohe]

OK, I know what the problem is.
Will fix, but it will take some time before the official test site is running the fixed code.

[selfissued]

About this, Roland wrote "This was/is a bug in pyoidc.
I have a Pull Request (PR) in the making just waiting for the one on phone_number_verified to be accepted.
I don’t like having many PRs active at the same time."

[zandbelt]

It is this PR that we're waiting for:
CZ-NIC/pyoidc#420
once it is included and pyoidc version 0.11.2(?) is released we will upgrade the production environment to OP 2.0.7 that will use that new version of pyoidc.

[rohe]

13 sep. 2017 kl. 11:35 skrev Hans Zandbelt ***@***.***>: It is this PR that we're waiting for: CZ-NIC/pyoidc#420

Yes.

once it is included and pyoidc version 0.11.2(?) is released we will upgrade the production environment to OP 2.0.7 that will use that new version of pyoidc.

Would be 0.11.1.0 I think.

…

-- Roland "Education is the path from cocky ignorance to miserable uncertainty.” - Mark Twain

[zandbelt]

pyoidc v0.12.0 is now deployed with OP release 2.0.7

[marsangr]

I would like to reopen this issue, something is not working properly, the error is a different one though and further down the workflow: The test client follows the redirection to the claims provider, but is not able to parse the answer and fails without further details (and a partial result)

Cf. the same test case
https://op.certification.openid.net:60381/test_info/OP-scope-All

25.851	request	{'body': None}
25.851	request_url	https://identityagent.de/userinfo
25.851	request_http_args	{'headers': {'Authorization': 'Bearer eyJraWQiOiJGT3Z5IiwiYWxnIjoiUlMyNTYifQ.eyJzdWIiOiJtYXJjb3MuZGVuaWMuZGUiLCJzY3AiOlsib3BlbmlkIl0sImNsbSI6WyJ3ZWJzaXRlIiwiem9uZWluZm8iLCJiaXJ0aGRhdGUiLCJlbWFpbF92ZXJpZmllZCIsImFkZHJlc3MiLCJnZW5kZXIiLCJwcm9maWxlIiwicGhvbmVfbnVtYmVyX3ZlcmlmaWVkIiwicHJlZmVycmVkX3VzZXJuYW1lIiwibG9jYWxlIiwibWlkZGxlX25hbWUiLCJnaXZlbl9uYW1lIiwicGljdHVyZSIsInVwZGF0ZWRfYXQiLCJuYW1lIiwibmlja25hbWUiLCJwaG9uZV9udW1iZXIiLCJmYW1pbHlfbmFtZSIsImVtYWlsIl0sImRhdCI6eyJyZWplY3RlZF9jbGFpbXMiOltdfSwiaXNzIjoiaHR0cHM6XC9cL2F1dGguZnJlZWRvbS1pZC5kZSIsImV4cCI6MTUxMjQ2OTI2NiwiaWF0IjoxNTEyNDY4NjY2LCJjaWQiOiJubXN6dGVxaGtuYW55In0.JLcqSHT-cmPj3hW75ag2PTYW5csnNftL7txkLy0Hi3D-z1Xc98fGO-4ZXmlUXXHiSX9zvyHExy0qyBypEyyA5MiL0tFgv23rscfN1nz3LLKhqEnp8KrbUWxiqZl5MiCVawW257mFUvaXJUiv0-MsM7WSru8RF-EgH5Ie-dmXV2hkni3fFz_c1d74s2zsipsBWEnn6eaQl2c7Dj7EEDyf6K3sSNCuGpC5SDyiox7MnYvcggwE1Unurj-3pp4aE48gET9rifSWwUr0uohYf6z3fvy7V8kIO6-mNJdQUWPY04EkoH3eJk30CSqXqychER0lLItLsdo0OmE92maCTVpmeQ'}}
26.507	http response	url:https://identityagent.de/userinfo status_code:200
26.51	condition	UserInfo:OP-scope-All: status=ERROR, message=

[zandbelt]

The error in the server logs is:

2017-12-05 10:11:07,541 requests.packages.urllib3.connectionpool:INFO Starting new HTTPS connection (1): identityagent.de
2017-12-05 10:11:08,195 requests.packages.urllib3.connectionpool:DEBUG "GET /userinfo HTTP/1.1" 200 1116
2017-12-05 10:11:08,197 oidctest.op.client:DEBUG Reponse text: 'eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJtYXJjb3MuZGVuaWMuZGUiLCJpc3MiOiJodHRwczovL2lkZW50aXR5YWdlbnQuZGUiLCJleHAiOjE1MTI0Njg5NTcsImNsYWltcyI6eyJwaG9uZV9udW1iZXIiOiI1NTUgMTIzNDUiLCJmYW1pbHlfbmFtZSI6IlNhbnogR3Jvc3NcdTAwZjNuIiwibmFtZSI6Ik1hcmNvcyBTYW56IEdyb3NzXHUwMGYzbiIsImVtYWlsX3ZlcmlmaWVkIjoic2FuekBkZW5pYy5kZSIsImdpdmVuX25hbWUiOiJNYXJjb3MiLCJlbWFpbCI6Im1hcmNvc0BkZW5pYy5kZSJ9LCJpYXQiOjE1MTI0Njg2NTcsIm5iZiI6MTUxMjQ2ODY1N30.WcLTstQA7t1aCxpZl5cxfCefFUZgGiebiXeKFWMVpJa9IPq035kbUmmGgaVWmxVFX8ygEOZCqCREjrcZW_zTq2sNmrp2DWf76B1hU_mF3r-LOmGM3YWn28cVcCKR5P8peXn7rkAKAvuRI3JqurkJOrbj9DKEgnIWlRWTGFtnqjfSUwGs72tv_I_MJ4geTMEzxvCSV7gosZe6xl7u5mjR-EMNZX-pu9S0BeJ3NuKQ09FNKLVqBqIDZ266R42Bsm7l9wOHhGwNefep6UhNVd6cenDTLFjeSJf_K1vLe0nN0EgejtbYs_vCAgETQ4SaqFCQP8JRhnCih-DosgAWjMhSafVAmc6sc44PCIwteQiyvTS7PgAvH9K_TXexwKUIFYnkGS1o-djZfm-6gu_wnInZlDBSlBtnLnLDXeVFbIP6j73UeNxAo5c6fTfuul8E29iHq6t1tdo4u7AFTon7W4U6fagJdBuEJGrkNre2vBu121ByDEzMghUcXYkbtbRNm7k0oyES0YnbOT_hRzuAsbMkhOHVwkLmzlkW1kCRS7eaqnfGT8kN2IQLcF7K0bKTI3hP-0fY9AMPk-_M9hsJTPz-Xe8Nq9SGoFSbBlflC-vL0McyW8Wm_1Y3bHLw0qLWIOB50zqNV_2yKgzlQfWoWqKLag63SkGKyZHWzRkogJBJglA'
2017-12-05 10:11:08,197 oic.oauth2.message:DEBUG Raw JSON: {'iat': 1512468657, 'claims': {'email_verified': 'sanz@denic.de', 'phone_number': '555 12345', 'given_name': 'Marcos', 'name': 'Marcos Sanz Grossón', 'email': 'marcos@denic.de', 'family_name': 'Sanz Grossón'}, 'iss': 'https://identityagent.de', 'sub': 'marcos.denic.de', 'nbf': 1512468657, 'exp': 1512468957}
2017-12-05 10:11:08,197 oic.oauth2.message:DEBUG JWS header: {'alg': 'RS256', 'typ': 'JWT'}
2017-12-05 10:11:08,197 oic.oauth2.message:ERROR Issuer "https://identityagent.de" not in keyjar
2017-12-05 10:11:08,197 oic.oauth2.message:DEBUG Found signing key.
2017-12-05 10:11:08,198 jwkest.jws:DEBUG Picking key by key type=RSA
2017-12-05 10:11:08,198 jwkest.jws:DEBUG Picking key based on alg=RS256, kid=None and use=
2017-12-05 10:11:08,198 jwkest.jws:DEBUG Picked: kid:wt25OgyR_nzG3OoQ7daa2rL6-gMnFdfRzBjhUVPu8RQ, use:sig, kty:RSA
2017-12-05 10:11:08,198 jwkest.jws:DEBUG Picked: kid:FOvy, use:sig, kty:RSA
2017-12-05 10:11:08,198 jwkest.jws:DEBUG Picked: kid:9wzy, use:sig, kty:RSA
2017-12-05 10:11:08,198 jwkest.jws:DEBUG Picked: kid:CXup, use:sig, kty:RSA
2017-12-05 10:11:08,200 otest.aus.tool:ERROR [UserInfo] ExcList: Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/otest-0.7.2-py3.5.egg/otest/aus/tool.py", line 86, in run_flow
    resp = _oper()
  File "/usr/local/lib/python3.5/dist-packages/otest-0.7.2-py3.5.egg/otest/operation.py", line 105, in __call__
    res = self.run(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/oidctest-0.7.0-py3.5.egg/oidctest/op/oper.py", line 322, in run
    user_info = self.conv.entity.fetch_distributed_claims(user_info)
  File "/usr/local/lib/python3.5/dist-packages/oic-0.12.0-py3.5.egg/oic/oic/__init__.py", line 1071, in fetch_distributed_claims
    userinfo_endpoint=spec["endpoint"])
  File "/usr/local/lib/python3.5/dist-packages/oidctest-0.7.0-py3.5.egg/oidctest/op/client.py", line 104, in do_user_info_request
    sender=self.provider_info["issuer"])
  File "/usr/local/lib/python3.5/dist-packages/oic-0.12.0-py3.5.egg/oic/oauth2/message.py", line 699, in from_jwt
    _jw.verify_compact(txt, key)
  File "/usr/local/lib/python3.5/dist-packages/pyjwkest-1.4.0-py3.5.egg/jwkest/jws.py", line 517, in verify_compact
    return self.verify_compact_verbose(jws, keys, allow_none, sigalg)['msg']
  File "/usr/local/lib/python3.5/dist-packages/pyjwkest-1.4.0-py3.5.egg/jwkest/jws.py", line 588, in verify_compact_verbose
    raise BadSignature()
jwkest.BadSignature

2017-12-05 10:11:08,200 otest.aus.tool:ERROR [UserInfo] Exception:

Looks like an issue with the key?

[marsangr]

Signature looks good on my side.

Please remember that with distributed claims the client has an additional step to carry, namely, to fetch anew a public key: the public key of the claims provider. The client cannot use the keyset of the issuer to validate the answer of the claims provider.

If you take a look at
https://identityagent.de/.well-known/openid-configuration
you'll see that the next JWKS URI is
https://identityagent.de/jwks.json

[zandbelt]

Reading up on distributed claims, it seems to be somewhat underspecified: http://openid.net/specs/openid-connect-core-1_0.html#AggregatedDistributedClaims.

The spec doesn't actually mention anything about verification of the JWT that carries the distributed claims so it doesn't automatically imply that one should be able to deduce the key with Discovery through an "iss" value as you'd do with regular OIDC. Though It is not far fetched either.

Implementations may actually rely on the TLS certificate presented on the "endpoint" for authenticity, I've seen at least one implementation doing that, though that's not explicitly specified either. (basically you'd end up with the same security model as obtaining keys from a TLS protected jwks_uri)

@selfissued @rohe any comments to that?