Standard operators should specify if array order is preserved or not

[ubamrein]

None of the operators, operating on arrays in https://openid.github.io/federation/main.html#name-operators specify, if a specific order should be preserved.

There might be metadata parameters that convey a meaning with a specific order, e.g. to express preferences. I think the specification should either mention that the order is not specified, or specify how the order is preserved.

[selfissued]

RFC 8259 says:

An array is an ordered sequence of zero or more values.

Order is always preserved in JSON arrays.

Does this address the issue at hand or is there something more specific that you'd recommend that we add?

[ubamrein]

Sorry I was a bit too unspecific. Yes the arrays themselves preserve order, I was more thinking about the result of the merging.

I am not sure if the operation of "union" or "intersection" determines a specific order. One might think that the order should be according to something I call "natural order" ([1]), but I think then the order in https://openid.github.io/federation/main.html#appendix-A.2.1 would be wrong (order of "token_endpoint_auth_methods_supported" in Figure 54 has client_secret_jwt appear before private_key_jwt where as the resulting metadata in Figure 67 has it the other way around).

I'm not sure if in that particular example the order has any meaning (at least I did not find anything in OAuth2/OpenID Connect), but it shows that the order maybe interpreted/implemented differently if not specified)

[1]: ["b", "a", "c"] intersection ["a", "b","d"] => ["b", "a"] (in contrast to [ "a", "b"])

[rohe]

Just looking at my own code I have a problem with preserving order (whatever that means).
Firstly I represent arrays at sets. That is any value in the array can only appear once.
Then I use the Python language set operator. For that operator, in what order the values in the result appear is undefined.

Given this I'd prefer that we keep it as it is and not make any attempt to preserve order.

I also am not clear as to what it means to preserve order.
Firstly, which array (if we have a union between two) is the 'master' given the assumption that it is the master's order we want to preserve. If that is the case, by the way, your example (if read left to right) provides the right answer.

[ubamrein]

I'd assume that order should follow the trust-chain, meaning elements higher up in the trust chain should be prioritised, as in be closer to the beginning of the array (if one wants to preserve order).

Given this I'd prefer that we keep it as it is and not make any attempt to preserve order.

Fair enough, I think it should be mentioned though that the order is unspecified! Also regarding conformance/integration tests one should take care, as comparing resulting metadata might not be deterministic (depending on how the merge happens, the order of attributes might be different every time)

Further, I was just wondering, if "downgrade" attacks would be possible, meaning a leaf could overrule a specific method providing higher security (I assume certain implementations would be somewhat deterministic in how they order the result, depending on the algorithms used) . As I said, I'm not aware of any particular example in the OAuth/OIDC space where order is important, so I think it is fine to leaf it as it is.

[selfissued]

I believe we should say that the order of the elements resulting from merging two sets is not defined.

[MichaelFraser1999]

Technically, maintaining any form of ordering when merging multiple sets will introduce a layer of unnecessary complexity to implementations that I don't believe would bring much value. I, too, would prefer we keep it as is

[ubamrein]

So I suggest adding something like

if so, under what conditions. 
+ The order of the result of such an operator value merge is not defined.

to:

federation/openid-federation-1_0.xml

Line 2073 in 6257b76

policy, and if so, under what conditions. If the operator does

[Razumain]

I'd assume that order should follow the trust-chain, meaning elements higher up in the trust chain should be prioritised, as in be closer to the beginning of the array (if one wants to preserve order).

This is a tricky one, and a good observation. And this might be more important than we first think.

A quick search gave this hit where order matters:

default_acr_values
OPTIONAL. Default requested Authentication Context Class Reference values. Array of strings that specifies the default acr values that the OP is being requested to use for processing requests from this Client, with the values appearing in order of preference.

In addition to this, OpenID federation may be used for other protocols than OAuth and OpenID Connect. Such protocols could very well have arrays where the order has some kind of importance.

I think preservation of order could be fairly straightforward. But I would propose the opposite to what was suggested by @ubamrein in the quote above.

The metadata should be the metadata as it is expressed by the leaf entity. Policy preferably only reduce the options expressed by the leaf. If a policy adds to the leaf metadata, then those options should be added to the end of the list. We should avoid that the metadata resolved by the peer is a different order than the leaf published as its preferences. Policy modifiers are not intended to modify order, and thus should not do so.

[rohe]

I agree with @MichaelFraser1999 , @ubamrein and @selfissued.
More specifically I think we should add the line that @ubamrein proposes.

[ubamrein]

The metadata should be the metadata as it is expressed by the leaf entity. Policy preferably only reduce the options expressed by the leaf. If a policy adds to the leaf metadata, then those options should be added to the end of the list. We should avoid that the metadata resolved by the peer is a different order than the leaf published as its preferences. Policy modifiers are not intended to modify order, and thus should not do so.

But reordering could happen during reduction of options. And since the leaf generally has lower trust than some party higher up in the trust chain I'd rather argue that the final, resolved metadata should reflect design decisions of the trust anchor - or any intermediaries for that purpose - (say the ordering of default_acr_values), as it might relate to policies required for passing a certification (I'm just handwaving some hypotheticals, as I don't have a concrete example).

As I do agree with @MichaelFraser1999 in his point of unnecessary complexity, and my lack of finding any really relevant example, I ended up with proposing to explicitly specify that order is not preserved. This still leafs open the option to use the leaf-statements ordering, if an implementation decides to do so, but prevents people relying on the order in their design of policies (and getting a wrong sense of "security").