New Token Format Proposal

[eternal-flame-AD]

The original threat model for this should be: attacker gets a read-only copy of the database, we should not allow them to make authenticated requests. In other words, BOTH the client AND the backend database record should be prevented from recovering or forging valid tokens.

The properties I think we should aim for:

• Contain data secret from the server, ideally in a way that can be used without ever being revealed back to the server
• Contain some authenticated or verifiable plaintext claims (like token type, application id, etc)
• Rotation and revocation should be possible without changing the claim itself
• Backwards compatible with verbatim tokens
• Not too long for webhook forms with length limits
• Not too rigidly bound to the entity ontology (i.e. a token doesn't necessarily need to have an associated user) for future features like sharing messages

A new token could look something like this:

• A literal G (or gtfy), to distinguish it with old tokens.
• Underscore.
• A sequence of claims separared by underscore, a single letter denoting the next claim type, like "U1" means user ID 1, "A2" means application ID 2, "P3" means accessing resources for plugin 3. This is already similar to the property based authorization system in the auth package.
• An underscore.
• The "key" encoded in the token, randomly generated by the server at creation time. The server only records response of HMAC(key=key,salt=claims) and never commits the key to the database, so someone with read-only access to the database still have an HMAC preimage problem to solve to find a valid "key". (HKDF might be better in order to protect against confused deputy scenarios and gives us more degrees of freedom on binding more necessary context without canonicalization headaches, but we can decide that as we write the PR)
• Optional/for future: A period (.) then a counter , IIRC way back in the original discussion I put forward some reservations on the practical security of merely hashing a token and yet demand the client to send the "answer" to you in every request. This part of the token being present causes the database to update the max counter recorded in the system for each token, and the "key" part being double hashed with a further message containing the counter instead of the key itself. We can detail this up in another discussion.
• For future proofing we might want to generate "keys" that are in addition to being unpredictable, but also valid private keys in a public key crypto system (Curve25519, etc)

Currently we will only issue tokens mapping back to the current system, like "gtfy_U1_A2_0123456789abc"

Footnote: the HMAC output can be simply stored by reusing the token field, we just need to make sure they don't start with "ACP" so they can't be confused with legacy verbatim tokens

References:

#325
#964 (comment)
An old PR that is finally unblocked with #964: #707

@jmattheis

[jmattheis]

Contain data secret from the server, ideally in a way that can be used without ever being revealed back to the server

With the HMAC/HKDF solution, this is not satisfied right? for validation the client has to send their secret to the server, so that gotify/server can recompute and compare the result against the stored mac.

Your optional proposal "Optional/for future: A period (.) then a counter" would solve this

(I just want to ensure I understand the solution).

Currently we will only issue tokens mapping back to the current system, like "gtfy_U1_A2_0123456789abc"

Do we need the user id for extra validation. The application/plugin/client id should be enough as we can identify the user with it.

The "key" encoded in the token, randomly generated by the server at creation time. The server only records response of HMAC(key=key,salt=claims) and never commits the key to the database, so someone with read-only access to the database still have an HMAC preimage problem to solve to find a valid "key". (HKDF might be better in order to protect against confused deputy scenarios and gives us more degrees of freedom on binding more necessary context without canonicalization headaches, but we can decide that as we write the PR)

HKDF doesn't seem much more difficult/different from just using HMAC validation. So I'm okay with both.

Optional/for future: A period (.) then a counter , IIRC way back in the original discussion I put forward some reservations on the practical security of merely hashing a token and yet demand the client to send the "answer" to you in every request. This part of the token being present causes the database to update the max counter recorded in the system for each token, and the "key" part being double hashed with a further message containing the counter instead of the key itself. We can detail this up in another discussion.

Does this mean, the client has to keep track of their counter too?

I like the benefits from this, but I don't think this is easily supported by our clients. We have concurrent authenticated requests e.g. for images where we probably can't guarantee the order without sending all requests in sequence.

Let's postpone this to a separate discussion.

For future proofing we might want to generate "keys" that are in addition to being unpredictable, but also valid private keys in a public key crypto system (Curve25519, etc)

For now this only means using [32]byte as secret key right?

A literal G (or gtfy), to distinguish it with old tokens.
Underscore.
A sequence of claims separared by underscore, a single letter denoting the next claim type, like "U1" means user ID 1, "A2" means application ID 2, "P3" means accessing resources for plugin 3. This is already similar to the property based authorization system in the auth package.
An underscore.

Sounds good, I'd probably prefer lowercase 'a', 'p' but it doesn't really matter. I like gtfy or gfy as it's probably easier to scan for exposed tokens as the bigger identifier reduces false positives.

---

I like the proposed approach, but as a side note: have you considered using JWT as format. This would mean including the claims in the token, but using a standard format sounds like a good idea too.

It would increase the token size, but probably in an acceptable way. I slightly prefer your proposed solution, but we should still consider JWT.

[eternal-flame-AD]

With the HMAC/HKDF solution, this is not satisfied right? for validation the client has to send their secret to the server, so that gotify/server can recompute and compare the result against the stored mac.

Yes, I think that's the original debate where I was slightly reserved about the point of doing this. The original issue was talking about a threat model where only a read only copy of the database was leaked, not the entire gotify server being compromised. For the latter scenario it would no longer suffice to use a token but an HMAC OTP challenge response (like the counter solution) or a signature (like HTTP signatures RFC9421)

---

The client doesn't necessarily need to keep track of the counter, it can be simply a ratchet that the server keeps track of to increase monotonically. Stateless clients can use hybrid Unix nano timestamps concatenated with a local counter (ie 12345123 for the 123th request sent since last application restart at timestamp 12345). We can also use arbitrary precision number (ie string comparison) to make sure there is no practical upper bound on granularity.

We could work around this stateless problem by using a real challenge response like smartcard mutual authentication, but it would require two requests (one for downloading a list of signed random challenges).

---

Yeah any random sequence should technically work as a Curve25519 keys but there are some canonicalization steps like clamping that a dedicated generation step would do to help with rare but not impossible duplicates, etc.

---

JWT would work as well but it would be pretty long to include in the URL and still only protect the leaked database backup case, not a man in the middle case. The biggest benefit would be interoperability with third party systems and pave the way of entirely externally managed tokens (gotify only verify the token against an issuer public key).

[eternal-flame-AD]

Regarding the app being harder to program I think it's a tradeoff that we must do if we want protection against some kind of read only man in the middle problem (like token in HTTP server logs). The counter can start at an empty string "" and we can check a token without a counter is only valid if the server recorded counter is "", this way the token works "out of the box" if the user choose to store and send them back verbatim but upgrades to an OTP as soon as they send a request with a counter.

The implementation is actually quite straightforward, the application would simply pick out the "key" part and double hash it with a counter. It should be doable with just 20 lines or so of Go code.