[4.0] Token authentication for the API application

[nikosdion]

Pull Request for Issue #26925. Closes gh-26925.

Summary of Changes

This PR implements token authentication for the Joomla API application. This is a much more secure approach as already discussed in length; see gh-26925

Testing Instructions

0. Initialization

Install a new site with this PR's branch. If applying on an existing site you need to also:

• Go to System, Database and fix the schema.
• Go to System, Discover and install the "API Authentication - Joomla Token" and "User - Joomla Token" plugins.
• Go to System, Plugins and enable the "API Authentication - Joomla Token" and "User - Joomla Token" plugins.

1. User interface test

Go to the site's backend, Users, Manage.

Edit your Super User.

Expected: There is a User Token tab with an empty token.

Click on Save.

A token is now visible. Copy your token, we'll need it below.

Note: I know the display of the Token tab is a bloody mess. That's a template issue, not something I can fix in my PR. I am simply using an XML form.

2. API access

Now we need to run a request against the API server with one of the following alternatives. In all
cases we assume that the site's URL is http://localhost/joomla4 and the token c2hhMjU2Ojk3MTpiM2IzYTZjNzRmNGUwY2U0NGM3OGVkMDcyMjdhYjFlOWFmMzExODExZjZhNmE3ZGFlZDUwZWRiNmJkZTVlYzJl.

Alternative A. cURL

From the command line:

curl -H "Authorization: Bearer c2hhMjU2Ojk3MTpiM2IzYTZjNzRmNGUwY2U0NGM3OGVkMDcyMjdhYjFlOWFmMzExODExZjZhNmE3ZGFlZDUwZWRiNmJkZTVlYzJl" http://localhost/joomla4/api/index.php/v1/users

Alternative B. WGet

From the command line:

wget --header="Authorization: Bearer c2hhMjU2Ojk3MTpiM2IzYTZjNzRmNGUwY2U0NGM3OGVkMDcyMjdhYjFlOWFmMzExODExZjZhNmE3ZGFlZDUwZWRiNmJkZTVlYzJl" -O - http://localhost/joomla4/api/index.php/v1/users

Alternative C. phpStorm

File, New Scratch File, HTTP Request. Enter the following at the end of the document:

GET http://localhost/joomla4/api/index.php/v1/users
Authorization: Bearer c2hhMjU2Ojk3MTpiM2IzYTZjNzRmNGUwY2U0NGM3OGVkMDcyMjdhYjFlOWFmMzExODExZjZhNmE3ZGFlZDUwZWRiNmJkZTVlYzJl
User-Agent: JoomlaDevelopment/4.0.0
Accept: application/vnd.api+json

###

Choose Run, Run scratch#whatever to execute the HTTP request. CTRL-click (CMD-click on macOS) on the filename appearing above the ### marker to view the response.

Expected Response

A JSON document listing your site's users.

Technical notes

For security reasons we are NOT storing the raw token in Joomla's database. If we were to do that we'd be a SQL injection away from full system compromise. A SQL injection would allow the attacker to dump the tokens. Since tokens are, by nature, both a username and a password they'd be able to fully compromise the site.

Instead, all we store in the database is a seed value that's currently hardcoded to a length of 32 bytes (256 bits). The seed value is generated with the cryptographically secure random bytes generator of Joomla's Crypt package and stored in the database in base64 encoded format. We store this information in the #__user_profiles table.

The token itself is the base64-encoded representation of a string algorithm:user_id:hmac. Below we discuss the various components, not in the same order present in the token.

The hmac is the main security feature of this token implementation. This is the RFC 2104 HMAC of the seed data using Joomla's site secret (the $secret in configuration.php) as the key. This is a very secure process which avoids the known attacks against hash algorithms such as SHA-1 and SHA-256 (reference).

The algorithm tells us which cryptographic hash function was used to generate the HMAC. Per the security considerations of RFC 6151 we forbid the use of the less secure cryptographic hash functions. In fact, the current implementation only allows SHA-256 and SHA-512 with SHA-256 being the (hardcoded) method used to generate the token displayed to the user. That is to say, SHA-512 is added for forward compatibility.

The user_id is the numeric user ID. This is used to make token validation possible. Note that if you have a token you have full access to the site, including the /v1/users endpoint. So please let's not have the pointless discussion of whether the token is "leaking" the user ID. If you can't understand why it doesn't you're not qualified to discuss token authentication.

The api-authentication plugin expects to see the token in the Authorization HTTP header in the request. We strongly recommend using the HTTP header method. The header MUST have the format:
Authorization: Bearer <token>
where <token> is your Joomla API token in base64 encoding. Note that the string Bearer is case sensitive and must have at least one space separating from the token. Any additional whitespace around the token is discarded.

Upon receiving the token, the api-authentication plugin breaks it into its known parts. As long as the algorithm is one of the supported values (sha256 or sha512) authentication proceeds. We read the user profile information and retrieve the seed data for the user_id in the token message. Using this seed data from the database and the site's secret from configuration.php we calculate the reference HMAC. The reference HMAC is compared to the retrieved HMAC from the token message using a time-safe comparison. It is worth noting that even when we encounter an invalid user ID we still go through all the motions for authentication to prevent timing attacks which would allow user enumeration.

As long as the HMACs match, the token is enabled and the user belongs to the allowed user groups and is not blocked, not yet activated or has requested a password reset we are going to grant them access to the API application.

Since the API application is currently only allowed for Super Users we limit the access to the token management UI and token authentication only to Super Users by default. This is user configurable in the user/token plugin.

For security and privacy reasons users can only view their own token, even if they are Super Users. Users with com_users edit privileges can also disable, enable or reset another user's token BUT they still cannot view it. This is useful in case there is a suspicion that the token of a user was compromised and another responsible adult needs to log into the site's backend and disable or reset it before things get out of hand.

There are two plugins, a user plugin to render the management UI and an api-authentication which performs the actual token validation. Combining them would pose architectural issues and give the wrong ideas to third party developers and users as discussed in gh-26925. A potentially confusing situation would arise when the api-authentication plugin is disabled and the user plugin is enabled: the management UI is there but nobody's home to answer the door. To prevent that the user plugin will refrain from rendering the token management UI when the api-authentication plugin is disabled. The reverse situation DOES NOT cause the api-authentication plugin to disable itself. There are valid reasons to do that, e.g. prevent a nosy client from breaking things while swearing that they didn't touch anything...

Finally, as it should be pretty obvious, the token authentication only works for Joomla's API application, NOT the frontend and backend applications. Moreover, due to the remote and unattended nature of API application access the token authentication bypasses Two Factor Authentication.

As far as security goes, at worst for us and best for them, an attacker would need to guess correctly the HMAC-SHA256 sum of a user on top of their user ID. An HMAC-SHA25 sum is 256 bits long and chaotic in nature. This means that an attacker would need to go through approximately half of the key space before cracking it, i.e. 2^128 requests. That's more than 340,000,000,000,000,000,000,000,000,000,000,000,000 requests to the server. We can expect the heat death of the universe before that happens.

HMAC-SHA256 algorithm attacks are not even in a theoretical stage yet. Besides, a such an attack would require at least two pairs of the seed data and HMACs -- but in this case you'd be already hacked since the attacker already has valid tokens for your site. Disclaimer: I am not a cryptographer so take this with a pinch of salt.

In practical terms, the only way you could conceivably beat the token is capturing the actual token with a man-in-the-middle attack or hacking the user's device or application / service which uses the token. This is an acceptable risk in the context of remote API access.

I will preemptively answer the inevitable question: why not encrypt the token itself in the database. The short answer is that it's not any more secure than HMAC sums. If anything, a SQL injection may give the attacker enough data to attack the encryption algorithm itself (there are known attacks if you know the message size and have enough ciphertext samples).

And the second inevitable question: why not keep the token itself hashed in the database. It's obvious. Because we need to show the token to the user. Storing a hash means we can never get the raw data back and show it to the user. As to why we can't save the raw data encrypted, see above.

Using a calculated HMAC-SHA256 or HMAC-SHA512 from two pieces of information, one in the database and one in a file, gives us better security than other methods while still allowing us to display the token to the user anytime they request it.

Backwards compatibility

While the code in this PR does not break backwards compatibility it does disable the api-authentication/basic plugin by default. This is intentional. The problem we started out solving was that the api-authentication/basic plugin allowed miscreants to brute force the Super User's password and gain access to the API application, therefore pwning (completely compromising) the site. Please DO NOT revert the SQL code that disables the basic authentication plugin, that would be security suicide.

Translation impact

A total of 4 translation files and 12 unique translation strings was added. Expected translation time impact: approximately 15 minutes.

Documentation Changes Required

Someone needs to tidy up the test instructions and technical notes in a coherent documentation page.

Updates to this PR

Some things changed since I submitted this PR. Noting here so the context of the comments below does make sense:

• Fixed the HTTP header name from Authentication (WRONG!) to Authorization. My bad.
• Removed the _authToken query string parameter. It is not actually necessary. Even Apple's Shortcuts app allows for custom HTTP headers.

[SharkyKZ]

Please DO NOT revert the SQL code that disables the basic authentication plugin, that would be security suicide.

If Basic plugin is so insecure, why not just remove it?

[nikosdion]

@SharkyKZ It's not my decision to remove it, it's something @wilsonge has to decide. I assume that once the token authentication is merged we can revisit that.

[brianteeman]

testing 1. User interface test

Instead of the label being reset how about calling it Create/Reset To me this would be more obvious when creating a token for the first time.
OR
Have two conditional fields
Create (if no token exists)
Reset (if token exists)

Secondly if we add an onchange event then it will be saved automatically - much more obvious to me and removes the need for the lengthy description.

I can supply a pull request to your branch if you agree

[nikosdion]

It's called Reset because it's only used to reset an existing token. It is not to create a token. A token is created automatically the first time you save your user profile without having a token already created.

I am not sure what onchange event you have in mind and I can't see how JavaScript would help when what you need is submit a form which might be in the frontend or the backend or rendered inside a third party user profile management component.

The only other alternative is magically creating a token for everyone editing their user profile. I don't like that approach. Besides, the only people who actually need an API token are power users who should have the presence of mind to read text on the screen and carry out a simple instruction like "save and come back here". If we can't trust our power users to do that let's just delete the entire 4.0 branch and relaunch Mambo 4.0 in its stead.

[brianteeman]

A token is created automatically the first time you save your user profile without having a token already created

I had no way of knowing that :)

[brianteeman]

For security and privacy reasons users can only view their own token, even if they are Super Users.

I agree with this but as it just caught me out then I think there should be an on screen message for this

[brianteeman]

Finally I can not get any further with testing as using curl from the cli I get a response of
{"errors":{"code":500,"title":"Internal server error"}}

And using Bearer with postman I get
{"errors":[{"title":"Forbidden"}]}

[brianteeman]

It's called Reset because it's only used to reset an existing token. It is not to create a token. A token is created automatically the first time you save your user profile without having a token already created.

That absolutely makes sense when you are creating a new user after you have enabled this plugin. The problem is when you add the plugin to an existing site and then open an existing user for editing.

[brianteeman]

When you try to block a user from the user list (or enable a blocked user) you now get the following error
Argument 4 passed to PlgUserToken::onUserAfterSave() must be of the type string, null given, called in C:\htdocs\joomla-cms\libraries\src\Plugin\CMSPlugin.php on line 285

[nikosdion]

That absolutely makes sense when you are creating a new user after you have enabled this plugin. The problem is when you add the plugin to an existing site and then open an existing user for editing.

Yes, I can fix that but it will cost me half a day. Is it worth it?

When you try to block a user from the user list (or enable a blocked user) you now get the following error

Thanks. I'll fix that. The problem is Joomla documents sod all about its events. Following the kinda-sorta documentation of docblocks of existing plugins and transcribing them to PHP 7 parameter types results in these kinds of issues :/

[nikosdion]

I agree with this but as it just caught me out then I think there should be an on screen message for this

That's another half day but, hey, if you think it's worth it.

Finally I can not get any further with testing as using curl from the cli I get a response of

That's probably something unrelated. I cannot reproduce it.

And using Bearer with postman I get

I don't know what postman is. I can only comment on using cURL, wget and phpStorm i.e. two standard, cross-platform CLI tools and the IDE used by the Joomla community developers.

[brianteeman]

I agree with this but as it just caught me out then I think there should be an on screen message for this

That's another half day but, hey, if you think it's worth it.

come on - its about 30 minutes maximum - even for me.

Postman == https://getPostman.org
Its the number one tool for testing and building api such as this

[brianteeman]

oops - typo - its https://getpostman.com

[wilsonge]

@SharkyKZ It's not my decision to remove it, it's something @wilsonge has to decide. I assume that once the token authentication is merged we can revisit that.

I'm happy to bin basic auth. It was something that allowed me to got an auth implementation working in the least amount of time possible. It wasn't to stick around long term if something better came along

I'm busy in the office today. And have a full day tomorrow outside of Joomla so I'll look into code review on Sunday and the full testing instructions you've written up.

Thankyou so much for working on this!

[brianteeman]

When you are the user

When you are not the user

Line12 add
use Joomla\CMS\Language\Text;
Line 74 add
echo "<div class='alert alert-info'>" . Text::_('PLG_USER_TOKEN_HIDDEN_NOT_USER') . "</div>";

And then add the language string

[nikosdion]

See latest commit.

[brianteeman]

Thank you

[nikosdion]

Adding alert classes to an input field is dirty and will probably no longer work in the future. These classes should only really be added to presentation elements, not input elements. What I'm doing is use three note fields, each one being displayed under different conditions. It makes the code much more unreadable but at least you get your correct message each time.

[wilsonge]

@nikosdion https://github.com/nikosdion/joomla-cms/pull/5 here's a PR to fix tests. @rdeutz do you wanna take a look too?

[nikosdion]

I have rebased the PR to the 4.0-dev branch and merged George's PR. 🤞

[wilsonge]

Always helps if i properly port my fixes from my dev branch into the commits :) https://github.com/nikosdion/joomla-cms/pull/6

[wilsonge]

Attempt 3 https://github.com/nikosdion/joomla-cms/pull/7

[wilsonge]

Just to confirm ran another test #28441 and drone passes with this final change

[wilsonge]

Tested this locally. Everything seems to work and as I'm happy I've got the tests working I'm merging this.

Thankyou very much for all your perseverance here @nikosdion !

[nikosdion]

Woohoo! This merge makes me VERY happy. It is a feature I need in my own software.

I was going to release the token management user plugin myself. Now that Joomla 4 has it built in I can limit my plugin's scope to Joomla! 3 and use the native 4.0 functionality. Yay, standardization!