Passwords: use better algorithm than md5 hash, use salts and maintain BC

[anonymous-matomo-user]

What I can see from code is that passwords are hashed with md5, but without a salt. That’s not secure enough, take look at here: http://ilia.ws/archives/68-MD5-Dictionary-Attacks.html

See also the request to use different salts for different hashes: #3051

[robocoder]

Might want to consider http://www.openwall.com/phpass (public domain password framework already adopted by other open source projects).

[anonymous-matomo-user]

All you need to do is add a salt before hashing the password (the longer the better), as reported in the summary.

[mattab]

We should have a salt for Password to avoid dictionnary attacks or prevent password leakages on other websites to impact a user login to Piwik.

[robocoder]

Note: salting is a compat buster. We can't salt existing passwords because they're already hashed.

To migrate:

• add a flag column to each user; if true, user must change their password when he/she next logins; regenerate token_auth at that time
• move current Login auth to a migration plugin that hooks into the authenticator; this plugin would be deactivated after everyone has changed their passwords

For more robustness:

• configureable hash() algorihtm (defaults to md5); increase md5password and token_auth size to 128 chars (hexits)
• implement double salt in the new Login auth

[alexlehm]

I would like to add two concerns about md5 unsalted passwords

a hash is resolvable by googling if the password is a plain word, you do not even need a dictionary, e.g. https://www.google.com/search?q=14c4b06b824ec593239362517f538b29

all hashes for passwords up to a given length can be resolved by a rainbow table attack, so a leaked password table can be processed without any brute force operation

(if you need additional arguments, take a look at a few recent pw leaks e.g. on linkedin)

[mattab]

When we implement per user salt, we should also add in

• when user change the password, also invalidate all other currently logged in sessions for this users (ie. change the user's salt?)

[halfdan]

We should use the new Password API from PHP 5.5 for this! At the end of the article there is a link to a compatibility implementation for earlier PHP versions.

https://gist.github.com/3707231

If we agree on something I am keen on implementing this one.

[mattab]

This new API looks great! But it will need to work of course for 5.1.x to 5.4.x as well, also we need to plan if "crypt" extension is not available (prior to 5.3 it seems crypt might not be available always)

[alexlehm]

I cannot believe that this is still buried as a low priority ticket and has been ignored for 6 years.

md5 password storage is completely insufficient, this poses a significant security risk.

md5 passwords with salt are also insufficient since md5 cannot be considered secure at all anymore.

To achieve an acceptable level of security, it is necessary to use a newer hash method like bcrypt, scrypt or another key derivation function, most previous pw methods are too weak and have been for some time.

Sacrificing this with the argument of backward compatibility is criminally stupid, sorry.

[robocoder]

@alexlehm In future, make your comments constructive. The challenge is migrating existing users.

[alexlehm]

I'm sorry if that is non-constructive, however the current pw scheme is such a major issue that this must be addressed sooner rather than later.

Serendipity used a rather smart method to migrate user pws to a new scheme a while back where they defined a cut-off date 180 days after the migration was activated (based on the installation, not on the release of the software) so that all active users migrated themselves without even knowing and inactive users were locked out after the grace period.

E.g. migrating the admin user would work like, since the user will log in anyway.
I am not sure how the auth mechanisms work internally, but for anything that is authenticating on a client API, an API hash would be better then using a real password.

[mattab]

Increasing priority... Yes it's an important ticket, but as you can see, there are a lot of important tickets to work on, the challlenge is that we have a very small team... Are you familiar with PHP? If you are interested and familiar with php, please consider joining the project or helping with this!

[alexlehm]

Actually I could do that, have to read up on the piwik source code first, even though I am using this for quite a while I have never looked at anything except the config file.

[halfdan]

@alexlehm It would be nice if you'd join up for sure :)

@mattab We discussed this a few days ago - I'd very much like to do this with the new PHP 5.5 password API (+ the 5.3 fallback).

The way this could work is that for all installations with PHP >= 5.3.7 we automatically update the passwords to bcrypt once they log in (the 5.3 fallback only works with >= 5.3.7 since there were issues in earlier PHP versions). This would leave quite a number of people out of the loop, but if they cared about security they wouldn't be running 5.3.x in the first place :)

Cheers!

[mattab]

Another challenge: maintain backward compatibility for token_auth

• Since token = md5($userLogin . $md5Password) (see UsersManager\API) then changing password will change the token. this may cause some problem and break some tools/apps using the previous token...
• we have to clarify all places where md5password is used, this will break some usersManager APIs. Can we keep BC in these APIs by keeping the md5Password parameter?

[alexlehm]

using the hashed password to create the token without any additional data is not so good, this makes it impossible to change the token without changing the password as well, it should be possible to invalidate the api tokens by changing the password to the same value (or to create a new token by clicking a button without affecting the user account at all).

Have to look into this a bit further, it should be possible to keep both but disable the old functionality when the user chooses so.

[halfdan]

I agree, the token should be decoupled from the password completely. We could simply generate a unique string with some random data.

It would be really cool to invalidate the token without having to change the password, so I'll be looking into that.

[mattab]

Things to think about:

• logme mechanism http://piwik.org/faq/how-to/faq_30/ - should not use md5 anymore but the more ecure hashing (via API?)
  • how do we keep BC for logme
• how do keep backward compatibility in general
• update tests
• Log analytics http://piwik.org/log-analytics/ import_logs.py https://github.com/piwik/piwik/blob/master/misc/log-analytics/import_logs.py#L574
• Other modules, see: https://github.com/piwik/piwik/search?l=php&q=md5*password&utf8=%E2%9C%93

[etjossem]

Please try to auto migrate users to bcrypt, etc. Unsalted MD5 is kind of a scary vulnerability.

See: #8753

[ircmaxell]

For improved security it would be really good to have the API tokens hashed. Just in case for some reasons someone gets access to DB etc. Of course they could then also just overwrite the token with a new one but it would be still good security to have it hashed. There are probably solutions for this but I haven't checked yet. Usually authentications consist of a key and a secret which makes it easier as only having the secret.

For this use-case, a simple SHA-256 on DB-insert should suffice.

[tsteur]

For this use-case, a simple SHA-256 on DB-insert should suffice.

That's a good point 👍

[J0WI]

It would be great to have the plain md5 hashes mitigated during the update even if the users will not log in for a while.

We could hash the current md5 hashes a second time with a stronger algorithm and with salt and pepper. These hashes can be flagged (so the login script knows that it need to hash them twice to verify) and updated during the next successful log in. If the hashes become public for any reason, we have no plain md5 hashes in DB.

The md5 hashes are the main issue here. So if the above solution won't be implemented, I would go with forcing a password reset for every user.
History showed that plain md5 password hashes are always evil, even for inactive users or hosts.

Edit:
If something similar is done with the API tokens, it might be possible to be backwards compatible. But for security reasons this should be behind a pref and removed completely in future updates.

[mneudert]

Is there any feature that could not be modified to use the API proxy defined in ee3bc9c? Or at least not short-term modified?

A search for token_auth usage find results like the DocumentationGenerator (example URLs for RSS feeds if I get it right?) or UserCountryMapController.

Hashing stuff is always good but if a feature breaks it might be a deal breaker.

If these are "user facing" features for "URLs without login" we could go with hashing and update the messages/documentation to point at the token management. Users without any tokens available would get displayed a nice message to create a token. And going back to a string length check we could have the legacy token only available for existing URLs or "people who know" while not including it in the check for potentially available tokens.

[tsteur]

We cannot use the solution in ee3bc9c as it is missing an nonce to prevent CSRF attacks. We would need to pass along an nonce token or a token/access key that is only valid for limited time but ideally an nonce. We also currently use the token_auth in API requests from JavaScript. This is pretty much the same case and here we would need to use something like nonce as well.

I think the most critical things for now are:

1. Migrate password via password_hash on login
2. Generate token auth randomly and not based on login / password

This will already be a great improvement.

Next step that will be needed eventually is to hash the token_auth which we will have to do for sure. If not in 3.0, then in 3.1 as it shouldn't break anything (I reckon).

It would be great to focus for now on the first two things. Once this is done we could do another evaluation on the steps.

[tsteur]

@mneudert or @sgiehl will you work on this?

[sgiehl]

@mneudert is working on that. See 3.x-dev...mneudert:password_hash

[J0WI]

How do you think about my proposal in #5728 (comment) to migrate plain md5 hashes during the update procedure and not only on login?

[tsteur]

be worth having a look into it for sure. Would probably only need to check how to flag it.

[mneudert]

Currently I plan to rehash every password with SHA256 during the upgrade combined with setting a "_legacy_password"-Option stored for each user. After a first valid login this option will get deleted and a proper rehashing will update the password.

Having all passwords rehashed also nicely shows where the password/token_auth is in use with all the tests breaking :D

[HybridAU]

I might be misunderstanding the process you are using here, and for what it's worth I really appreciate you doing this work so I don't want this to come across as just complaints from the peanut gallery.

But I think if your going to go through all the effort of changing the password storage method you should move to something like PBKDF2 or bcrypt. They are designed for password storage and have things like a salt and configurable iterations built in.

Rather than moving to SHA256 which while better than MD5 still shares many of it's faults (e.g. With out a salt two passwords which are the same will have the same hash and so rainbow tables can be made. Also there is no easy way to increase the number of required iterations and so the time to create a hash which is already quite low, will only ever get lower.)

[tsteur]

I'm not quite sure but I think the SHA256 is only for auth tokens and not for end user passwords see discussion further above. end user passwords should be using http://us2.php.net/manual/en/function.password-hash.php (bcrypt)

[mneudert]

Yeah, that's right. I think however that needs some deeper refactorings for example regarding the tests matching password hashes in their XML fixtures. Plan is to change SHA256 with bcrypt for old password once new password are hashed that way. Then everything should be in place to handle that switch a lot easier than right of the bat.

[mattab]

Hi @mneudert @sgiehl - what is the status for this security improvement? if possible, we would love to include it in the Piwik 3.0.0-beta2 or beta3 in coming days. We've just released the first public beta of Piwik 3.0.0: https://piwik.org/changelog/piwik-3-0-0-beta1/

[mneudert]

Currently I have switched the password hashing to bcrypt (re-hashing of old passwords yet to be added) and worked on the token separation/extraction. If it helps I could open a WIP pull request for that part (with a list of caveats attached as some things might seem somewhat "quirky" otherwise).

There are still some things to sort out like a proper handling of md5 hashed passwords used in API requests or what should happen if a user has no API token (last one deleted or none ever created).

[tsteur]

PR with caveats sounds good 👍

For now we wouldn't need the feature to create multiple tokens etc and be good to only migrate existing tokens (each user has one token) but if you're keen on implementing this as well with multi tokens 👍 . Maybe it would be possible to split it into 2 PRs as just making passwords and token more secure by hashing them could be much more simple compared to managing multiple tokens maybe. Ideally the UI would not rely on using API tokens and only use nonce or generated short-term tokens that are always different per session or so for better security. The UI should eventually not use an API token and therefore, when a user deletes all tokens the API would be basically not usable which is a good feature.

Main feature would be: On login bcrypt password if not done yet, on update hash all tokens with SHA256 and generate the tokens in the future more randomly. Those three things would be main improvement for now. Everything else, like app-specific tokens is great to have :)

[mattab]

Awesome work @mneudert @tsteur - this important security improvement merged in #10926 🎉

[mlissner]

I've only skimmed this thread, but I don't see anywhere that we're keeping track of the bcrypt configuration parameters. Ideally when we upgrade people's existing passwords, we will save all of the following:

• Algorithm
• Iterations
• Salt
• Hashed password

For example, this is the value that Django stores for every user. It's a $ delimited field in the database:

pbkdf2_sha256$20000$random-salt-here$hashed-password-here

This says:

The password was hashed using 20000 rotations of the pdkdf2 sha256 algorithm using the salt random-salt-here and resulted in the hash hashed-password-here.

Apologies if this is all being taken care of already (I didn't check the code), but we need to store ALL of this so that we can later upgrade the algorithm, the number of rotations, etc. For example, Django regularly upgrades the number of rotations as computer hardware and software become faster. If we don't do this now, we'll have another headache down the road when we need to upgrade things again.

Here's Django's documentation. It's very good.

[tsteur]

The algorithm, cost and salt etc is stored in the password see http://us2.php.net/manual/en/function.password-hash.php

[mlissner]

Fantastic! I'll go on my now. Apologies for the drive by. I'm very happy to see this improvement.