App specific token_auths

[tsteur]

fix #6559

• We now store token_auth no longer in the user table but hashed in a new table user_token_auth
• To not needing to rewrite the entire app re piwik.token_auth basically now storing in the session an extra token auth which is randomly generated when a user logs in. This random token is basically just like an nonce and because we need to store it in the session, it is stored in plain text in the database (base64 encoded in the session table). The plain text storage is not an issue though since it is not really a token auth but more an nonce and it only works if you also have the sessionID. And if you have the sessionId, you are authenticated anyway
• Also created Store session ID hashed in the DB #15390 so sessionIDs are hashed as well.
• Also removes login and the weakly hashed token_auth from the cookie.
• To not break BC for existing token_auths we migrate them automatically.
• Token auths still have same permission and access as the actual user. Created Provide ability to restrict auth tokens to site, access, scope #15368 which may be interesting in the future.
• We differentiate between system tokens and regular user tokens.
  • User auth token is a token created by the user and a user can delete them etc.
  • A system auth token is created by the system and not visible to the user. It's used in the background when needed. This tokens have an expiry date and are invalidated/deleted typically within a few hours or days. They were needed because eg when we archive using http (when CliMulti cannot be used), then we need to issue an HTTP request from CoreArchive and this request has to be authenticated. In this case CoreArchive generates a token auth that is valid for say 2 days and it can use this generated token to authenticate the archiving requests.
• We store the tokens hashed as sha512. Was going to use sha3-512 but so far sha512 seems more "proven". There is a column in the table storing the algorithm for future migrations.
• Introduces a new page "Security" where these tokens can be managed plus the password and 2FA.

Todo before release

• We'll need to adjust the mobile app before the release to support this new flow of accessing tokens and also to let them enter a token directly.
• We'll need to adjust some docs/faqs as we now only show the token after creation only.

[tsteur]

@diosmosis applied the feedback now eg no longer removing the token_auth column. Also fixed the merge issues with 4.X and updated tests

[diosmosis]

@tsteur looks like omnifixture might need an update? The UI tests are failing but I'm not sure why...

[sgiehl]

@diosmosis updating omnifixture will likely change nearly all screenshots, due to various changes in tracked visits. Planning to do that in #15520. Will try to finish that somewhen this week. Might make more sense to update the omnifixture here afterwards to see if any test fails due to some changes here...

[diosmosis]

@sgiehl makes sense, there was an odd error for this build but it seems unrelated to omnifixture (unless I forgot about another test). I'm in the process of fixing it.

[tsteur]

Thanks for your help on this @diosmosis 💯

[sgiehl]

@diosmosis @tsteur I'm getting an error now in the browser when the update is triggered (after checking out 4.x-dev branch). Guess the problem might be, that I'm not logged in. Doing a core:update on command line works

ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] Uncaught exception: /srv/matomo/libs/Zend/Db/Statement/Pdo.php(234): SQLSTATE[42S02]: Base table or view not found: 1146 Table 'matomo.matomo_user_token_auth' doesn't exist
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] SQLSTATE[42S02]: Base table or view not found: 1146 Table 'matomo.matomo_user_token_auth' doesn't exist
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #0 /srv/matomo/libs/Zend/Db/Statement.php(300): Zend_Db_Statement_Pdo->_execute(Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #1 /srv/matomo/libs/Zend/Db/Adapter/Abstract.php(479): Zend_Db_Statement->execute(Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #2 /srv/matomo/libs/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('SELECT * FROM m...', Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #3 /srv/matomo/core/Db/Adapter/Pdo/Mysql.php(309): Zend_Db_Adapter_Pdo_Abstract->query('SELECT * FROM m...', Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #4 /srv/matomo/libs/Zend/Db/Adapter/Abstract.php(755): Piwik\Db\Adapter\Pdo\Mysql->query('SELECT * FROM m...', Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #5 /srv/matomo/plugins/UsersManager/Model.php(334): Zend_Db_Adapter_Abstract->fetchRow('SELECT * FROM m...', Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #6 /srv/matomo/plugins/UsersManager/Model.php(419): Piwik\Plugins\UsersManager\Model->getTokenByTokenAuthIfNotExpired('299b4455fd7fe3b...')
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #7 /srv/matomo/plugins/Login/Auth.php(104): Piwik\Plugins\UsersManager\Model->getUserByTokenAuth('anonymous')
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #8 /srv/matomo/plugins/Login/Auth.php(62): Piwik\Plugins\Login\Auth->authenticateWithLoginAndToken('anonymous', 'anonymous')
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #9 /srv/matomo/core/Access.php(183): Piwik\Plugins\Login\Auth->authenticate()
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #10 /srv/matomo/core/FrontController.php(405): Piwik\Access->reloadAccess(Object(Piwik\Plugins\Login\Auth))
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #11 /srv/matomo/core/dispatch.php(33): Piwik\FrontController->init()
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #12 /srv/matomo/index.php(27): require_once('/srv/matomo/cor...')
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #13 {main},
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] caused by: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'matomo.matomo_user_token_auth' doesn't exist
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #0 /srv/matomo/libs/Zend/Db/Statement/Pdo.php(228): PDOStatement->execute(Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #1 /srv/matomo/libs/Zend/Db/Statement.php(300): Zend_Db_Statement_Pdo->_execute(Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #2 /srv/matomo/libs/Zend/Db/Adapter/Abstract.php(479): Zend_Db_Statement->execute(Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #3 /srv/matomo/libs/Zend/Db/Adapter/Pdo/Abstract.php(238): Zend_Db_Adapter_Abstract->query('SELECT * FROM m...', Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #4 /srv/matomo/core/Db/Adapter/Pdo/Mysql.php(309): Zend_Db_Adapter_Pdo_Abstract->query('SELECT * FROM m...', Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #5 /srv/matomo/libs/Zend/Db/Adapter/Abstract.php(755): Piwik\Db\Adapter\Pdo\Mysql->query('SELECT * FROM m...', Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #6 /srv/matomo/plugins/UsersManager/Model.php(334): Zend_Db_Adapter_Abstract->fetchRow('SELECT * FROM m...', Array)
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #7 /srv/matomo/plugins/UsersManager/Model.php(419): Piwik\Plugins\UsersManager\Model->getTokenByTokenAuthIfNotExpired('299b4455fd7fe3b...')
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #8 /srv/matomo/plugins/Login/Auth.php(104): Piwik\Plugins\UsersManager\Model->getUserByTokenAuth('anonymous')
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #9 /srv/matomo/plugins/Login/Auth.php(62): Piwik\Plugins\Login\Auth->authenticateWithLoginAndToken('anonymous', 'anonymous')
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #10 /srv/matomo/core/Access.php(183): Piwik\Plugins\Login\Auth->authenticate()
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #11 /srv/matomo/core/FrontController.php(405): Piwik\Access->reloadAccess(Object(Piwik\Plugins\Login\Auth))
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #12 /srv/matomo/core/dispatch.php(33): Piwik\FrontController->init()
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #13 /srv/matomo/index.php(27): require_once('/srv/matomo/cor...')
ERROR Piwik\ExceptionHandler[2020-03-18 08:42:37 UTC] [7964a] #14 {main}