[4.0] TLS encryption for database connections: Corrections and enhancements

[richard67]

Pull Request for #26888 (comment) and review comments in PR's #26888 and #27348 .

Replaces PR #27320 .

Summary of Changes

• Undo PR [4.0] No TLS transfer encryption for database connections if localhost #26889 so the database connection encryption feature is not completely hidden anymore for sites using "localhost" as database host.
• Correct showon condition for "Path to CA File" field like it was meant to be done by PR [4.0] TLS encryption for database connections: Correct language texts and showon for fields #27320 .
• Change language texts "One-way encryption" and "Two-way encryption" for dropdown options to "One-way authentication" and "Two-way authentication" like it was meant to be done by PR [4.0] TLS encryption for database connections: Correct language texts and showon for fields #27320 .
• Remove nearly useless field "Path to CA Folder" to simplify UI as discussed and agreed in PR [4.0] Add TLS transfer encryption for database connections at installation - 2nd version #27348 for new installation.
• Change names of some language texts so they are less misleading.
• Don't use JS anymore to clear unused fields but add new database connection parameter validation function to the configuration model - this can be extended by further validations for the scenario that someone moves database and data with export and import to a new server and wants to check the new connection in backend, and for a validation in case of remote databases like we have it in J3 at least for new installations (see issue [4.0] Installer security checks are broken #18736 ).

With this PR here applied, UI and validation of fields in global configuration which are related to database connection encryption will work in the same way as they will work for new installations when PR #27320 is applied.

Testing Instructions

Pre-conditions

Enable database connection encryption with TLS (in some docs often called "ssl") on your database server.

If you have a standard installation of MySQL 5.7 or later or comparable MariaDB server, or a PostgreSQL 10 or later, the database server comes with a self signed server certificate and enabled encryption support, so one-way (i.e. server side) authentication without certificate verification works out of the box. On PostgreSQL it just might be necessary to enable ssl in the postgresql server configuration file by uncommenting the corresponding line and then restart the server to make one-way authentication without certificate verification work.

For a webspaces with databases provided by a hosting provider it may be different, especially with shared hosting. But on my shared hosting space one-way authentication without certificate verification works, too.

For certificate verification it is required that the common name (CN) in the server certificate is equal to the server name used by the DB client. This is not the case with the default certificates used by the standard database setup mentioned above.

Some links to documentation of connection encryption for the particular kinds of databases:

• MySQL 8.0 https://dev.mysql.com/doc/refman/8.0/en/encrypted-connections.html
• MariaDB https://mariadb.com/kb/en/library/data-in-transit-encryption/ and https://mariadb.com/kb/en/library/securing-connections-for-client-and-server/
• PostgreSQL 11 https://www.postgresql.org/docs/11/ssl-tcp.html

But for testing this PR it is enough to have one-way encryption without certificate verification working.

Instructions

Summary: Same tests as in PR #27348 for new installation except that here we test Global Configuration, i.e. we click "Save" or "Save and close" button instead of button to proceed with installation.

1. Apply the patch of this PR either on a clean (i.e. no other patches applied, no exotic extensions installed, ...) installation of current 4.0-dev branch or latest nightly build.

2. Login to administration (backend) and go to the "Database" section on the "Server" tab of Global Configuration.

3. Check the available values in drop-down "Connection Encryption".
   Result: 3 possible values "Default (server controlled)", "One-way authentication" and "Two-way authentication".

4. Change value of drow-down "Connection Encryption" to "Default (server controlled)".
   Result: No further options related to connection encryption are shown.

5. Change the value of drop-down "Connection Encryption" to "One-way authentication".
   Result: Below the drop-down a switcher "Verify Server Certificate" is shown.

6. Toggle the value of the "Verify Server Certificate" switcher.
   Result: If the switcher is off, no more options for connection encryption are shown below the switcher.
   If the switcher is on, a text field "Path to CA File" is shown below the switcher.

7. Switch "Verify Server Certificate" to "No" and change the value of drop-down "Connection Encryption" to "Two-way authentication".
   Then toggle toggle the value of the database type drop-down check which fields are shown below the "Verify Server Certificate" switcher.
   Result:

• MySQLi and MySQL (PDO): Fields "Path to Private Key File", "Path to Certificate File" and "Supported Cipher Stuite (optional)" are shown.
• PostgreSQL (PDO): Only fields "Path to Private Key File" and "Path to Certificate File" are shown.

8. Switch "Verify Server Certificate" to "Yes". Then toggle toggle the value of the database type drop-down at the top and check which fields are shown below the "Verify Server Certificate" switcher.
   Result:

• MySQLi and MySQL (PDO): Fields "Path to Private Key File", "Path to Certificate File", "Path to CA File" and "Supported Cipher Stuite (optional)" are shown.
• PostgreSQL (PDO): Only fields "Path to Private Key File", "Path to Certificate File" and "Path to CA File" are shown.

9. Fill in some text field and then change an option so the text field is not shown anymore, and then change the option back. E.g. swich server certificate verification on and fill in the CA file field and then toggle server certificate verification off and on again, or do the same with text fields for 2-way authentication and toggle the value of drop-down "Connection Encryption".
   Result: Text fields which are not used for a particular combination of options are not cleared when the options are changed. Unneccessary text fields will later be cleared on form validation instead.

10. Change the value of drop-down "Connection Encryption" to "One-way authentication" or "Two-way authentication", select correct database type, enter "localhost" as host name, enter username, password and database name and if neccessary table prefix and click either the "Save" or the "Save & Close" button at the top.
    Result: The form isn't closed and changes aren't saved. An alert at the top shows an error about using "localhost" and database connection encryption.
    

11. Change the host name to the real hostname for your installation, using either "127.0.0.1" or "::1" or a real host name instead of "localhost".

12. Change the value of drop-down "Connection Encryption" to "Two-way authentication", switch "Verify Server Certificate" to "No" and make sure that field "Path to Private Key File" is empty. Then click either the "Save" or the "Save & Close" button at the top.
    Result: The form isn't closed and changes aren't saved. An alert at the top shows an error about "Path to Private Key File" being empty or not a valid path.

13. Change the value of field "Path to Private Key File" so it contains invalid characters for a path, e.g. colons. Then click either the "Save" or the "Save & Close" button at the top.
    Result: See step 12.

14. Change the value of field "Path to Private Key File" so it contains a valid path which does not exist or is not accessible. Then click either the "Save" or the "Save & Close" button at the top.
    Result: The form isn't closed and changes aren't saved. An alert at the top shows an error about the file in field "Path to Private Key File" not existing or not being accesible.

15. Change the value of field "Path to Private Key File" so it contains a valid path which exists and is accesible, and make sure that field "Path to Certificate File" is empty. Then click either the "Save" or the "Save & Close" button at the top.
    Result: The form isn't closed and changes aren't saved. An alert at the top shows an error about "Path to Certificate File" being empty or not a valid path.

16. Change the value of field "Path to Certificate File" so it contains invalid characters for a path, e.g. colons. Then click either the "Save" or the "Save & Close" button at the top.
    Result: See step 15.

17. Change the value of field "Path to Certificate File" so it contains a valid path which does not exist or is not accessible. Then click either the "Save" or the "Save & Close" button at the top.
    Result: The form isn't closed and changes aren't saved. An alert at the top shows an error about the file in field "Path to Certificate File" not existing or not being accesible.

18. Change the value of field "Path to Private Key File" so it contains a valid path which exists and is accesible, switch "Verify Server Certificate" to "Yes" and make sure that field "Path to CA File" is empty. Then click either the "Save" or the "Save & Close" button at the top.
    Result: The form isn't closed and changes aren't saved. An alert at the top shows an error about field "Path to CA File" being empty or not a valid path.

19. Change the value of field "Path to CA File" so it contains invalid characters for a path, e.g. colons. Then click either the "Save" or the "Save & Close" button at the top.
    Result: See step 18.

20. Change the value of field "Path to CA File" so it contains a valid path which does not exist or is not accessible. Then click either the "Save" or the "Save & Close" button at the top.
    Result: The form isn't closed and changes aren't saved. An alert at the top shows an error about the file in field "Path to CA File" not existing or not being accesible.

21. Change the value of field "Path to CA File" so it contains a valid path which exists and is accessible but not a valid certificate, e.g. a text file, so that database connection should fail. Then click either the "Save" or the "Save & Close" button at the top.
    Result: The result depends on server configuration and configuration of the database user (if allowed to connect unencrypted) in case of errors when connecting.
    In case of connection error, the result depends on the database driver.

• MySQLi: It seems the error message is lost or missing. This could be an issue in the database driver.
  
• MySQL (PDO): The error message is cryptic, but a search in Google will tell what it means.
  
• PostgreSQL (PDO): The error message contains information in English words about the reason for the error.
  
  If due to some strange configuration the connection is successful but not using encryption, a validation check added by this PR will fail.
  
  If the server doesn't support connection encryption, or support is switched off on the server:
  
  Which of these error messages you will see in this test step depends on your server and database user and client configuration. But validation should fail in any case.

22. Select correct database type, enter a valid host name (not "localhost") or "127.0.0.1" or "::1" as host name, enter username, password and database name and if neccessary table prefix and adjust connection encryption parameters to something which should work, e.g. one-way authentication without server certificate verification, which should work out of the box in most cases.
    Then click either the "Save" or the "Save & Close" button at the top.
    Result: Changes are saved.

23. Verify in configuration.php that the adjusted options have been saved and unneeded options have been cleared (empty string).

Expected result

UI and validation of global configuration parameters related to database connection encryption works in the same way as PR #27348 proposes it for new installation.

Actual result

• Field "Path to CA File" is only shown if "Two-way encryption" is chosen for field "Connection Encryption", regardless of the value of field "Verify Server Certificate". This is wrong. The CA file is used only for server certificate verification.
• The texts "One-way encryption" and "Two-way encryption" for the drop-down values are misleading because the connection will be encrypted with both of these values, it is authentication which is done either one or two way.
• If host name of database server is "localhost", the database connection encryption options and so the functionality are completely hidden from the user. The reason for this is that with "localhost" it might not work because a socket connection will be used. But there are better ways to solve this than to hide the complete functionality.
• If options are changed, not all unused fields are cleared, e.g. when changing from two-way to one-way authentication.
• There is no validation for the paths to files for key and certificates.
• Filter for fields with paths to files for key and certificates was "text" and not "path", so it was possible to use invalid characters.
• There is no validation check after database connection if the connection really is encrypted, so if the server allows to silently fall back to unencrypted connection also in case of wrong parameters there is no way to see that some parameters are wrong if there is no connection failure.

Additional information

The validation of database connection parameters added with this PR here should be extended in a future PR by further validation checks in case if database host name or database name or similar fields change. Ths could also include the special security check when using a remote database host as mentioned in issue #18736 for new installations.

Documentation Changes Required

Same as PR #26375 : New feature has to be documented. It would need to extend documentation of the server section of Global Configuration, the Privacy Dashboard Module and the System Information view.

[richard67]

Thanks @HLeithner .

[HLeithner]

Tests with Postgresql Server 12.1:

Two-way with correct client certificate no server verify

Test Ok ✔️

Two-way with correct client certificate wrong server verify path expected error.

Test Ok ✔️

The file entered in field "Path to CA File" does not exist or is not accessible.

Two-way with correct server and client certificate

Test Ok ✔️

Two-way with wrong certificate no server verify expected error.

Test Ok ✔️

Database connection test failed with the following error: 7: Could not connect to PDO: SQLSTATE[08006] [7] SSL error: sslv3 alert illegal parameter
Database connection settings changes were not saved.

Server controlled, but server requires certificate for user. expected error.

Test Ok ✔️

Database connection test failed with the following error: 7: Could not connect to PDO: SQLSTATE[08006] [7] FATAL: connection requires a valid client certificate FATAL: no pg_hba.conf entry for host "", user "user", database "database", SSL off
Database connection settings changes were not saved.

One-way encryption with wrong invalid ca. expected error.

Test Ok ✔️

Database connection test failed with the following error: 7: Could not connect to PDO: SQLSTATE[08006] [7] SSL error: certificate verify failed
Database connection settings changes were not saved.

Tested with MariaDB 10.3 and mysqli driver

Two-way with correct client certificate no server verify

Test Ok ✔️

Two-way with correct client certificate wrong server verify path expected error.

Test Ok ✔️

The file entered in field "Path to CA File" does not exist or is not accessible.

Two-way with correct server and client certificate

Test Ok ✔️

Two-way with wrong certificate no server verify expected error.

Test Ok ✔️

Database connection test failed with the following error: 2002: Could not connect to database:
Database connection settings changes were not saved.

Server controlled, but server requires certificate for user. expected error.

Test Ok ✔️

Database connection test failed with the following error: 1045: Could not connect to database: Access denied for user 'user'@'hostname' (using password: YES)
Database connection settings changes were not saved.

One-way encryption with wrong invalid ca. expected error.

Test Ok ✔️

Database connection test failed with the following error: 2002: Could not connect to database:
Database connection settings changes were not saved.

Summary

All tests successful.

[HLeithner]

I have tested this item ✅ successfully on 68a87d5

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/27351.}

[richard67]

@brianteeman I'd be happy if you could find some time for testing this.

[wilsonge]

Thanks!

[richard67]

Thanks.