Validate common statistics DB schema errors on start

[emontnemery]

Proposed change

Validate and repair common recorder database schema errors of the statistics tables, which are caused by incorrectly restoring or migrating tables between database engines:

• detect text columns not supporting 4-byte UTF-8, e.g. emojis if connected to a MariaDB or MySQL database
• detect timestamp columns not supporting µs precision if connected to a MariaDB, MySQL or PostgreSQL database
• detect float columns not supporting double precision if connected to a MariaDB, MySQL or PostgreSQL database

Tested with the following SQL script on a MariaDB database:

ALTER TABLE statistics_meta CONVERT TO CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci, LOCK=EXCLUSIVE;
ALTER TABLE statistics MODIFY mean FLOAT;
ALTER TABLE statistics MODIFY `start` DATETIME;
ALTER TABLE statistics_short_term MODIFY min FLOAT;
ALTER TABLE statistics_short_term MODIFY `last_reset` DATETIME;

On startup, Home Assistant should report:

2022-10-14 10:47:45.188 WARNING (Recorder) [homeassistant.components.recorder.migration] Database is about to correct DB schema errors: {'statistics.µs precision', 'statistics_short_term.double precision', 'statistics_meta.4-byte UTF-8', 'statistics.double precision', 'statistics_short_term.µs precision'}
2022-10-14 10:47:45.191 INFO (SyncWorker_7) [homeassistant.loader] Loaded ipp from homeassistant.components.ipp
2022-10-14 10:47:45.191 INFO (SyncWorker_3) [homeassistant.loader] Loaded brother from homeassistant.components.brother
2022-10-14 10:47:45.193 INFO (MainThread) [homeassistant.components.zeroconf] Starting Zeroconf broadcast
2022-10-14 10:47:45.194 WARNING (Recorder) [homeassistant.components.recorder.statistics] Updating character set and collation of table statistics_meta to utf8mb4. Note: this can take several minutes on large databases and slow computers. Please be patient!
2022-10-14 10:47:45.216 WARNING (Recorder) [homeassistant.components.recorder.migration] Modifying columns mean, min, max, state, sum in table statistics. Note: this can take several minutes on large databases and slow computers. Please be patient!
2022-10-14 10:47:45.245 WARNING (Recorder) [homeassistant.components.recorder.migration] Modifying columns last_reset, start in table statistics. Note: this can take several minutes on large databases and slow computers. Please be patient!
2022-10-14 10:47:45.273 WARNING (Recorder) [homeassistant.components.recorder.migration] Modifying columns mean, min, max, state, sum in table statistics_short_term. Note: this can take several minutes on large databases and slow computers. Please be patient!
2022-10-14 10:47:45.319 WARNING (Recorder) [homeassistant.components.recorder.migration] Modifying columns last_reset, start in table statistics_short_term. Note: this can take several minutes on large databases and slow computers. Please be patient!

Ather startup, the modified tables and column should be changed back to the correct settings.

Validating the events and states tables will be done in follow-up PRs

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Deprecation (breaking change to happen in the future)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue: fixes #
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• The code has been formatted using Black (black --fast homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
• Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• No score or internal
• 🥈 Silver
• 🥇 Gold
• 🏆 Platinum

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[homeassistant]

Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration (recorder) you are listed as a code owner for? Thanks!

[emontnemery]

Set to draft until tests have been added

[balloob]

Should this be a notification ?

[balloob]

oh I guess the frontend is not loaded yet.

[emontnemery]

We could however create a repair issue