Prevent config fields being assigned to None in Generic Camera options

[davet2001]

Proposed change

#75265 Adjusting the config for generic camera using the options flow resulted in some config fields being assigned to None in the stored config. These values are used as defaults when the user tries to configure a second time, but are rejected by the config validator. This PR prevents the problem and cleans up existing configs.

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Deprecation (breaking change to happen in the future)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Additional information

During the options flow, a merge operation happens between the existing config and what the user has entered. The issue is that this merge uses the get() operation. If a field doesn't exist (ok for optional fields), the get() operation returns None. This then creates a field assigned to None, e.g.

data = {
    ...
    CONF_STILL_IMAGE_URL: user_input.get(CONF_STILL_IMAGE_URL),
    ...
}

Actually this causes no problem initially because the None data is ignored by the integration base code. The problem is that this None sits in the stored config and is put back to the UI on the second attempt. For the fields defined as optional with a data type of str, this is invalid (the field can be omitted or str, but not None). So the user sees an error message. And there's not a lot they can do about it.

This PR fixes the problem in two steps:

1. Fixes the merging operation so that None fields aren't assigned anymore.
2. Adds a config migrator to get rid of None values on startup.
   Tests are added to ensure that the new code works.

• This PR fixes or closes issue: fixes Cannot Reconfigure Generic Camera with No Still Image #75265
• This PR is related to issue:
• Link to documentation pull request:

Checklist

• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• The code has been formatted using Black (black --fast homeassistant tests)
• Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
• Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

• No score or internal
• 🥈 Silver
• 🥇 Gold
• 🏆 Platinum

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[github-actions]

There hasn't been any activity on this pull request recently. This pull request has been automatically marked as stale because of that and will be closed if no further activity occurs within 7 days.
Thank you for your contributions.

[EnochPrime]

...

[davet2001]

Yes, @stalebot this is still valid and still wanted.

[emontnemery]

Instead of the migration, which prevents reverting to an older HA core version, can't the problem be handled in the build_schema so it doesn't use None as suggested_value?

         vol.Optional(
             CONF_STILL_IMAGE_URL,
-            description={"suggested_value": user_input.get(CONF_STILL_IMAGE_URL, "")},
+            description={"suggested_value": user_input.get(CONF_STILL_IMAGE_URL) or ""},
         ): str,

[davet2001]

Converting to draft as currently unable to reproduce the original problem.

[github-actions]

There hasn't been any activity on this pull request recently. This pull request has been automatically marked as stale because of that and will be closed if no further activity occurs within 7 days.
Thank you for your contributions.

[davet2001]

Closed and replaced with smaller PR: #86256