Add infra to skip inapplicable modules

[aciba90]

Proposed Commit Message

meta-schema: add infra to skip inapplicable modules

Add new `activate_by_schema_keys` property to `MetaSchema` defining
the schema keys that must be present, to avoid skipping the module.
Add this new key to our schema docs.
Add documentation to `topics/module_creation`.

Additional Context

SC-1132
Original draft PR: #1515

Test Steps

$ lxc launch ubuntu-daily:jammy jj
$ lxc exec jj -- grep -i 'no applicable' /var/log/cloud-init.log
2022-07-15 15:15:56,292 - modules.py[INFO]: Skipping modules 'snap' because no applicable config is provided.

Checklist:

• My code follows the process laid out in the documentation
• I have updated or added any unit tests accordingly
• I have updated or added any documentation accordingly

[blackboxsw]

@aciba90 looks really good and behaves well. A couple of minor comments, one related to moving the importlib changes out of this PR if we can.

I think there could be a possibility that we may not want to continue to do the initial "skip inapplicable" modules check if cloud-config instance-data hasn't changed. But, that is a minor recurring cost per boot during module discovery. It may be nice at some point if we grew the ability to report module status from cloud-init status --long [--format json] that could report whether the module participated in boot, or was skipped (either due to previously running or due to inactivity/inapplicable config). This is not something for this PR, but future feature question for us to discuss long-term.

medium term, maybe for separate PR as well: the write_files vs write_files_deferred case is potentially interesting. If we have write_files: defered: true in config we could skip running the base cc_write_files. This is the only case like this were specific values of config key indicates whether or not to skip the module. But, I think it's a corner case that may not be worth investing time in.

[blackboxsw]

😄 on doc updates.

[aciba90]

Updated to:

return f"**Activate only if given keys:** {schema_keys}\n\n"

[aciba90]

A couple of minor comments, one related to moving the importlib changes out of this PR if we can.

Extracted to #1605

I think there could be a possibility that we may not want to continue to do the initial "skip inapplicable" modules check if cloud-config instance-data hasn't changed. But, that is a minor recurring cost per boot during module discovery. It may be nice at some point if we grew the ability to report module status from cloud-init status --long [--format json] that could report whether the module participated in boot, or was skipped (either due to previously running or due to inactivity/inapplicable config). This is not something for this PR, but future feature question for us to discuss long-term.

That would be interesting. Added to SC-1192 to not forget about its discussion.

medium term, maybe for separate PR as well: the write_files vs write_files_deferred case is potentially interesting. If we have write_files: defered: true in config we could skip running the base cc_write_files. This is the only case like this were specific values of config key indicates whether or not to skip the module. But, I think it's a corner case that may not be worth investing time in.

If this is the only corner case, I would say that it is not worth to implement a more complex way to express logic within activate_by_schema_keys. I am thinking about something like:

{
    "activate_by_schema_keys": ["write_files.defered == True"]
}

But if this necessity grows, for sure we could do it.

Thanks, @blackboxsw, for the review and constructive feedback!
I think I have covered all the open topics. Please, have a second review.

[blackboxsw]

Looks really good. one minor rendered text change request (which I realize affects your unit tests). Otherwise +1 from me. I'm testing this now

[blackboxsw]

Let's mark up the key names with ``<key_name>`` to align with other doc config key references. It makes readthedocs decorate it in red.

Suggested change

	schema_keys = ", ".join(meta["activate_by_schema_keys"])
	return f"**Activate only if given keys:** {schema_keys}\n\n"
	schema_keys = ", ".join(
	f"``{k}``" for k in meta["activate_by_schema_keys"]
	)
	return f"**Activate only on keys:** {schema_keys}\n\n"

[holmanb]

I think the commit message needs to be updated, but otherwise looks good to me!

[blackboxsw]

Pushed the minor doc change and will land when CI passes

[aciba90]

Thanks, @blackboxsw and @holmanb, for handling and landing this.