--plugin-secret over-rides existing metadata.json plugin config

[simonw]

This means if you use --plugin-secret at all (with e.g. publish cloudrun) any existing plugin configuration in your metadata.json will be ignored.

datasette/datasette/publish/cloudrun.py

Lines 98 to 109 in af9cd4c

	if plugin_secret:
	extra_metadata["plugins"] = {}
	for plugin_name, plugin_setting, setting_value in plugin_secret:
	environment_variable = (
	"{}_{}".format(plugin_name, plugin_setting)
	.upper()
	.replace("-", "_")
	)
	environment_variables[environment_variable] = setting_value
	extra_metadata["plugins"].setdefault(plugin_name, {})[
	plugin_setting
	] = {"$env": environment_variable}

[simonw]

Actually that code isn't the problem, since extra_metadata has not yet been read from metadata.json at that point.

The bug is here:

datasette/datasette/utils/__init__.py

Lines 362 to 368 in d55fe8c

	if metadata:
	metadata_content = parse_metadata(metadata.read())
	else:
	metadata_content = {}
	for key, value in extra_metadata.items():
	if value:
	metadata_content[key] = value

I need to merge dictionaries in a smarter way.

[simonw]

Lot's of recipes for deep dictionary merge on https://stackoverflow.com/q/7204805/6083

[simonw]

I think I'll use https://github.com/clarketm/mergedeep (MIT license)