Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "parameterize" setting generator #637

Merged
merged 1 commit into from
Feb 1, 2020
Merged

Add "parameterize" setting generator #637

merged 1 commit into from
Feb 1, 2020

Conversation

tjkirch
Copy link
Contributor

@tjkirch tjkirch commented Jan 10, 2020

This lets settings be generated based on a template and parameters that
reference other settings. The first example is regional container image URIs,
where we replace part with the value of settings.aws.region.

The template and parameters are stored as new metadata keys, next to the
setting-generator metadata, which parameterize will read from the API. For
example, if the template is "foo-BAR" and the parameters are '{"BAR":
"settings.bar"}' and "settings.bar" is set to "baz", then the setting output
will be "foo-baz".

To be able to reference other settings from sundog-spawned generators, we need
to commit settings that have already been patched, e.g. from user data. To do
this, this commit eliminates the settings-committer service, instead using the
settings-committer binary as an ExecStartPre on any service that requires
committed settings.


Builds on #636.

Testing done:

Tested with the following PR that enables this feature for container image URIs.

Instances in us-west-2 and us-east-1 were both healthy, pods ran OK, and I saw no errors in the logs.

In both regions, parameterize correctly used the region to build image URIs and they launched OK.

us-east-1:

[ec2-user@ip-192-168-33-74 ~]$ apiclient -u /settings
{
...
  "host-containers": {
    "admin": {
      "source": "328549459982.dkr.ecr.us-east-1.amazonaws.com/thar-admin:v0.2",
      "enabled": true,
      "superpowered": true
    },
    "control": {
      "source": "328549459982.dkr.ecr.us-east-1.amazonaws.com/thar-control:v0.2",
      "enabled": true,
      "superpowered": false
    }
  },
...
  "aws": {
    "region": "us-east-1"
  }
}

us-west-2:

[ec2-user@ip-192-168-109-243 ~]$ apiclient -u /settings
{
...
  "host-containers": {
    "admin": {
      "source": "328549459982.dkr.ecr.us-west-2.amazonaws.com/thar-admin:v0.2",
      "enabled": true,
      "superpowered": true
    },
    "control": {
      "source": "328549459982.dkr.ecr.us-west-2.amazonaws.com/thar-control:v0.2",
      "enabled": true,
      "superpowered": false
    }
  },
...
  "aws": {
    "region": "us-west-2"
  }
}

Copy link
Contributor

@zmrow zmrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗝

workspaces/api/parameterize/README.md Outdated Show resolved Hide resolved
@tjkirch
Copy link
Contributor Author

tjkirch commented Jan 13, 2020

This push fixes the typo caught by zmrow.

Copy link
Contributor

@zmrow zmrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

☃️

@tjkirch tjkirch requested a review from bcressey January 21, 2020 22:31
packages/workspaces/workspaces.spec Outdated Show resolved Hide resolved
workspaces/api/settings-committer/src/main.rs Outdated Show resolved Hide resolved
workspaces/api/parameterize/src/main.rs Outdated Show resolved Hide resolved
@tjkirch
Copy link
Contributor Author

tjkirch commented Jan 23, 2020

This push just rebases on the changes in #636.

@tjkirch
Copy link
Contributor Author

tjkirch commented Jan 23, 2020

This push updates the settings-committer docstring I missed, as caught by bcressey.

@tjkirch
Copy link
Contributor Author

tjkirch commented Jan 28, 2020

This push addresses #637 (comment) by moving to handlebars-based templating inside the template metadata key, rather than using substring replacement from the (now-removed) "parameters" metadata key. It also addresses his naming request by renaming the generator to "schnauzer" :P

Since this is nearly identical to what thar-be-settings did, I moved the common functions into schnauzer's library, which t-b-s and the schnauzer binary now use.

Testing still healthy, as in description.

Copy link
Contributor

@zmrow zmrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

🏋

workspaces/api/schnauzer/README.md Show resolved Hide resolved
workspaces/api/schnauzer/src/lib.rs Show resolved Hide resolved
@tjkirch
Copy link
Contributor Author

tjkirch commented Jan 28, 2020

This push just rebases on a typo fix from #636.

Copy link
Contributor

@bcressey bcressey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍟

workspaces/api/apiserver/src/server/mod.rs Outdated Show resolved Hide resolved
@tjkirch
Copy link
Contributor Author

tjkirch commented Jan 29, 2020

This push addresses @bcressey's concern by changing the API path to "templates". I confirmed that new AMIs and the new path worked.

@tjkirch tjkirch changed the base branch from platform-settings to develop February 1, 2020 00:13
This lets settings be generated based on a template that reference other
settings.  The first example is regional container image URIs, where we replace
part with the value of settings.aws.region.

The template is stored as a new metadata key, next to the setting-generator
metadata, which schnauzer will read from the API.  For example, if the template
is "foo-{{ settings.bar }}" and "settings.bar" is set to "baz", then the
setting output will be "foo-baz".

To be able to reference other settings from sundog-spawned generators, we need
to commit settings that have already been patched, e.g. from user data.  To do
this, this commit eliminates the settings-committer service, instead using the
settings-committer binary as an ExecStartPre on any service that requires
committed settings.
@tjkirch
Copy link
Contributor Author

tjkirch commented Feb 1, 2020

This push just rebased on develop; the only change was removing the first-party exception for schnauzer from cargo-deny.

@tjkirch tjkirch merged commit 2da8684 into develop Feb 1, 2020
@tjkirch tjkirch deleted the parameterize branch February 1, 2020 00:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants