Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

effective customizations chapter #2659

Merged
merged 9 commits into from
Feb 4, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion site/book/06-deploying-packages/00.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
In the last chapter of this book, we are going to cover how you deploy a kpt
In this chapter of this book, we are going to cover how you deploy a kpt
package to a Kubernetes cluster and how the cluster state is managed as the
package evolves over time.

Expand Down
17 changes: 17 additions & 0 deletions site/book/07-effective-customizations/00.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
Kubernetes configuration packages and customizations go hand in hand, all the
packaging tools enable package customization. In this chapter we cover effective
mikebz marked this conversation as resolved.
Show resolved Hide resolved
customizations techniques the kpt hydration and packaging enables. We show how
mikebz marked this conversation as resolved.
Show resolved Hide resolved
providing customization through parameters has some pitfalls and recommend
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

alternatives where the contents of the package are no hidden behind a facade.
mikebz marked this conversation as resolved.
Show resolved Hide resolved
Some of these alternatives are only possible because kpt has made an investment
into bulk editing with KRM functions and upstream merging.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


### Prerequisites

Before reading this chapter you should familiarize yourself with [chapter 4]
which talks about using functions as well as updating a package page in
mikebz marked this conversation as resolved.
Show resolved Hide resolved
[chapter 3].

[chapter 4]: /book/04-using-functions/
[chapter 3]: /book/03-packages/

Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
## Scenario

I have a single value replacement in my package, I don’t want package consumers to look through all the yaml files to find the value I want them to set, it seems easier to just create a parameter for this value and have the user look at Kptfile for inputs.
mikebz marked this conversation as resolved.
Show resolved Hide resolved

## Problems

1. With popularity the single values inevitably expand to provide a facade to a large portion of the data defeating the purpose of minimizing the cognitive load.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be pretty easy to find a helm chart to link to as an example. I've used the jenkins chart in the past. A search for "universal helm chart" found this:
https://github.com/OleksandrUA/universal-helm-charts/blob/master/chart-service/_common/templates/deployment.yaml

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I call parameterizing every field the struct constructor pattern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that it's not hard to find a chart that suffers from the parameter problem. I think what @droot is asking for here is before and after so I'd have to create a feasible package for whatever it is that I showcase. There is also a limit to the size of the example to show case the end result. Maybe I can figure out how to just show code snippets here.

1. Some values like resource names are used as references so setting them in one place needs to trigger updates in all the places where they are referenced.
1. If additional resources that have similar values are added to the package new string replacements need to be added.
1. If a package is used as a sub-package the string replacement parameters need to be surfaced to the parent package and if the parent package already expects some values to be set and the parameters do not exist, the sub-package needs to be updated.

## Suggestions

1. kpt allows the user to edit a particular value directly in the configuration data and will handle upstream merge. When [editing the yaml] directly the consumers are not confined to the parameters that the package author has
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also explain this as similar to kustomize but where the patch is automatically derived

provided. kpt has made an investment into upstream merge that allows
[updating a package] that has been changed with a text editor. )
1. Attributes like resource names which are often updated by consumers to add prefix or suffix (e.g. *-dev, *-stage, *-prod, na1-*, eu1-*) are best handled by the [ensure-name-substring] function that will handle dependency updates as well as capture all the resources in the package.
1. Instead of setting a particular value on a resource a bulk operation can be applied to all the resources that fit a particular interface. This can be done by a custom function or by [search-and-replace] , [set-labels] and [set-annotations] functions.


[editing the yaml]: /book/03-packages/03-editing-a-package
[updating a package]: /book/03-packages/05-updating-a-package
[ensure-name-substring]: https://catalog.kpt.dev/ensure-name-substring/v0.1/
[search-and-replace]: https://catalog.kpt.dev/search-replace/v0.2/
[set-labels]: https://catalog.kpt.dev/set-labels/v0.1/
[set-annotations]: https://catalog.kpt.dev/set-annotations/v0.1/
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## Scenario:

I’d like to limit what my package consumers can do with my package and it feels safer to just provide a string replacement in one place so they know not to alter the configuration outside of the few places that I designated as OK places to change.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to re-read the sentence couple of times to understand the intent :) Can we start with a concrete use-case to describe the problem that can help the reader visualize the scenario better.

And then suggest how to do it the kpt way and then follow up with how existing tools (or techniques do) and discuss the trade-offs.


## Problems:

1. The limitation by parameters does not guarantee that consumers are in fact going to limit their changes to the parameters. A popular pattern is using kustomize to change Helm packages beyond what the package author has allowed by parameters.
1. String replacements rarely describe the intent of the package author.
When additional resources are added I need additional places where parameters need to be applied.

## Solutions:

1. General ways to describe policy already exist. kpt has a [gatekeeper] function that allows the author to describe intended limitations for a class of resources or the entire package giving the consumer the freedom to customize and get an error or a warning when the policy is violated.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is very generic and as a user it's not clear to me how to really apply it. We should describe the solution a bit more concretely here in context of an example. For ex.. in the tenant use case, enforcing the package to have only one namespace was one invariant we enforced using a custom function.


[gatekeeper]: https://catalog.kpt.dev/gatekeeper/v0.2/
15 changes: 15 additions & 0 deletions site/book/07-effective-customizations/03-generation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## Scenario:

When using template solutions like Helm I am able to provide conditional statements based on parameter values. This allows me to ask the user for a little bit of information and generate a lot of boilerplate configuration.

## Problems:

1. Over time the templating logic becomes its own language that becomes very complex. Debugging the template generation becomes a task of its own.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for linking to this, this is a good size package that we can possibly adopt since it's small and very readable.

1. The interplay between different conditionals and loops is interleaved in the template making it hard to understand what exactly is configuration and what is the logic that alters the configuration.
1. Templates are generally monolithic, when a change is introduced the package consumers need to either pay the cost of updating or the new consumers pay the cost of having to decipher more optional parameters.

## Solutions:

1. When a complex configuration needs to be generated the package author can create a generator function using turing complete languages and debugging tools. Example of such a function is [folder generation]. The output of the function is plain old KRM.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest a ordering change because reading it as a user feels we are anchoring too much on the negatives of other existing techniques before even showing how to do it well in kpt.

Starting scenario with a concrete use-case. For ex. where we talk a bit about the folder generation use-cases show sparse config and what resources we want to stamp out and then discussing how to do it the kpt way and then discussing other techniques (and how other tools do it) and explain the trade offs will be better ordering.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generation pattern isn't well supported, so I don't think the folder generator is a good example

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The case for a general-purpose programming language would be similar to Pulumi or cdk8s. The difference between representing the configuration as code and assuming exclusive actuation and operating on the configuration with code is composability and interoperability.

[folder generation]: https://catalog.kpt.dev/generate-folders/v0.1/
2 changes: 2 additions & 0 deletions site/book/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ This book is organized as follows:
- [Chapter 4] covers how to use kpt functions to automate configuration changes.
- [Chapter 5] guides you through developing custom functions.
- [Chapter 6] covers how to deploy a package to a Kubernetes cluster.
- [Chapter 7] covers effective customizations techniques.

Let's get started!

Expand All @@ -22,3 +23,4 @@ Let's get started!
[chapter 4]: /book/04-using-functions/
[chapter 5]: /book/05-developing-functions/
[chapter 6]: /book/06-deploying-packages/
[chapter 7]: /book/07-effective-customizations/
4 changes: 4 additions & 0 deletions site/sidebar.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@
- [6.1 Initializing a Package for Apply](book/06-deploying-packages/01-initializing-a-package-for-apply.md)
- [6.2 Applying a Package](book/06-deploying-packages/02-applying-a-package.md)
- [6.3 Handling Dependencies](book/06-deploying-packages/03-handling-dependencies.md)
- [7 Effective Customizations](book/07-effective-customizations/)
- [7.1 Single Value Replacement](book/07-effective-customizations/01-single-value-replacement.md)
- [7.2 Limiting Package Changes](book/07-effective-customizations/02-limiting-package-changes.md)
- [7.3 Generation](book/07-effective-customizations/03-generation.md)
- [Reference](reference/)
- [CLI](reference/cli/)
- [pkg](reference/cli/pkg/)
Expand Down