From dc12fbfed30693fbc8a84c0af6632ec833a02b90 Mon Sep 17 00:00:00 2001 From: Eric Fritz Date: Sun, 3 Jan 2021 11:51:05 -0600 Subject: [PATCH] Clear readme. --- README.md | 154 +----------------------------------------------------- 1 file changed, 1 insertion(+), 153 deletions(-) diff --git a/README.md b/README.md index ed14e81..1ea0469 100644 --- a/README.md +++ b/README.md @@ -4,156 +4,4 @@ Configuration loading and validation for [nacelle](https://nacelle.dev). --- -Often, [initializers and processes](https://nacelle.dev/docs/core/process) will need external configuration during their startup process. These values can be pulled from a **configuration loader** backed by a particular [source](#sourcers) (e.g. environment or disk) and assigned to tagged fields of a configuration struct. - -You can see an additional example of loading configuration in the [example repository](https://github.com/go-nacelle/example): [definition](https://github.com/go-nacelle/example/blob/843979aaa86786784a1ca3646e8d0d1f69e29c65/internal/redis_initializer.go#L13) and [loading](https://github.com/go-nacelle/example/blob/843979aaa86786784a1ca3646e8d0d1f69e29c65/internal/redis_initializer.go#L36). - -### Configuration Struct Definition - -Configuration structs are defined by the application or library developer with the fields needed by the package in which they are defined. Each field is tagged with a *source hint* (e.g. an environment variable name, a key in a YAML file) and, optionally, default values and basic validation. Tagged fields must be exported in order for this package to assign to them. - -The following example defines configuration for a hypothetical worker process. For the application to start successfully, the address of an API must be supplied. All other configuration values are optional. - -```go -type Config struct { - APIAddr string `env:"api_addr" required:"true"` - CassandraHosts []string `env:"cassandra_hosts"` - NumWorkers int `env:"num_workers" default:"10"` - BufferSize int `env:"buffer_size" default:"1024"` -} -``` - -### Configuration Loading - -At initialization time of an application component, the particular subset of configuration variables should be populated, validated, and stored on the service that will later require them. - -```go -func (p *Process) Init(config nacelle.Config) error { - appConfig := &Config{} - if err := config.Load(appConfig); err != nil { - return err - } - - // Use populated appConfig - return nil -} -``` - -The `Load` method fails if a value from the source cannot be converted into the correct type, a value from the source cannot be unmarshalled, or is required and not supplied. After each successful load of a configuration struct, the loaded configuration values will are logged. This, however, may be a concern for application secrets. In order to hide sensitive configuration values, add the `mask:"true"` struct tag to the field. This will omit that value from the log message. Additionally, configuration loader object can be initialized with a blacklist of values that should be masked (values printed as `*****` rather than their real value) instead of omitted. These values can be configured in the [bootstrapper](https://nacelle.dev/docs/core). - -#### JSON Unmarshalling - -The values that are loaded into non-string field of a configuration struct are interpreted as JSON. Supplying the environment `CASSANDRA_HOSTS='["host1", "host2", "host3"]'` to the configuration struct above will populate the `CassandraHosts` field with the values `host1`, `host2`, and `host3`. The defaults for such fields can also be supplied as a JSON-encoded string, but must be escaped to preserve the format of the struct tag. - -```go -type Config struct { - CassandraHosts []string `env:"cassandra_hosts" default:"[\"host1\", \"host2\", \"host3\"]"` -} -``` - -#### Conversion and Validation - -After successful loading of a configuration struct, the method named `PostLoad` will be called if it is defined. This allows a place for additional validation (such as mutually exclusive settings, regex value matching, etc) and deserialization of more complex types (such enums from strings, durations from integers, etc). The following example parses and stores a `text/template` from a user-supplied string. - -```go -import "text/template" - -type Config struct { - RawTemplate string `env:"template" default:"Hello, {{.Name}}!"` - ParsedTemplate *template.Template -} - -func (c *Config) PostLoad() (err error) { - c.ParsedTemplate, err = template.New("ConfigTemplate").Parse(c.RawTemplate) - return -} -``` - -An error returned by `PostLoad` will be returned via the `Load` method. - -#### Anonymous Structs - -Loading configuration values also works with structs containing composite fields. The following example shows the definition of multiple configuration structs with a set of shared fields. - -```go -type StreamConfig struct { - StreamName string `env:"stream_name" required:"true"` -} - -type StreamProducerConfig struct { - StreamConfig - PublishAttempts int `env:"publish_attempts" default:"3"` - PublishDelay int `env:"publish_delay" default:"1"` -} - -type StreamConsumerConfig struct { - StreamConfig - FetchLimit int `env:"fetch_limit" default:"100"` -} -``` - -#### Sourcers - -A sourcer reads values from a particular source based on a configuration struct's tags. Sourcers declare the struct tags that determine their behavior when loading configuration structs. The examples above only work with the environment sourcer. All sources support the `default` and `required` tags (which are mutually exclusive). Tagged fields must be exported. The following six sourcers are supplied. Additional behavior can be added by conforming to the *Sourcer* interface. - -
-
Environment Sourcer
-
An environment sourcer reads the env tag and looks up the corresponding value in the process's environment. An expected prefix may be supplied in order to namespace application configuration from the rest of the system. A sourcer instantiated with NewEnvSourcer("APP") will load the env tag fetch_limit from the environment variable APP_FETCH_LIMIT and falling back to the environment variable FETCH_LIMIT.
- -
Test Environment Sourcer
-
A test environment sourcer reads the env tag but looks up the corresponding value from a literal map. This sourcer can be used in unit tests where the full construction of a nacelle process is too burdensome.
- -
Flag Sourcer
-
A flag sourcer reads the flag tag and looks up the corresponding value attached to the process's command line arguments.
- -
File Sourcer
-
A file sourcer reads the file tag and returns the value at the given path. A filename and a file parser musts be supplied on instantiation. Both ParseYAML and ParseTOML are supplied file parsers -- note that as JSON is a subset of YAML, ParseYAML will also correctly parse JSON files. If a nil file parser is supplied, one is chosen by the filename extension. A file sourcer will load the file tag api.timeout from the given file by parsing it into a map of values and recursively walking the (keys separated by dots). This can return a primitive type or a structured map, as long as the target field has a compatible type. The constructor NewOptionalFileSourcer will return a no-op sourcer if the filename does not exist.
- -
Multi sourcer
-
A multi-sourcer is a sourcer wrapping one or more other sourcers. For each configuration struct field, each sourcer is queried in reverse order of registration and the first value to exist is returned. This is useful to allow a chain of configuration files in which some files or directories take precedence over others, or to allow environment variables to take precedence over files.
- -
Directory Sourcer
-
A directory sourcer creates a multi-sourcer by reading each file in a directory in alphabetical order. The constructor NewOptionalDirectorySourcer will return a no-op sourcer if the directory does not exist.
- -
Glob Sourcer
-
A glob sourcer creates a multi-sourcer by reading each file that matches a given glob pattern. Each matching file creates a distinct file sourcer and does so in alphabetical order.
-
- -### Tag Modifiers - -A tag modifier dynamically alters the tags of a configuration struct. The following five tag modifiers are supplied. Additional behavior can be added by conforming to the *TagModifier* interface. - -
-
Default Tag Setter
-
A default tag setter sets the default tag for a particular field. This is useful when the default values supplied by a library are inappropriate for a particular application. This would otherwise require a source change in the library.
- -
Display Tag Setter
-
A display tag setter sets the display tag to the value of the env tag. This tag modifier can be used to provide sane defaults to the tag without doubling the length of the struct tag definition.
- -
Flag Tag Setter
-
A flag tag setter sets the flag tag to the value of the env tag. This tag modifier can be used to provide sane defaults to the tag without doubling the length of the struct tag definition.
- -
File Tag Setter
-
A file tag setter sets the file tag to the value of the env tag. This tag modifier can be used to provide sane defaults to the tag without doubling the length of the struct tag definition.
- -
Env Tag Prefixer
-
A environment tag prefixer inserts a prefix on each env tags. This is useful when two distinct instances of the same configuration are required, and each one should be configured independently from the other (for example, using the same abstraction to consume from two different event busses with the same consumer code).
- -
Flag Tag Prefixer
-
A flag tag prefixer inserts a prefix on each flag tag. This effectively looks in a distinct top-level namespace in the parsed configuration. This is similar to the env tag prefixer.
- -
File Tag Prefixer
-
A file tag prefixer inserts a prefix on each file tag. This effectively looks in a distinct top-level namespace in the parsed configuration. This is similar to the env tag prefixer.
-
- -Tag modifiers are supplied at the time that a configuration struct is loaded. In the following example, each env tag is prefixed with `ACME_`, and the CassandraHosts field is given a default. Notice that you supply the *field* name to the tag modifier (not a tag value) when targeting a particular field value. - -```go -if err := config.Load( - appConfig, - NewEnvTagPrefixer("ACME"), - NewDefaultTagSetter("CassandraHosts", "[127.0.0.1:9042]"), -); err != nil { - // handle error -} -``` +TODO