[Core] Configuring Components

[vaastav]

In GitLab by @JonathanMace on Sep 15, 2023, 21:34

Components can have configuration associated with them. For example, RPC plugins can have a timeout associated with them. Similarly a clientpool modifier will have a configuration value for the number of clients in the clientpool.

The current approach to configuration is direct and does not support a number of use cases. This issue is to incorporate a general notion of configuration into Blueprint and covers:

1 Introducing new IR nodes to represent configuration
1 Exposing configuration to users through the wiring spec
1 Providing configuration APIs to plugins that wish to offer more complex configuration than the current direct approach

Background: Current Approach

Configuration is currently possible without any additional support from core Blueprint components. However, it only supports a limited use-case: plugins can offer configuration only by directly exposing it to users in the wiring spec.

Currently, plugins define their own helper methods for adding nodes to the wiring spec. For example, the GRPC plugin has the helper method grpc.Deploy(wiring blueprint.WiringSpec, serviceName string) that will deploy a service using GRPC.

If a plugin wishes to expose configuration options in the wiring spec, it can do the following:

1 Simple and direct: A plugin can provide multiple helper methods, to allow for extra configuration options. For example, to expose a Timeout option, the plugin could also expose a method grpc.DeployWithTimeout(wiring blueprint.WiringSpec, serviceName string, timeout int)
2 More general approach: If a plugin provides multiple options, then it can define an options struct and provide an additional method that takes the option struct, e.g. grpc.DeployWithOptions(wiring blueprint.WiringSpec, serviceName string, options GRPCOptions); any options set in the struct will be used, otherwise defaults will be used

The above approach does not change the plugin's IR nodes substantially. Now, all they need to do is store the configuration values on the existing IR node(s).

The above approach also does not substantially affect a plugin's code generation. The existing code generation would just be expanded to take into account any configuration values on the IR node. The generated code would directly bake in the config values.

Limitations of Current Approach

The current approach fails the following use-cases

1 Workflow spec nodes, specifically, do not support configuration values, because the workflow spec plugin does not support it
2 Plugins that support the same concepts -- e.g. Thrift and gRPC both have timeouts -- are configured in arbitrarily different ways; potentially they could use the same configuration transparently (thus requiring fewer wiring spec changes when swapping between plugins)
3 Users cannot override configuration values at runtime
4 In the generated system, configuration is not centralized in any way -- values are dispersed into code arbitrarily, making it difficult for users assess the configuration by looking at the generated code

Proposed Approach

The sub-issues related to this issue cover the various implementation pieces related to configuration for Blueprint and for specific plugins.

[vaastav]

In GitLab by @JonathanMace on Sep 16, 2023, 24:23

marked this issue as related to #3

[JonathanMace]

The ability to do configuration is implemented by e2f88b4 but no plugin has yet made use of it so it might need further fleshing out. Closing for now.