Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose code that translates IConfiguration to List<RouteConfig> and List<ClusterConfig> #1698

Open
davidfowl opened this issue May 7, 2022 · 5 comments
Assignees
Labels
Type: Idea This issue is a high-level idea for discussion.
Milestone

Comments

@davidfowl
Copy link
Contributor

What should we add or change to make your life better?

Expose this logic via a public API

Why is this important to you?

I need the ability to merge config from various sources that are added an removed dynamically (single aggregate dynamic config provider).

@davidfowl davidfowl added the Type: Idea This issue is a high-level idea for discussion. label May 7, 2022
@samsp-msft
Copy link
Contributor

Duplicate of #1713?

@rwkarg
Copy link
Contributor

rwkarg commented May 13, 2022

We're doing a similar thing merging configs discovered from multiple ephemeral Kuberentes clusters and currently use our own RouteInfo and ClusterInfo types (right now only the properties we currently use/discover from k8s) to serialize to/from instead of copy/pasting this code.

We use basically InMemoryConfigProvider (in steps 2-4) to gather and merge remote config then expose the merged config, but this IConfiguration ->Route/ClusterConfig would be useful in step 2 below when gathering the config the remote sources provide before merging.

  1. Kubernetes clusters discover annotated Services and write config (with TTL) to some central storage (Table storage, database, etc.)
  2. Proxies monitor central storage for changes in config and pull them down (deserialize to Route/ClusterConfig)
    a. This would be nice to use the documented JSON config format instead of our custom type
  3. Proxies merge configs (collapse distinct Routes, merge Clusters)
  4. Proxies update merged config

@karelz karelz added this to the YARP 2.0.0 milestone May 17, 2022
@karelz
Copy link
Member

karelz commented Jun 16, 2022

Triage: We are not sure we understand what and why is needed here -- @MihaZupan can you chime in from k8s point of view?

@davidfowl
Copy link
Contributor Author

The scenario I had was distributing YARP config per cluster in separate folders:

sites/site1/yarp.config.json
sites/site2/yarp.config.json
sites/site3/yarp.config.json

The custom configuration provider reads the site folder and watches it for changes. It builds an uber config and feeds that into YARP.

@Tratcher
Copy link
Member

Was that config laid out any differently than the standard config? Could you use the normal IConfiguration merging instead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Idea This issue is a high-level idea for discussion.
Projects
Status: 📋 Backlog
Development

No branches or pull requests

6 participants