Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add package for loading application config #89

Merged
merged 6 commits into from
Jul 28, 2021
Merged

Conversation

bluekeyes
Copy link
Member

@bluekeyes bluekeyes commented Jul 10, 2021

The appconfig package defines a standard (but customizable) flow for loading repository-level configuration:

  1. Try a list of paths in the repository, in order
  2. If a path exists, check if it contains a remote reference
    1. If it does, follow the reference
    2. Otherwise, return the content
  3. If no paths exist, try a different list of paths in a owner-default (e.g. .github) repository

The package doesn't require the configuration to have any format (it returns raw bytes), but by default assumes that remote references are encoded as YAML (this is customizable.)

This will hopefully make it easier to implement a consistent config loading mechanism in our apps.

Right now, this contains the public interface and what I think are all the exported types. The logic for actually loading config is unimplemented, so I'm most interested in comments on the API. Some starting questions/thoughts:

  • Does it make sense to support loading remote config from private repos in other organizations? This was requested in Policy Bot once, but I'm not sure how useful it is. Maybe more useful if it is also needed for internal repos?
  • I did not define a Loader interface because there's only one implementation. But since there's only one method, it's easy enough for clients to define the necessary interface if they want to provide mocking or different implementations.
  • I made this a new top-level package because it defines enough types that I didn't want to mix it into githubapp. Since oauth2 is already at the top-level, I stuck with that convention instead of nesting it in githubapp.
  • appconfig felt like a good name because config was too generic, but I'm open to other suggestions if anyone thinks appconfig will get confused with the server configuration.

Copy link
Member

@asvoboda asvoboda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a big fan of this draft. I think this will be a really nice qol upgrade, and should help out a ton to remove these "default" style configs from repos.

Do you think it would be nice/reasonable to create a "global" default fallback, after organization specific checks?

appconfig/appconfig.go Show resolved Hide resolved
appconfig/options.go Outdated Show resolved Hide resolved
@bluekeyes
Copy link
Member Author

Do you think it would be nice/reasonable to create a "global" default fallback, after organization specific checks?

My current thought for this is that it's something implemented in each application. If loading config returns an undefined value, the application can choose to either substitute a default config or ignore the repo. That gives more flexibility in how the default is defined and means you don't have to encode it as a []byte to pass to this package just to decode it again.

@IainSteers
Copy link
Contributor

Given some of the aims we have at the moment with internal github apps and config centralization I'm massively in favour of this as drafted.

I think the current method and approach makes sense to me.

I don't see the use-case for loading configuration from a private repo in another org, feels very strange to me and liable to leak whatever they're trying to keep secret anyway, but I'm not familiar with the original request

appconfig feels sufficiently descriptive to me and I agree that it doesn't necessarily belong in githubapp

@bluekeyes bluekeyes marked this pull request as ready for review July 24, 2021 00:38
@bluekeyes
Copy link
Member Author

Implemented the core logic and added some basic tests, so I think this is ready for review.

README.md Outdated Show resolved Hide resolved
Copy link
Member

@asvoboda asvoboda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Largely in support of this PR! I have a couple of small qs about the config, but I think it makes a lot of sense and will be really valuable.

func NewLoader(paths []string, opts ...Option) *Loader {
defaultPaths := make([]string, len(paths))
for i, p := range paths {
defaultPaths[i] = strings.TrimPrefix(p, ".github/")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this path get trimmed out?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly just default behavior I decided on. The idea is that if you are using a .github repository, you should be able to set configuration for the .github repository that is different from the configuration it contains for all the other organization repositories. If the path was the same, the configuration you used for all repos would have to be the same as the configuration used for the .github repository itself.

This is all customizable with the WithOwnerDefault option.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should also add that stripping this path matches the idea of the .github repo being like the shared contents of a .github folder in a single repo.

Comment on lines +177 to +180
// Referencing a remote file that does not exist is an error because
// this condition is annoying to debug otherwise. From the perspective
// of a repository, it appears that the application has a configuration
// file and it is easy to miss that e.g. the ref is wrong.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥇

ld := Loader{
paths: paths,
parser: YAMLRemoteRefParser,
defaultRepo: ".github",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a configurable server option?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is customizable with the WithOwnerDefault option.

appconfig/appconfig.go Show resolved Hide resolved
@bluekeyes
Copy link
Member Author

Merging this for now, will try to put a prototype conversion for policy-bot or bulldozer later this week.

@bluekeyes bluekeyes merged commit e37723e into develop Jul 28, 2021
@bluekeyes bluekeyes deleted the bkeyes/appconfig branch July 28, 2021 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants