Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow value-concatenating an object to each list element #24

Open
carrot-garden opened this issue Jul 11, 2012 · 15 comments
Open

allow value-concatenating an object to each list element #24

carrot-garden opened this issue Jul 11, 2012 · 15 comments

Comments

@carrot-garden
Copy link

  1. this is a feature request

  2. use case:

  • imaging you have network service with some 100 end points;
  • each end point has some 30 parameters, mostly common to all points;
  • may be 2 to 5 of the parameters are to be overridden by each point;
  1. the way I understand current mechanisms of HOCON
    https://github.com/typesafehub/config/blob/master/HOCON.md
    this would make this definition very verbose;

  2. how about new/proposed object$builder syntax:

//
// the list definition below will be looking for this builder
//
netty_point_list$builder {
   id = "you-must-override"
   localAddress = "1.2.3.4:12345"
   remoteAddress = "you-must-override"
   packetTTL = 5
   pipeline = "default-pipeline"
   pipeline_param_0 = 123
   // 20 more parameters here; mostly come from include  or variable substitution
}

//
// netty_point_list will be looking for optional netty_point_list$builder
// to initialize default list items parameters before override
//
netty_point_list [
   { id = "service-1", remoteAddress = "1.1.1.1:12345", pipeline = "advanced" }
   { id = "service-2", remoteAddress = "1.1.1.2:12345", localAddress = "3.3.3.3:12345" }
   // 100 more service definitions with little variations on the builder-defined parameters
]
  1. am I missing some HOCON feature or what? :-)
@carrot-garden
Copy link
Author

for those interested, a solution:

https://github.com/carrot-garden/carrot-conf/tree/master/carrot-conf-list

@viktorklang
Copy link
Contributor

Won't withFallBack work here just fine?

@carrot-garden
Copy link
Author

yes, but you must tell withFallBack where to get it from;

if you have a lot of lists, would it not make sense to have a convention with a list builder?

convention rules:

  • if it is a list, look for a builder first
  • look on the same level as list only
  • look for builder name pattern "${list-name}/builder"
  • if builder present, use it withFallBack(builder) on each list item;

@viktorklang
Copy link
Contributor

I'd be surprised if it was more than 5 lines of code to create a method that does that for any ConfigMap.

@carrot-garden
Copy link
Author

you are right; and this argument can be extended to include all of jdk :-)

but actually it is more then 5 lines
https://github.com/carrot-garden/carrot-conf/tree/master/carrot-conf-list

@havocp
Copy link
Collaborator

havocp commented Jul 31, 2012

The way you can have a fallback for each list item right now is something like this:

mydefaults = {
 a=1, b=2
}

mylist = [
  { a = 3 } ${mydefaults}
  { b=10 } ${mydefaults}
]

In the spec this is "value concatenation"

it might be nice to avoid the repetition of ${mydefaults} after each list entry, but a syntax that feels elegant and general (maybe covers other similar cases? not sure) would need some thought.

You can also do this in code of course; when you load that list, map _.withFallback(getConfig("mydefaults")) over the list.

At some point this is supposed to be a (relatively) obvious declarative file, so I'm reluctant to start making it into a mini programming language...

I believe right now that concatenating/merging lists and objects is an error:

  [ a, b ] { foo: bar}  // error (I _think_), concat list with object

Viktor suggested the object in that case could be interpreted as "merge with each list element." One issue here is what happens if you have multiple lists in multiple places for the key, like say you have in reference.conf:

  [] { foo: bar}

Then in application.conf, someone wants to merge in the defaults to their list, how does that happen... you'd sort of need that {foo: bar} to be available as a substitution, so reference.conf would need to do:

mydefaults = { foo: bar}
mylist = [] ${mydefaults}

and application.conf:

mylist = [ a, b ] ${mydefaults}

but then that's getting a little tricky to explain to people.

If I were expecting third parties to edit application.conf, I'd probably go with the withFallback in code just so it'd be nice and easy to use.

@carrot-garden
Copy link
Author

wow, you have a lot of cool ideas, thanks for sharing;

"value concatenation" is way too verbose; but this one is my favorite:

mylist = [ {a=1}, {b=2} ] ${mydefaults}

I hope you can accept it?

@viktorklang
Copy link
Contributor

In the case of multiple defaults just apply them all?

@havocp
Copy link
Collaborator

havocp commented Aug 1, 2012

If this is part of the existing "value concatenation" mechanism then multiples are already defined I guess. Value concatenation lets you have as many values as you want (a bunch of adjacent strings => one big concatenated string, a bunch of adjacent lists => one big concatenated list, a bunch of adjacent objects => one big merged object).
The idea here would extend the rule to make adjacent list+object do something instead of being an error.

One thing I just thought of, currently value concatenation is associative in the mathematical sense because concatenation and merging are both associative. So it doesn't matter if the concatenation "operator" (syntactic adjacency) is left or right associative. However in this list+object case, it gets a little confusing. []{}[]{} if left associative would fallback each element in the first list to the object in position 2, then concatenate the list in position 3, then fallback each element in the resulting list to the object in position 4. If right associative, it would fallback each element of the list in position 3 to the object in position 4, then I guess puke because an object on the left and list on the right wouldn't make sense.

Multiple objects just works with either associativity; []{}{} would either fallback each element to position 2 object, then fallback each element to position 3 object; OR merge the two objects, then fallback each element in the list to the merged object. Those should be equivalent so it doesn't matter.

This doesn't really give you a way to specify defaults in the sense that reference.conf specifies defaults. It just gives you a shorthand. Because the "concatenation" trick is a purely local transformation. If reference.conf happens to build its list with the []{} syntax, it still is just making a list. Later attempts to append to that list would not be affected (i.e. they wouldn't get the "defaults").

Another thought, this might naturally generalize such that a list of strings followed by a string value-concats the string to each element, but it breaks with a list of lists followed by a list since that already means something... inconsistent.

Another line of thought here would be to have some explicit "map" operator which would maybe be clearer, or maybe not, since I don't really know how that operator would be written (the word map? starts to be uncomfortably programming-language-like). But it avoids the inconsistency in the case of wanting to "map" a list onto a list of lists.

I don't know. I'm thinking we keep thinking about it and it's sort of a post-1.0 thing...

@carrot-garden
Copy link
Author

wow! this reads like scala type system spec now :-)

can we get a taste of some simple interpretation of

mylist = [ {a=1}, {b=2} ] ${mydefaults}

make adjacent list+object do something instead of being an error

for v 0.6 ?

just to let more people see it and possibly share their thoughts?

@viktorklang
Copy link
Contributor

Please feel free to take a stab at it!

@carrot-garden
Copy link
Author

ok, here is my stab:

  1. this already works, right:
mydefaults = {
 a=1, b=2
}
mylist = [
  { a=3  } ${mydefaults}
  { b=10 } ${mydefaults}
]
  1. then do not throw any errors, just expand
mydefaults = {
 a=1, b=2
}
mylist = [
  { a=3  } 
  { b=10 } 
] ${mydefaults}

into

mydefaults = {
 a=1, b=2
}
mylist = [
  { a=3  } ${mydefaults}
  { b=10 } ${mydefaults}
]

and process as before

@viktorklang
Copy link
Contributor

Do you have code + tests?

@carrot-garden
Copy link
Author

yes

#31

@havocp havocp changed the title list item template / builder allow value-concatenating an object to each list element Mar 7, 2015
@havocp
Copy link
Collaborator

havocp commented Mar 7, 2015

for additional discussion see #31

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants