YOML = YAML object mapping language [BIG!]

[ahgittin]

As discussed on mailing list...

I've been spiking a new approach to our YAML modelling for java objects.

The root problem I want to solve is to that it's tedious, inconsistent, and poorly documented how YAML should map on to the Java items. Every new place we want to do this we use ad hoc code. This is an obstacle to writing new java items which can easily be configured in YAML by any Brooklyn user.

The idea of the solution is to have a schema for mapping Java <-> YAML so you can simply annotate the Java classes and the job's done. You get a working schema for reading from YAML without having to think about it. The thinking is that this lets us make many more things expressible in YAML, including:

• effectors defined as sequences of tasks
• common predicates (as has been discussed in Added attribute to DSL for immediate return of sensor data #282 [1] and the mailing list)
• more sensor feeds, policies, etc

I've built a prototype called "YOML" (YAML Object Mapping Language) and opened a PR for it [2].

As it stands it supports reading and writing (so we could improve our persistence model and be able to output reusable plans from the current state of the world -- though I've not worked on those), and it's designed to be very flexible, optimizing for the experience of the human reading and writing it.

It also has hooks for reflecting about itself -- meaning we could generate documentation and code completion proposals with explanations for everything that is YOML'd (though again not implemented).

It's integrated with the new BrooklynTypeRegistry (code which replaces the old fixed-type-category catalog, allowing arbitrary types and aliases; it's used currently, but not persisted, and we're not taking advantage of it yet; we could also use this to restrict who can use which types) and has a TypePlanTransformer so we can use it in specific places. You can use anywhere the YAML DSL is supported by saying $brooklyn:object-yoml: ..., and it's hooked in specially in one place, under brooklyn.initializers if you supply a map (previously only a list was allowed), with some sensor/effector tests at [3], in short allowing:

• type: org.apache.brooklyn.entity.software.base.EmptySoftwareProcess
  brooklyn.initializers:
  echo-hi-name-from-config:
  type: ssh-effector
  command: echo hi ${NAME}
  env:
  NAME: $brooklyn:config("name")
  date:
  type: ssh-sensor
  command: date
  period: 100ms
  brooklyn.config:
  name: bob

A detailed write-up is at [4].

I could do an online brown-bag session next week if folks are interested.

[1] #282
2
[3] https://github.com/ahgittin/brooklyn-server/blob/yoml/camp/camp-brooklyn/src/test/java/org/apache/brooklyn/camp/yoml/YomlTypeRegistryEntityInitializersTest.java
[4] https://github.com/ahgittin/brooklyn-server/blob/yoml/utils/common/src/main/java/org/apache/brooklyn/util/yoml/sketch.md

[ahgittin]

test failures unrelated, described in #560

[ahgittin]

More goddamn merge conflicts. Any objections to commit-then-review on this one?

[aledsage]

@ahgittin with such a huge pull request, I think we should definitely stick with review before merging. We are hopefully going to find time next week to review this properly, with a concrete plan for how more members of the community would become familiar with this large code base, to build on and maintain it long term.

[ahgittin]

Any update @aledsage ? Was it was reviewed the second week of May as planned and there were no comments? ;)

I'd like to use this for the catalog.bom syntax. Instead I'm having to write new ad-hoc parsing in order to support OSGi auto-wrapping. Please a high-level review.

[ahgittin]

Updated for merge conflicts. Fixes access to AddSensor.params. Also incorporates latest OSGi works (#799 - not yet merged but including a fix to an issue this PR tested for but others didn't, recursive references being treated as templates).