Support endpoint DSL in YAML/JSON

[nicolaferraro]

Instead of:

route:
  from:
    uri: telegram/bots?chatId=1234
  steps:
  - to:
      uri: knative:endpoint/sink

We may have (alternative):

route:
  from:
    telegram:
      type: bots
      chatId: 1234
  steps:
  - knative:
      type: endpoint
      name: sink

It's not only syntactic sugar, the most important part of the syntax below is that it's possible to define the full json/yaml schema of the endpoints, so that a web console or IDE can automatically suggest components and their properties and also completely validate the DSL.

There are tools for IDE that support URI validation, but they need a language server. This would be LSP free.

Maybe we can start moving the DSL to apache camel in order simplify impl.

[lburgazzoli]

For consistency should it be:

route:
  telegram:
    type: bots
    chatId: 1234
  steps:
  - knative:
      type: endpoint
      name: sink

or do you think we should retain the from ?

[nicolaferraro]

Mmh.. that looks ok at first. What I fear is that after it's serialized and read again, we end up with this one, which is less clear:

route:
  steps:
  - knative:
      type: endpoint
      name: sink
  telegram:
    type: bots
    chatId: 1234

[lburgazzoli]

fix that damn json encoder/decoder :D

[nicolaferraro]

I can't :D

We've introduced the "flow" section in the integration CR, which makes the YAML DSL to belong to the same tree of the custom resource. Then it's no longer under control of our parser. The Kube API server "normalizes" the json putting keys into alphabetical order (that happens for all Kube resources).

[lburgazzoli]

fix that damn kubernetes

[lburgazzoli]

I've hacked something but now the issue is how to integrate it with the yaml dsl schema as the generated json schema is quite big (~ 41700 lines)

• camel-yaml-dsl.json: 3761 lines
• camel-yaml-endpoint.json: 41710 lines

I see two options:

1. to embedding it

   the resulting scheme will then be ~ 42000 lines but don't know if that could cause any degradation to the yaml language server

2. to reference it

   the link between the yaml dsl schema and the endpoint schema will be done through $ref to an external resources, something like:

   "aws2-s3": {
       "$ref": "https://github.com/apache/camel-k-runtime/blob/master/camel-k-loader-yaml/camel-k-loader-yaml/src/generated/resources/camel-yaml-endpoint.json#/definitions/aws2-s3",
   },

   this has the additional problem that we need to deal with versions so we probably need to reference the right branch which makes thing a little more complex.

I'd personally go for option 1 but we may need some hint from @lhein @apupier

[apupier]

1. the resulting scheme will then be ~ 42000 lines but don't know if that could cause any degradation to the yaml language server

no idea neither.
I will try manually with the camel-yaml-endpoint.json.
If it doesn't explode, I can write a test with "current schema" and with "embedded schema" and compare execution time. Do you have already a version of the schema with everything embedded? or a version with the references?

[apupier]

I modified https://github.com/lburgazzoli/apache-camel-k-runtime/blob/github-366/camel-k-loader-yaml/camel-k-loader-yaml/src/generated/resources/camel-yaml-endpoint.json to have a property defined and have it in completion. So created
https://raw.githubusercontent.com/apupier/camel-k-runtime/github-366-modfiedFortest/camel-k-loader-yaml/camel-k-loader-yaml/src/generated/resources/camel-yaml-endpoint.json
It seems to work well and load fast. but I added only 2 components.
Also, there was this message when adding the schemas:
For performance reasons, document symbols and folding ranges have been limited to 5000 items. Use setting 'json.maxItemsComputed' to configure the limit. Even though I don't understand very well why it is impacting a large schema. I would expect to impact only an data yaml file.

current schema is incorrect for required properties. it is using the camel case name although the dashed name is used for declaring the properties. Provided PR lburgazzoli#2

[lburgazzoli]

If it doesn't explode, I can write a test with "current schema" and with "embedded schema" and compare execution time. Do you have already a version of the schema with everything embedded? or a version with the references?

Not yet but I can work on it later today