[EE] Discuss how libres and ERT should communicate

[sondreso]

We need to find a suitable protocol for ERT and libres to communicate over. This should preferably not be something we invent ourselves. We should also make sure we solve this without creating a circular dependency between ERT and libres!

[jondequinor]

Doing a quick survey, it seems that https://cloudevents.io/ is a potential winner. They have also done a survey of existing event formats, for reference.

It's protocol agnostic, the events are JSON and have the following required attributes:

• id
• source
• specversion
• type

Example:

{
    "specversion" : "1.x-wip",
    "type" : "com.github.pull_request.opened",
    "source" : "https://github.com/cloudevents/spec/pull",
    "subject" : "123",
    "id" : "A234-1234-1234",
    "time" : "2018-04-05T17:31:00Z",
    "comexampleextension1" : "value",
    "comexampleothervalue" : 5,
    "datacontenttype" : "text/xml",
    "data" : "<much wow=\"xml\"/>"
}

There's a Python SDK https://github.com/cloudevents/sdk-python

Is well on its way of being blessed by the CNFC…

They provide a vocabulary with which we can talk about events https://github.com/cloudevents/spec/blob/master/spec.md#notations-and-terminology

[sondreso]

https://github.com/cloudevents/spec/blob/master/spec.md

All events have:
  time
  data
  source
  type
  id
  (subject)

Following events are defined

Forward Model Job

  - Start
    - data:
    - id
    - type: "ert.forward_model_job.start"
  - Running
    - data: max_memory_usage, current_memory_usage, std_err, std_out
  - Success
    - data:
  - Failure
    - data: exit_code (optional), error_msg
  - Event attributes:
    - id: wonder if this should just be UUID for the time being
    - source: /ert/job/<ee_id>/<real_id>/<step_id>/<job_id>
    - subject: 


{
  time: "2015-05-12…",
  data: "{max_memory_usage: 10000}",
  source: "/ert/job/0/23/",
  type: "com.equinor.ert.ensemble_evaluator.forward_model.job.start",
  id: "abcdef-123456-…-…",
}

Forward Model Step (Job dispatch)

(Local queue for executing job_dispatch, aka meta job_dispatch)

  - Init
    - data: jobs
  - Failure
    - data: error_msg
  - Success
    - data:
  - Event attributes:
    - id: UUID
    - source: /ert/forward_model/<run_id>:<real>:<step_id>
    - source: /ert/ee/<ee_id>/real/<real_id>/step/<step_id>

{
  time: "2015-05-12…",
  data: "{max_memory_usage: 10000}",
  source: "/ert/ee/<ee_id>/real/<real_id>/step/<step_id>",
  type: "com.equinor.ert.ensemble_evaluator.forward_model.step.init",
  id: "abcdef-123456-…-…",
}

Forward Model Stage (Queue Manager)

  - Start
  - Waiting
  - Pending
  - Running
  - Success
  - Failure
  - Unknown
  - Event attributes:
    - id: UUID
    - source: /ert/forward_model/<something_evaluator_unique>/real/<real_id>


{
  time: "2015-05-12…",
  data: "{}",
  source: "/ert/ee/<ee_id>/real/<real_id>",
  type: "com.equinor.ert.ensemble_evaluator.forward_model.stage.waiting",
  id: "abcdef-123456-…-…",
}

Ensemble Evaluator (TBD)

  - EnsembleResponseEvent
    - data: map(…)
  - Event attributes:
    - id: UUID
    - source: /ert/ensemble_evaluator/<uuid>