Expose a http api to get saga state changes

[andreasohlund]

Request:

GET /api/sagas/{saga-id}

Response

{
   "id":"93bfbbfe-f841-4ea5-a0fe-a2a800d8e08d",
   "saga_id":"93bfbbfe-f841-4ea5-a0fe-a2a800d8e08d",
   "saga_type":"MySaga",
   "changes":[
      {
         "start_time":"2014-01-03T12:09:37.5015257Z",
         "finish_time":"2014-01-03T12:09:37.5577757Z",
         "status":"new",
         "state_after_change":"[{\"$type\":\"MySagaData, ServiceControl.Sample\",\"Id\":\"93bfbbfe-f841-4ea5-a0fe-a2a800d8e08d\",\"Originator\":\"ServiceControl.Plugin.SagaAudit.Sample@ANDREAS\",\"OriginalMessageId\":\"124984c0-9742-460d-a6bc-a2a800d8ddd4\",\"SomeId\":\"00000000-0000-0000-0000-000000000000\"}]",
         "initiating_message":{
            "is_saga_timeout_message":false,
            "originating_endpoint":"ServiceControl.Plugin.SagaAudit.Sample",
            "originating_machine":"ANDREAS",
            "time_sent":"2014-01-03T12:09:35.284928Z",
            "message_type":"Message1"
         },
         "outgoing_messages":[
            {
               "destination":"ServiceControl.Plugin.SagaAudit.Sample@ANDREAS",
               "time_sent":"2014-01-03T12:09:37.54275Z",
               "message_type":"Message2"
            }
         ]
      }
   ]
}

[andreasohlund]

Reads data provided by #62

[dannycohen]

@andreasohlund -
We need the full history of the Saga, so that we can visualize it as defined in Particular/ServiceInsight#71
This includes:

1. Saga Type

2. Saga Id

3. Current Saga status (New|Updated|Completed)

4. Message History, for each message that initiated the saga:

   1. Initiating Message Id
   2. Initiating Message Type
   3. Initiating Message Time Sent
   4. Initiating Message state (indicating inflight message; see Expose a http api to get saga state changes #105 (comment))
   5. Initiating Message Conversation Id
   6. messages sent by Initiating Message (for each message sent by the initiating message)
      1. Sent Message Id
      2. Sent Message state (indicating inflight message; see Expose a http api to get saga state changes #105 (comment))
      3. Sent Message Type
      4. Sent Message Time Sent
   7. Change Type: {New|Updated|Completed}
   8. Change timestamp
   9. Properties state
      • state of each property after the change
      • assuming we can reliably deduce the state before the change by examining the state associated with the preceding message (based on the change timestamp sort order)
   10. Request Timeout Messages (for each message the initiating message requested timeout for)
       1. Initiating Message Type
       2. Requested Timeout Timespan
       3. Timeout Message Id (if timeout was invoked; assuming it is not available until then)
       4. Timeout Message Time Sent (if timeout was invoked; assuming it is not available until then)
       5. Timeout Message state (indicating inflight message; see Expose a http api to get saga state changes #105 (comment))

   Thoughts ?

[andreasohlund]

9: Do we really need special treatment for this? (since a tm is just a
message)

Regarding the message details: To avoid duplicating effort how about:

1. Return all relevant message ID's
2. Provide a bulk get of messages by id

?

On Mon, Nov 11, 2013 at 9:13 PM, Danny Cohen notifications@github.comwrote:

@andreasohlund https://github.com/andreasohlund -

1. We need the full history of the Saga, so that we can visualize it as
   defined in Add Saga view display ServiceInsight#71Add Saga view display ServiceInsight#71

• This includes
  1. Saga Type
  2. Saga Id
  3. Current Saga status (New|Updated|Completed)
  4. Message History, for each message that initiated the saga:
     1. Initiating Message Id
     2. Initiating Message Type
     3. Initiating Message Time Sent
     4. Initiating Message Conversation Id
     5. messages sent by Initiating Message (for each message sent by
        the initiating message)
        1. Initiating Message Id
        2. Initiating Message Type
        3. Initiating Message Time Sent
           1. Change Type: {New|Updated|Completed}
     6. Change timestamp
     7. Properties state
        • state of each property after the change
        • assuming we can reliably deduce the state before the change
          by examining the state associated with the preceding message
        • Request Timeout Messages (for each message the initiating
          message requested timeout for)
          1. Initiating Message Type
          2. Timeout Message Id (if timeout was invoked; assuming it is
             not available until then)
          3. Timeout Message Time Sent (if timeout was invoked;
             assuming it is not available until then)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/105#issuecomment-28234425
.

[dannycohen]

9: Do we really need special treatment for this? (since a tm is just a
message)

We basically need an indication that a Timeout Message is a Timeout Message, and what is the requested timeout timespan.
It can be with the other messages, but the visualization will be different (see 1 vs. 2 in the image below)

Regarding the message details: To avoid duplicating effort how about:

1. Return all relevant message ID's
2. Provide a bulk get of messages by id

I'm OK with that. It will require 1 more round trip request, but that's not significant.

I guess the deciding factor on whether to make message data available via the SC HTTP API call to Saga data or via a separate bulk get by msg id, is whether the time you save in developing the Saga HTTP API, is more than the time required to add the bulk get for msg id and the processing logic required in SI.
What do you think ?

[andreasohlund]

Talked this through with the team. We can provide all data in a single call
since that will make it easier for consumers

On Mon, Nov 11, 2013 at 9:53 PM, Danny Cohen notifications@github.comwrote:

9: Do we really need special treatment for this? (since a tm is just a
message)

We basically need an indication that a Timeout Message is a Timeout
Message, and what is the requested timeout timespan.
It can be with the other messages, but the visualization will be different
(see 1 vs. 2 in the image below)

[image: image]https://f.cloud.github.com/assets/3889023/1516347/eb4dbe84-4b12-11e3-988b-b13e0c0bbab6.png

Regarding the message details: To avoid duplicating effort how about:

1. Return all relevant message ID's
2. Provide a bulk get of messages by id

I'm OK with that. It will require 1 more round trip request, but that's
not significant.

I guess the deciding factor on whether to make message data available via
the SC HTTP API call to Saga data or via a separate bulk get by msg id, is
whether the time you save in developing the Saga HTTP API, is more than the
time required to add the bulk get for msg id and the processing logic
required in SI.
What do you think ?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/105#issuecomment-28237695
.

[dannycohen]

@andreasohlund - 👍

[andreasohlund]

As discussed we will add a "message status" that can be either "InFlight","SuccessfullyProcessed", "Failed" (please rephrase as needed @dannycohen )

[andreasohlund]

Also we will send the needed message info straight to SC from the endpoint plugin in order to capture those inflight messages

[dannycohen]

@andreasohlund - excellent idea.
Added "Message State" to proposed format (see #105 (comment)) for Initiating messages, Sent Messages, and Request Timeout Messages.

[dannycohen]

See API Response proposal below, (based on #105 (comment)).

Note that goal of API is to provide the data required in order to visualize Saga View. See (Particular/ServiceInsight#71).

Request:

GET /api/sagas/{saga-id}

Response:

{
  "SagaId": "845bad08-3171-49ab-9626-ad5f17e7ee3c",
  "SagaType": "VideoStore.OrderProcessingSaga",
  "ProcessedMessages": [
    {
      "Id": "dfdc3a9f-f339-400c-855c-d955509b3fac",
      "Type": "VideoStore.Messages.Events.OrderPlaced",
      "TimeSent": "2013-11-17T08:45:32.1114342+02:00",
      "TimeProcessed": "2013-11-17T08:45:32.1134359+02:00",
      "State": "SuccessfullyProcessed",
      "SendingEndpoint": {
        "Name": "VideoStore.ECommerceWeb",
        "QueueMachine": "VM111",
        "HostingMachine": "Host1"
      },
      "ProcesssingEndpoint": {
        "Name": "VideoStore.OrdersMgmt",
        "QueueMachine": "VM113",
        "HostingMachine": "Host3"
      },
      "ConversationId": "0d35ec60-0725-44d6-ac57-68c3ada15342",
      "SagaChangeTimestamp": "2013-11-17T08:33:32.1124353+02:00",
      "SagaStatusChange": "New",
      "SagaProperties": {
        "firstname": {
          "Before": "John",
          "After": "Johnny"
        },
        "city": {
          "Before": "Constantinople",
          "After": "Istanbul"
        }
      },
      "SentMessages": [
        {
          "Id": "9cadcf3a-d54c-46e3-826f-d892f8fdfccb",
          "Type": "VideoStore.Messages.Events.OrderProcessed",
          "TimeSent": "2013-11-17T08:45:32.1134359+02:00",
          "TimeProcessed": "0001-01-01T00:00:00",
          "State": "SuccessfullyProcessed",
          "SendingEndpoint": {
            "Name": "VideoStore.CustomerMgmt",
            "QueueMachine": "VM115",
            "HostingMachine": "Host15"
          },
          "ProcesssingEndpoint": {
            "Name": "VideoStore.ContentMgmt",
            "QueueMachine": "VM112",
            "HostingMachine": "Host12"
          }
        }
      ],
      "TimeoutMessages": [
        {
          "RequestedTimeout": "00:15:00",
          "Id": "e8bd1456-2e73-466e-a006-372578997f6c",
          "Type": "VideoStore.Messages.Events.OrderProcessTimeout",
          "TimeSent": "2013-11-17T08:45:32.1134359+02:00",
          "TimeProcessed": "0001-01-01T00:00:00",
          "State": "SuccessfullyProcessed",
          "SendingEndpoint": {
            "Name": "VideoStore.OrdersMgmt",
            "QueueMachine": "VM113",
            "HostingMachine": "Host3"
          },
          "ProcesssingEndpoint": {
            "Name": "VideoStore.OrdersMgmt",
            "QueueMachine": "VM113",
            "HostingMachine": "Host3"
          }
        }
      ]
    }
  ]
}

Questions:

1. It would be easiest for clients if SagaProperties contain both before and after values of Saga Properties. This has drawbacks on size. Please comment on whether it is advisable / feasible / easy to implement etc.
2. A message may have more than one Saga associated with it (example UI). It would be nice (not a must) to be able to show the types and link to the ID's of other Sagas the messages is associated with. Comments please.
3. Endpoint data includes a separation between Queue machine (where queue is located) and Hosting machine (on which endpoints host is located). This is based on Azure machine names should be instance names or hostnames #112 (comment). Is this correct ?
4. Preferences on representing empty date values ? (e.g. null, default(DateTime) or other. see TimeProcessed)

[andreasohlund]

Before and after will be very tricky. I suggest that we just include the
state as it was after the invocation and let the client (SI in this case)
apply the smartness (Json diffs etc)

Regarding queue machine is makes sense for azure, problem is that our
address system has no support for this so until we can have it fixed:

https://github.com/Particular/NServiceBus/tree/Address

i'd say we need to stick with a

"SendingEndpoint": {
"Name": "VideoStore.OrdersMgmt",
"Address": "{what ever the transpor put in the address}",
}

If not we need alot of if transport == X in SC :(

Preferences on representing empty date values ? (e.g. null, default(DateTime) or other. see TimeProcessed)

No preference lets ask Fielding ? (@johnsimons what do you suggest)

On Sun, Nov 17, 2013 at 7:58 AM, Danny Cohen notifications@github.comwrote:

See API Response proposal below, (based on #105 (comment)#105 (comment)
).
Request:

GET /api/sagas/{saga-id}
Response:

{
"SagaId": "845bad08-3171-49ab-9626-ad5f17e7ee3c",
"SagaType": "VideoStore.OrderProcessingSaga",
"ProcessedMessages": [
{
"Id": "dfdc3a9f-f339-400c-855c-d955509b3fac",
"Type": "VideoStore.Messages.Events.OrderPlaced",
"TimeSent": "2013-11-17T08:45:32.1114342+02:00",
"TimeProcessed": "2013-11-17T08:45:32.1134359+02:00",
"State": "SuccessfullyProcessed",
"SendingEndpoint": {
"Name": "VideoStore.ECommerceWeb",
"QueueMachine": "VM111",
"HostingMachine": "Host1"
},
"ProcesssingEndpoint": {
"Name": "VideoStore.OrdersMgmt",
"QueueMachine": "VM113",
"HostingMachine": "Host3"
},
"ConversationId": "0d35ec60-0725-44d6-ac57-68c3ada15342",
"SagaChangeTimestamp": "2013-11-17T08:33:32.1124353+02:00",
"SagaStatusChange": "New",
"SagaProperties": {
"firstname": {
"Before": "John",
"After": "Johnny"
},
"city": {
"Before": "Constantinople",
"After": "Istanbul"
}
},
"SentMessages": [
{
"Id": "9cadcf3a-d54c-46e3-826f-d892f8fdfccb",
"Type": "VideoStore.Messages.Events.OrderProcessed",
"TimeSent": "2013-11-17T08:45:32.1134359+02:00",
"TimeProcessed": "0001-01-01T00:00:00",
"State": "SuccessfullyProcessed",
"SendingEndpoint": {
"Name": "VideoStore.CustomerMgmt",
"QueueMachine": "VM115",
"HostingMachine": "Host15"
},
"ProcesssingEndpoint": {
"Name": "VideoStore.ContentMgmt",
"QueueMachine": "VM112",
"HostingMachine": "Host12"
}
}
],
"TimeoutMessages": [
{
"RequestedTimeout": "00:15:00",
"Id": "e8bd1456-2e73-466e-a006-372578997f6c",
"Type": "VideoStore.Messages.Events.OrderProcessTimeout",
"TimeSent": "2013-11-17T08:45:32.1134359+02:00",
"TimeProcessed": "0001-01-01T00:00:00",
"State": "SuccessfullyProcessed",
"SendingEndpoint": {
"Name": "VideoStore.OrdersMgmt",
"QueueMachine": "VM113",
"HostingMachine": "Host3"
},
"ProcesssingEndpoint": {
"Name": "VideoStore.OrdersMgmt",
"QueueMachine": "VM113",
"HostingMachine": "Host3"
}
}
]
}
]}

Questions:

• It would be easiest for clients if SagaProperties contain both
  before and after values of Saga Properties. This has drawbacks on size.
  Please comment on whether it is advisable / feasible / easy to implement
  etc.
• Endpoint data includes a separation between Queue machine (where
  queue is located) and Hosting machine (on which endpoints host is located).
  This is based on Azure machine names should be instance names or hostnames #112 (comment)Azure machine names should be instance names or hostnames #112 (comment).
  Is this correct ?
• Preferences on representing empty date values ? (e.g. null,
  default(DateTime) or other. see TimeProcessed)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/105#issuecomment-28643870
.

[dannycohen]

I suggest that we just include the state as it was after the invocation and let the client (SI in this case)
apply the smartness (Json diffs etc)

OK.

Regarding queue machine is makes sense for azure

I assume it also makes sense for SQL Server (and ActiveMQ?).
Can the hosting machine name be added as an additional header ? (possibly by the plugin; note that the hosting machine name is required for SP licensing )

[andreasohlund]

Yes for now it makes sense for SqlServer + Active as well

We already have the name of the machine hosting the sending + receiving
endpoint as headers:

https://github.com/Particular/NServiceBus/blob/develop/src/NServiceBus/Headers.cs#L190
'

https://github.com/Particular/NServiceBus/blob/develop/src/NServiceBus/Headers.cs#L200

and SC already used them.

On Sun, Nov 17, 2013 at 11:28 AM, Danny Cohen notifications@github.comwrote:

I suggest that we just include the state as it was after the invocation
and let the client (SI in this case)
apply the smartness (Json diffs etc)

OK.

Regarding queue machine is makes sense for azure

I assume it also makes sense for SQL Server (and ActiveMQ?).
Can the hosting machine name be added as an additional header ? (possibly
by the plugin; note that the hosting machine name is required for SP
licensing )

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/105#issuecomment-28646058
.

[dannycohen]

@andreasohlund - So the behavior should be consistent:

1. SQL Server, we do not show in OriginatingMachine or ProcessingMachine the name of the SQL Server machine, but we show the machine on which the endpoint service is installed. Correct ?
2. So the same should apply to Azure (which currently returns the machine on which the Queue / Azure Service Bus is located.
3. Later on, we can add the Queue hosting machine, as a low prio.

?

[andreasohlund]

Correct

Sent from my iPhone

On 17 nov 2013, at 12:49, Danny Cohen notifications@github.com wrote:

@andreasohlund - So the behavior should be consistent:

SQL Server, we do not show in OriginatingMachine or ProcessingMachine the name of the SQL Server machine, but we show the machine on which the endpoint service is installed. Correct ?
So the same should apply to Azure (which currently returns the machine on which the Queue / Azure Service Bus is located.
Later on, we can add the Queue hosting machine, as a low prio.
?

—
Reply to this email directly or view it on GitHub.