Event Domain doesn't support a CloudEvents v 1.0

[romankiss]

Consuming a CloudEvents by the AEG event domain endpoint failed at the source attribute.

For example:

[
  {
    "id":"00000000-0000-0000-0000-000000000000",
    "source":"https://my-source.azerwebsites.net/",
    "specversion":"1.0",
    "type":"recordInserted",
    "dataschema":"#1.0",
    "subject":"/myapp/vehicles/motorcycles",
    "time":"2019-11-23T16:43:22.5111403Z",
    "data":{
      "make":"Ducati",
      "model":"Monster"
    }
  }
]

will generate the following error:

{
  "error": {
    "code": "BadRequest",
    "message": "Required property 'topic' needs to either be a full resource id or should be just the name of the domain topic. Found: https://my-source.azerwebsites.net/. Report 'c6bb2d89-c7b6-4bb1-ab2e-9e253eb1d79e:4:11/25/2019 10:52:07 AM (UTC)' to our forums for assistance or raise a support ticket.",
    "details": [
      {
        "code": "InputJsonInvalid",
        "message": "Required property 'topic' needs to either be a full resource id or should be just the name of the domain topic. Found: https://my-source.azerwebsites.net/. Report 'c6bb2d89-c7b6-4bb1-ab2e-9e253eb1d79e:4:11/25/2019 10:52:07 AM (UTC)' to our forums for assistance or raise a support ticket."
      }
    ]
  }
}

The another compatibility issue with a CloudEvents v 1.0 is using a batching for single event, in other words, the event domain didn't allow to consume a single event message (JObject), it must be wrapped by array.

For EventGridSchema we can use a topic property for domain topic, but for consuming the CloudEvents messages from any producer, the source attribute should be flowed to the subscriber via filtering without any modification. In other words, the domain topic can not be hardcoded to the source attribute such as an URI-reference type, for examples:

https://github.com/cloudevents
urn:uuid:6e8bc430-9c3a-11d9-9669-0800200c9a66
/cloudevents/spec/pull/123
abcd

Thanks
Roman

Update

Using a batching vs single CloudEvents v1.0 events is working well for custom topic and also for event domain. It is based on the Content-Type header such as

application/cloudevents-batch+json  

or

application/cloudevents+json

The compatibility issue when the CloudEvents v1.0 is used can be simple tested within the AEG in the following cases (cascading events, for example: blob storage -> custom/domain topic):

1. subscribing a custom topic for delivery and input = CloudEventSchemaV1_0

2. subscribing an event domain for delivery and input = CloudEventSchemaV1_0

Note, that the case 1 will work very well included a validation handshaking, but in the case 2, the validation is passed, but every Notification events will failed for reason a source attribute.

(error = BadRequest, "Required property 'topic' was found to be an invalid resource id. ...")

Another incompatibility with the CloudEvents v1.0 is that the request is not validate for required attributes of the CloudEvents schema and basically can be used only the following attributes, for example:

    {
       "type":"com.github.pull.create",
       "id":"A234-1234-1234",
    }

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: ddd55e19-eb11-3b0f-c6e1-3710f712fd58
• Version Independent ID: f413f074-6a81-73fa-0674-f8bc98aeac1d
• Content: Use Azure Event Grid with events in CloudEvents schema
• Content Source: articles/event-grid/cloudevents-schema.md
• Service: event-grid
• GitHub Login: @banisadr
• Microsoft Alias: babanisa

[mike-urnun-msft]

@romankiss Thank you for your feedback! We will review and update as appropriate.

[mike-urnun-msft]

cc: @banisadr

[jimmcslim]

I also observe this. The documentation doesn't make clear what attribute of a CloudEvents message corresponds to topic. We are assuming it is subject. However, the Cloud Events SDK requires subject to be a URI type. However, if you try and create an Event Domain Topic with a name that looks like a URI (ie. 'urn:something:something', '/something/something') it fails to create as only A-Za-z0-9 are permitted. Disappointing.

[acomley]

Any update on this? This issue makes it very hard to use CloudEvent schema.

It is also undocumented that the source maps to the topic. The previous iteration used subject with topic#subject format.

[SeanFeldman]

Six months since it was raised. Seems to be related to Azure/azure-sdk-for-net#10684

@jfggdl is this something on the radar?

[acomley]

I noticed that you can now specify the field that maps to topic when creating the Event Grid Domain. I added a custom field called route to handle the topic. Verified this all works by just making a direct request to the endpoint (no sdk) using a Content-Type: application/cloudevents-batch+json.

[jfggdl]

Thanks @romankiss and @SeanFeldman. We are doing our backlog grooming and we will prioritize this ask. @ahamad-MS or someone else from our team can start providing some information about what is supported. We will also fix any resulting documentation issues.

[jfggdl]

#reassign:ahamad-MS

[PRMerger8]

ahamad-MS is already assigned.

[spelluru]

@ahamad-MS - could you review and add your input here? Thanks!

[ahamad-MS]

@romankiss ... CloudEvent support was recently added in the new generation of SDK https://www.nuget.org/packages/Azure.Messaging.EventGrid/4.0.0-beta.2 .

Please confirm you no longer have this issue.

[romankiss]

@ahamad-MS - The issue is still there such as in the AEG domain implementation. My comment was not related to the SDK.

This issue can be duplicated posting the event to the domain event endpoint via the REST:

Payload (example from the CloudEvent spec 1.0:

{
  "specversion":"1.0",
  "type":"com.github.pull_request.opened",
  "source":"https://github.com/cloudevents/spec/pull",
  "subject":"123",
  "id":"A234-1234-1234",
  "time":"2018-04-05T17:31:00Z",
  "comexampleextension1":"value",
  "comexampleothervalue":5,
  "datacontenttype":"text/xml",
  "data":"<much wow=\"xml\"/>"
}

response:

{
  "error": {
    "code": "BadRequest",
    "message": "Required property 'topic' needs to either be a full resource id or should be just the name of the domain topic. Found: ....",
    "details": [
      {
        "code": "InputJsonInvalid",
        "message": "Required property 'topic' needs to either be a full resource id or should be just the name of the domain topic. Found: ..."
      }
    ]
  }
}

another example:

when the source property has a simple string value (internally name of the topic), then the request is passed, see the following example:

Thanks
Roman

[spelluru]

@ahamad-MS - please review the latest post from Roman as soon as you can. Thank you.

[ahamad-MS]

@romankiss we are investigating this issue and will update the thread with findings. However, as a general comment, this is not a real document bug and fit better as service issue. Please use user voice or open a support ticket to handle similar issues in the future. Thanks

[spelluru]

@romankiss - Appreciate if you could submit an idea on User Voice(https://feedback.azure.com/forums/909934-azure-event-grid), or open a support ticket for @ahamad-MS to investigate further. Thank you.

@ahamad-MS - please feel free to reopen the issue if you want to add any information. Thanks.

#please-close

[MarkAtAgilliance]

Is there any movement on this. It is confusing to consumers to be subscribing to a topic, named identify a boundary for an audience of subscribers but uses "Source" concept in CloudEvents to map that event to a topic. Seems like source and destination concepts are being used here.

Referring to MSFT documentation for cloud event source., states for source property "Gets or sets the context in which an event happened" - so this is event centric statement.

In this doc, topics are described as a boundary for subscriptions..

My consumers are asking why the source property contains the topic name (in our case their Partner ID just like the MSFT example).