ICRC-12: Canister Events

[SuddenlyHazel]

icrc: 12
title: ICRC-12 Canister Events 
author: Hazel Rowell <hazel@departure.dev>,
discussions-to: https://github.com/dfinity/ICRC/issues/12
status: Deliberating 
category: ICRC
requires: None
created: 2023-02-26
updated: 2023-02-26

Moving from the forum to continue discussion.

Summary

A minimum set of types and methods to facilitate the development of event driven systems on the IC.

Introduction

This proposal aims to enable the development of event-driven systems on the IC by introducing a set of types and methods. Our approach in this proposal is omnibus. We propose the use of the PublisherMessage, which can contain either a PublisherNotification or an Event. The Event is loosely self-describing with the help of the event_schema property. Our goal is to provide a framework that is tight enough to allow developers to start publishing and subscribing immediately while being flexible enough to accommodate future scaling needs.

Publisher

icrc12_subscribe & icrc12_unsubscribe

type SubscriptionError = variant {
    NotAuthorized;
};

type SubscriptionResponse = variant {
    Ok : null;
    Err : opt SubscriptionError;
};

type Publisher = service {
  icrc12_subscribe : (handler : (PublisherMessage) → ()) → (SubscriptionResponse);
  icrc12_unsubscribe : (handler : (PublisherMessage) → ()) → (SubscriptionResponse);
}

Subscriber

icrc12_handler

type Event = record {
    event_schema : text;
    event_type : text;
    created_at : nat64;
    data : vec nat8;
};

type PublisherNotification = variant {
    NewSender  : record {p : principal};
};

type PublisherMessage = variant {
    Event : Event;
    Notification : opt PublisherNotification;
};

type Subscriber  = service : {
    icrc12_handler: (PublisherMessage) -> ();
}

[SuddenlyHazel]

Updated the above to include icrc12_subscribe & icrc12_unsubscribe following feedback by rossberg here

[SuddenlyHazel]

Updated the above to remove event_type : text; following comments from skilesare

[testpuddle]

I am wondering if Publisher should provide an interface to allow icrc12_servicediscovery that could return the name of the service provider along with the services offered. This would help identify which service is being run (e.g. if someone is running instances of the same service with a dev, test, prod and demo_intergration instantiation - it would be easy to mix up which one is which. This way it would be possible to do a service discovery ping. Just an idea.

[SuddenlyHazel]

I am wondering if Publisher should provide an interface to allow icrc12_servicediscovery that could return the name of the service provider along with the services offered. This would help identify which service is being run (e.g. if someone is running instances of the same service with a dev, test, prod and demo_intergration instantiation - it would be easy to mix up which one is which. This way it would be possible to do a service discovery ping. Just an idea.

Interesting! Are you referring to something like interface detection? Or, more a way to tag the events coming out of the service?

[testpuddle]

The canister interface is exposed via Candid, unfortunately, there isn't an ORB like naming service and we have no clue what the service would do when exposed.

E.g. An interface that has a method:"PressButtonWhenReady" what service is it connected to?

While we can identify a canister by its CanisterID, we do not know its name or purpose. That is why I think a naming service would help for discovery and confirming that correct interface is being involed. The canister could respond to a query of Name and version of the API in lieu of a ORB naming service, .

I am not certain it would be required at the event layer. It would increase the payload and cost if were to be implemented there. However, if message routing at the app-tier level becomes a thing, it could serve as a way to route messages between different versions of an API by using a message broker.

Not sure if this makes sense. Thoughts?

[skilesare]

I wanted to make sure that you all saw https://github.com/fury02/example-async-data-deliveries and https://github.com/fury02/async-data-deliveries.

We had these built(and GLdev was building a rust version that tracked this, but he seems to have disappeared). I think it is worth exploring if these should be in the standard some how with the publisher able to give the subscriber a way to specify the behavior.

It might be as easy as:

type Event = record {
    event_id : Nat;
    event_schema : text;
    event_type : text;
    created_at : nat64;
    ack: bool;
    data : vec nat8;
};

  type Publisher = service {
    icrc12_subscribe : (handler : (PublisherMessage) → ()) → (SubscriptionResponse);
    icrc12_unsubscribe : (handler : (PublisherMessage) → ()) → (SubscriptionResponse);
    icrc12_ack : (Nat, Bool) → ();
  }

and

  type Subscriber  = service : {
      icrc12_handler: (PublisherMessage) -> ();
      icrc12_fin: (Nat) → ();
  }

If the publisher sends ack = true it expects you to call icrc12_ack or it will try again.

If the subscriber acks with the Bool true then it expects the publisher to call icrc12_fin or it will try to ack again.

Maybe there is a better place to put these than in the Event.

[SuddenlyHazel]

Thanks for sharing here! Looks like some great ideas in here tbh.

[SuddenlyHazel]

The canister interface is exposed via Candid, unfortunately, there isn't an ORB like naming service and we have no clue what the service would do when exposed.

Ahh, I see what you're saying now, thank you. I think this definitely makes sense at the routing layer; especially if we start aggregating events.

[fredericrous]

linking the following ic rust library here: https://github.com/seniorjoinu/ic-event-hub
I haven't tried it yet but it seems well thought. It comes with tutorials and examples