Design of Input/Ouptut component of Kuai runtime

[Keith-CY]

Runtime of Kuai has been divided into 5 components(#7) and this issue is to elaborate the architecture and technical design of the Input/Output component, better to include use cases, interfaces, MVP or PoC description, internal architecture, and technical design.

Feel free to add any updates in this issue and the final document will be appended in the top message.

[homura]

Kuai Input/Output Layer

Input/Output is an abstract layer between Kuai's business logic and external entities, which may be a dapp user or CKB.

sequenceDiagram
  participant client as UI/User
  participant kuai as Kuai I/O
  participant ckb as CKB
  
  Note over client,kuai: Input/Output(as HTTP Server)
  kuai -->> ckb: listen
  client ->> kuai: action
  kuai ->> client: unsignedMsg
  client ->> kuai: action + signedMsg
  
  Note over kuai,ckb: Output(as Tx Sender)
  kuai ->> ckb: signedTx

  Note over kuai,ckb: Input(as Listener)
  ckb -->> kuai: txComiitted  

Loading

Builtins Input/Output Layer Components

Listeners

A listener is an input-only component that is used to input data to Kuai without having to return a value.

It can be used to listen to CKB nodes, to listen to alert services, or to listen to price changes from a centralized exchange, etc.

interface Listener<T> {
  on(listen: (obj: T) => void): void;
}

// built-in listeners
interface TipHeaderListener extends Listener {}
interface BlockListener extends Listener {}
interface TransactionListener extends Listener {}
interface ScriptListener extends Listener {}

CKB Provider

The CKB Provider is used to interact with CKB nodes

interface CellCollector {
  // An API similar to the RXJS operator, 
  // but with the steeper learning curve of RXJS, 
  // Kuai can provide a similar API directly 
  // without requiring developers to learn RXJS from scratch
  takeWhileAggregate(
    initialValue: T,
    aggregate: (aggr: T, cell: Cell, currentIndex: number) => T,
    takeWhile: (aggr: T, currentIndex: number) => boolean,
    excludeLast?: boolean,
  ): Promise<T>;
}

interface CkbProvider extends Rpc {
  getFeeRate(): Promise<BI>;
  collector(searchKey: SearchKey): CellCollector;
}

Middleware System

Koa's middleware system is intuitive to developers, and we can design Kuai's middleware to mimic Koa's middleware system

flowchart LR
  m1[middleware_1]
  m2[middleware_2]
  
  app --dispatch--> m1
  m1 --> |next| m2
  m1 -..-> |err| app
  m2 --> |ok| app

Loading

// `CoR` (Chain of Responsibility), abbr for chain of responsibility (maybe that's not a good name)
// a module that strings middleware together in order
interface Cor {  
  use(plugin: Middleware): void;  
  dispatch<Payload, Ok>(payload: Payload): Promise<Ok>;
}

type Middleware = (ctx: Context, next: () => Promise<void>) => Promise<void>;

interface Context {
  payload: JsonValue;  
  
  ok(): void;  
  ok<OK>(x: OK): void;  
  
  err(): void;  
  err<Err>(x: Err): void;  
}

type JsonValue = null | number | string | { [key: string]: JsonValue } | JsonValue[];

Examples

Basic - Give Me Five

declare function isThirteenPayload(x: unknown): x is { type: 'is-thirteen'; value: unknown };  
declare function isFivePayload(x: unknown): x is { type: 'is-five'; value: unknown };  
  
cor.use(async (ctx, next) => {  
  const payload = ctx.payload;

  // route like handler here
  if (isThirteenPayload(payload)) {  
    if (payload.value === 13) ctx.ok();  
    else ctx.err();
  // the second route
  } else if (isFivePayload(payload)) {  
    if (payload.value === 5) ctx.ok('gives you five');  
    else ctx.err('please give me five');  
  } else {  
    await next();  
  }  
});


cor.dispatch({ type: 'is-five', value: 6 })
   .then(console.log, console.error)
                    //      👆
                    // please give me five

Logger Plugin

This example shows how to write a logger plugin that logs the time

const tracer: Middleware = async (ctx, next) => {  
  const tracerId = crypto.randomUUID();  
  ctx.tracer.id = tracerId;  
  
  console.log(Date.now(), tracerId, ctx.payload);  
  await next();  
  console.log(Date.now(), tracerId, ctx.payload);  
};  
  
cor.use(tracer);

Works with TypeScript

To work better with TypeScript, developers can use the declare module method to extend the definition of Context

declare module '@kuaiproject/io' {  
  interface Context {  
    ckbProvider: CkbProvider;  
  }  
}

declare const ckbProviderPlugin: MIddleware;

cor.use(ckbProviderPlugin)
cor.use(async (ctx) => {
  const blockNumber = await ctx.ckbProvider.getBlockNumber()
  // ...
});

Works with an Event Listener

declare const blockTipChanged: Listener<number>;

blockTipChanged.on((tipNumber) => {
  cor.dispatch({
    type: 'listener/BLOCK_TIP_CHANGED',
    value: tipNumber,
  })
})

Enhance ok/err

cor.use(async (ctx, next) => {
  const ok = ctx.ok;
  const err= tx.err;

  if (payload && payload.type === 'SOME_TYPE') {
    ctx.ok = compose(ok, enhanceResult);
    ctx.err = compose(err, enhanceErr)
  }

  await next()
})

Integration with Other Frameworks

Kuai middleware can be integrated with other web frameworks in the form of a convention + Adapter

graph LR
  payload["payload.type get/nfts"]
  payload --adaptToExpress--> express["express :: app.get('/nfts', ...)"]
  payload --adaptToKoaRouter--> koa["koa :: koaRouter.get('/nfts', ...)"]
  payload --adaptToFastify--> fastify["fastify :: fastify.get('/nfts', ...)"]

Loading

adaptToExpress(app /* express instance*/, cor)
adaptToKoa(app /* koa instance*/, cor)

[Keith-CY]

Is the design of IO component ready?

[homura]

Is the design of IO component ready?

Yeah, I think it's ready

[homura]

To make it easier to see the change history, I put the design in the repo as a PR #18

[Keith-CY]

The illustration of middleware system might be incorrect if it's inspired by Koa

Koa adopts an onion model in its middleware system, where the messages flow bidirectionally between middlewares

[homura]

Koa adopts an onion model in its middleware system, where the messages flow bidirectionally between middlewares

So maybe Kuai's middleware system should provide a built-in middleware like this?

interface Context {
  onionResult: unknown;
}

const onionModelMiddleware: Middleware = async (ctx, next) => {
  const oritinalOk = ctx.ok;
  ctx.ok = (value) => {
    ctx.onionResult = value;
  }
  await next();
  
  oritinalOk(ctx.onionResult);
}

// place the middleware first
cor.use(onionModelMiddleware);
cor.use(otherMiddleware);

[Keith-CY]

Koa adopts an onion model in its middleware system, where the messages flow bidirectionally between middlewares

So maybe Kuai's middleware system should provide a built-in middleware like this?

interface Context {
  onionResult: unknown;
}

const onionModelMiddleware: Middleware = async (ctx, next) => {
  const oritinalOk = ctx.ok;
  ctx.ok = (value) => {
    ctx.onionResult = value;
  }
  await next();
  
  oritinalOk(ctx.onionResult);
}

// place the middleware first
cor.use(onionModelMiddleware);
cor.use(otherMiddleware);

It would be decided on your investigation and design, the point I mentioned here is that

flowchart LR
  m1[middleware_1]
  m2[middleware_2]
  
  app --dispatch--> m1
  m1 --> |next| m2
  m1 -..-> |err| app
  m2 --> |ok| app

Loading

is inconsistent with

Koa's middleware system is intuitive to developers, and we can design Kuai's middleware to mimic Koa's middleware system