SQS Queue type-safe integration class

[sam-goodwin]

Provide a wrapper class, Queue<T>, that models the SQS Queue Functionless integrations.

Must integrates with the following:

1. AppSync

const queue = new Queue<string>(new aws_sqs.Queue(..))

new AppsyncResolver<{id: string}>($context => {
  queue.sendMessage({
     MessageBody: id,
     MessageAttributes: ..
  });
});

2. Step Functions

new StepFunction(scope, id, (id: string) => {
  queue.sendMessage({
     MessageBody: id,
     MessageAttributes: ..
  })
})

3. Event Bridge

bus
  .when(scope, "rule", event => event.source === "lambda")
  .map(event => event.message)
  .pipe(queue)

4. Lambda

new Function(scope, id, async () => {
  await queue.sendMessage(..);
});

[sam-goodwin]

How should we go about layering this?

1. L1 Queue - no type information, all APIs use the vanilla AWS SDK APIs, e.g. MessageBody: string
2. L2 Queue<T> - captures type information and provides serialization/deserialization utility?

For supporting an arbitrary T, we will need some Codec<T> type that encapsulates serialization/deserialization logic.

export interface ReadCodec<Value, Data> {
  read(data: Data): Value;
}
export interface WriteCodec<Value, Data> {
  write(value: Value): Data;
}
export interface Codec<Value, Data>
  extends ReadCodec<Value, Data>,
    WriteCodec<Value, Data> {}

export class JsonCodec<T> implements Codec<T, string> {
  public read(data: string): T {
    return JSON.parse(data);
  }
  public write(value: T): string {
    return JSON.stringify(value);
  }
}

[thantos]

Could we do the same contract as secret?

Textqueue and jsonqueue?

Roll parse into the integration?

[sam-goodwin]

Could we do the same contract as secret?

Textqueue and jsonqueue?

Roll parse into the integration?

I was thinking the same, although we may need to consider cases like Avro, ProtoBuf, Flatbuffers, etc. So i was wondering if we could have a Codec abstraction - quickly run into limitations with closure serialization and this though.

[sam-goodwin]

That said, JsonSecret likely covers 99% of cases ha

[thantos]

Keep it simple? Anyone can make another overload and more l2s?

[sam-goodwin]

for JsonQueue<T>, where should we place the parsed value?

interface SQSRecord<T> {
  body: string; // <- this is the vanilla AWS API
  value: T; // <- maybe we add a new field? Or should we override body?
}

[thantos]

Make it explicit to not polite the sfn state?

[sam-goodwin]

Make it explicit to not polite the sfn state?

I don't know what you mean?

[sam-goodwin]

Hmm, if this is the interface

public forEach(
    handler: Function<SQSEvent<T>, lambda.SQSBatchResponse>,
    props?: aws_lambda_event_sources.SqsEventSourceProps
  ): void {
    handler.resource.addEventSource(
      new aws_lambda_event_sources.SqsEventSource(this.resource, props)
    );
  }

And we want to inject code that parses the body string, how would we inject that into the Function?

Should we instead change the interface to be

public forEach(scope: Construct, id: string, props: FunctionProps & SqsEventSourceProps, handle: (event: ..) => ..)

And then have the forEach instantiate the Function?

[sam-goodwin]

Yeah, I think this is better! We can provide an escape hatch if folk want to pass a pre-existing function

[thantos]

Looks good to me, and make the scope optional?

[sam-goodwin]

Looks good to me, and make the scope optional?

Yep, even id can default to "forEach"

[sam-goodwin]

Looks good to me, and make the scope optional?

Yep, even id can default to "forEach"

Which would work in 99% of cases because it's not like you'd ever attach two consumers

[thantos]

let n = 0;
let name = "forEach";
do {
   const child = queue.node.tryFindChild(name);
   if(!child) {
      break;
   }
  name = `forEach${++n}`;
} while(true);

[sam-goodwin]

let n = 0;

let name = "forEach";

do {

   const child = queue.node.tryFindChild(name);

   if(!child) {

      break;

   }

  name = `forEach${++n}`;

} while(true);

Sure that would work. But it has risks because of you remove one in the middle they all change.

[thantos]

Uhhgg, yeah, good point. Lets enforce the singleton when the automatic name is used?