[@aws-sdk/client-lambda] - Payload data in uint8array format - how to resolve data to expected output?

[NickLiffen]

Describe the bug

When using the new AWS Client Lambda NPM Module, and when invoking a lambda, data from Payload is returned in a Uint8Array. I can't find a way to resolve/get the data in the original JSON format that it should be?

Your environment

SDK version number

"@aws-sdk/client-lambda": "^3.12.0",

Is the issue in the browser/Node.js/ReactNative?

NodeJS

Details of the browser/Node.js/ReactNative version

Node 14.6.1

Steps to reproduce

Lambda A tried to invoke Lambda GITHUB_TOKEN_FUNCTION

const { LambdaClient, InvokeCommand } = require('@aws-sdk/client-lambda');

module.exports = {
  getGitHubToken: async () => {
    const client = new LambdaClient({ region: process.env.REGION });

    const params = {
      FunctionName: process.env.GITHUB_TOKEN_FUNCTION,
      LogType: 'Tail',
      Payload: '',
    };

    const command = new InvokeCommand(params);

    try {
      const { Payload } = await client.send(command);
      console.log(Payload)
      return str;
    } catch (error) {
      console.log(error.message);
      throw error;
    }
  },
};

GITHUB_TOKEN_FUNCTION should return this:

const res = {
      statusCode: 200,
      body: JSON.stringify({
        token,
      }),

However, when I console.log(Payload) I get this:

In the old module (v2) I never got this, I just got the data back as returned from the GITHUB_TOKEN_FUNCTION. Which should be:

const res = {
      statusCode: 200,
      body: JSON.stringify({
        token,
      }),

I have checked the CloudWatch logs for GITHUB_TOKEN_FUNCTION and it is successfully sending that object. Lambda A just receives it in a different format that I can't seem to get into the expected one.

Observed behavior

I get some sort of Unit8Array back that I can't find out how to turn into the data I expect.

Expected behavior

Payload ===

const res = {
      statusCode: 200,
      body: JSON.stringify({
        token,
      }),

My Question:

How do I resolve data from Unit8Array to the data I was expecting from the Lambda response? Which is a JSON Object?

[NickLiffen]

Okay, I got it working:

const asciiDecoder = new TextDecoder('ascii');
const data = asciiDecoder.decode(Payload);

A question, why is this module not doing this 🤔 / why is there not an option to do this? 🤔 The reason we use this module is to not have to interact with the apis and services natively. It would be good if this module handled it.

[NickLiffen]

Sorry to keep on moaning here but also it makes testing really hard 🤔 We use Jest and there are a bunch of problems with TextDecode and TextEncode. As found here and here.

[NickLiffen]

I have looked at the "style" of this library and I think It isn't intended for the module to "by default" decode for the user. I think when it comes to Lambda Invoke especially; it would be good if you could maybe pass in a boolean or something to:

    const params = {
      FunctionName: process.env.LAMBDA_ARN,
      LogType: 'Tail',
      Payload: '',
    };

called: decode: true/false.

The value of this module is the abstraction and I think this would really help out people using this module 👍 Just a thought.

[NickLiffen]

I see there is some sort of encoder interface, is this exposed by any chance to the client so I can run this within my code?

[plummet555]

I came across the same issue. It does seem odd that we have to decode and then re-parse the output like this:
const asciiDecoder = new TextDecoder('ascii');
const creds = JSON.parse(asciiDecoder.decode(data.Payload));

[ffxsam]

Just a slight correction to the above (for those using Node.js, where ascii doesn't work as an argument):

const asciiDecoder = new TextDecoder('utf-8');

This method works too, and requires a little less code:

const creds = JSON.parse(Buffer.from(data.Payload)); // if using TypeScript, you'll need to use Buffer.from(data.Payload).toString() to be type-safe

I agree, I'm not sure why the decision was made to use Uint8Array. It's really inconvenient.

[ajredniwja]

@NickLiffen thanks for opening this issue, apologies for late reply, I think this is the expected behavior for this SDK as you can see payload is now of type unit8array

aws-sdk-js-v3/clients/client-lambda/models/models_0.ts

Line 3490 in 3bd30bb

Payload?: Uint8Array;

.

I have verified that the @ffxsam code which seems to work.

With that being said, I think this behavior can be documented but I believe its not a bug. I will mark it as a documentation issue.

[dgurns]

Agreed with the above posters. In a JavaScript/TypeScript environment, being able to access the payload data as JSON, even by passing in a flag, would be a nice quality of life improvement.

[toto922]

This offical util can be uesd to decode and encode uint8array format ↓

https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/modules/_aws_sdk_util_utf8_node.html

https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/modules/_aws_sdk_util_utf8_browser.html

[sebastien-abbal]

For typescript, I don't have errors if I use that kind of payload :

lambda.invoke({
  FunctionName: 'my-lamba',
  InvocationType: 'RequestResponse',
  Payload: Buffer.from(JSON.stringify({ foo: 'bar' })),
});

Hope it will be usefull :)