Bug: idempotency utility throws IdempotencyPersistenceLayerError - Failed to update success record to idempotency store

[cbarlow1993]

Expected Behavior

Successful idempotency write of successful request.

Current Behavior

After a successful write to the DynamoDB table for the idempotency record, once the handler has successfully executed with no errors thrown, it then throws the below error as the last log line of the execution.

It does successfully run the DeleteItemCommand hust before throwing this error and have verified with setting the dynamoDB client with a logger.

{
    "errorType": "IdempotencyPersistenceLayerError",
    "errorMessage": "Failed to update success record to idempotency store",
    "name": "IdempotencyPersistenceLayerError",
    "stack": [
        "IdempotencyPersistenceLayerError: Failed to update success record to idempotency store",
        "    at #saveSuccessfullResult (/var/task/handler.js:210624:13)",
        "    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",
        "    at async IdempotencyHandler.handleMiddyAfter (/var/task/handler.js:210554:5)",
        "    at async after (/var/task/handler.js:210922:5)",
        "    at async runMiddlewares (/var/task/handler.js:202652:17)",
        "    at async runRequest (/var/task/handler.js:202627:7)"
    ]
}

Code snippet

I'm using a HTTP V2 lambda event, expecting to use an idempotency key of the header of header."idempotency-key". This seems to work.

import { IdempotencyConfig } from '@aws-lambda-powertools/idempotency';
import { DynamoDBPersistenceLayer } from '@aws-lambda-powertools/idempotency/dynamodb';

export const idempotencyPersistenceStore = new DynamoDBPersistenceLayer({
  tableName: 'idempotencyTable',
  clientConfig: {
    region: 'eu-west-1',
    logger: console,
  },
});

export const idempotencyConfig = new IdempotencyConfig({
  expiresAfterSeconds: 60 * 60 * 24, // 1 day
  eventKeyJmesPath: 'headers."idempotency-key"',
  throwOnNoIdempotencyKey: false,
});

export const create = middy(lambdaHandler)
  .use(inputOutputLogger({ logger: (request) => logger.info(request) }))
  .use(
    makeHandlerIdempotent({
      persistenceStore: idempotencyPersistenceStore,
      config: idempotencyConfig,
    }),
  )

Steps to Reproduce

Use code snippet above

Possible Solution

Tried many different variations of the config and implementation and continuously the same error.

Powertools for AWS Lambda (TypeScript) version

latest

AWS Lambda function runtime

20.x

Packaging format used

npm

Execution logs

Initial write is successful:
{
  clientName: 'DynamoDBClient',
  commandName: 'PutItemCommand',
  input: {
    TableName: 'idempotencyTable',
    Item: {
      id: [Object],
      expiration: [Object],
      status: [Object],
      in_progress_expiration: [Object]
    },
    ExpressionAttributeNames: {
      '#id': 'id',
      '#expiry': 'expiration',
      '#in_progress_expiry': 'in_progress_expiration',
      '#status': 'status'
    },
    ExpressionAttributeValues: {
      ':now': [Object],
      ':now_in_millis': [Object],
      ':inprogress': [Object]
    },
    ConditionExpression: 'attribute_not_exists(#id) OR #expiry < :now OR (#status = :inprogress AND attribute_exists(#in_progress_expiry) AND #in_progress_expiry < :now_in_millis)',
    ReturnValuesOnConditionCheckFailure: 'ALL_OLD'
  },
  output: {},
  metadata: {
    httpStatusCode: 200,
    requestId: '5OIR7F73V0MVF2HDRCV9JOPS9NVV4KQNSO5AEMVJF66Q9ASUAAJG',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  }
}

My code continues to execute and is successfully returning a HTTP V2 response of { statusCode: 200, JSON.stringify({ result: true}) }

{
  clientName: 'DynamoDBClient',
  commandName: 'DeleteItemCommand',
  input: {
    TableName: 'idempotencyTable',
    Key: { id: [Object] }
  },
  output: {},
  metadata: {
    httpStatusCode: 200,
    requestId: 'EIM7659KJ9JBD9ABUIKJBDT567VV4KQNSO5AEMVJF66Q9ASUAAJG',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  }
}

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #typescript channel on our Powertools for AWS Lambda Discord: Invite link

[cbarlow1993]

To add some more info, I added in some logging:

    #saveSuccessfullResult = async (result) => {
        try {
            console.log(result)
            await this.#persistenceStore.saveSuccess(this.#functionPayloadToBeHashed, result);
        }
        catch (error) {
            console.log(error)
            throw new IdempotencyPersistenceLayerError('Failed to update success record to idempotency store', { cause: error });
        }
    };

Result to be saved:

{
  statusCode: 201,
  headers: {
    'Content-Type': 'application/json',
    'Access-Control-Allow-Origin': '*',
    'Cache-Control': 'no-store'
  },
  body: '{"id":"03de6e22-a7b7-4c25-9275-262696145c6b", "updatedAt":"2024-08-16T22:11:13.490159+00:00"}'
}

And then I think the DB util client errors with the following:

Error: Unsupported type passed: [object Object]. Pass options.convertClassInstanceToMap=true to marshall typeof object as map attribute.
    at convertToAttr (/var/runtime/node_modules/@aws-sdk/util-dynamodb/dist-cjs/index.js:133:9)
    at /var/runtime/node_modules/@aws-sdk/util-dynamodb/dist-cjs/index.js:197:20
    at convertToMapAttrFromEnumerableProps (/var/runtime/node_modules/@aws-sdk/util-dynamodb/dist-cjs/index.js:201:5)
    at convertToAttr (/var/runtime/node_modules/@aws-sdk/util-dynamodb/dist-cjs/index.js:111:12)
    at marshall (/var/runtime/node_modules/@aws-sdk/util-dynamodb/dist-cjs/index.js:307:26)
    at DynamoDBPersistenceLayer._updateRecord (/var/task/handler.js:211442:68)
    at DynamoDBPersistenceLayer.saveSuccess (/var/task/handler.js:211207:16)
    at #saveSuccessfullResult (/var/task/handler.js:210626:36)
    at IdempotencyHandler.handleMiddyAfter (/var/task/handler.js:210554:38)
    at after (/var/task/handler.js:210927:30) 

[dreamorosi]

Hi @cbarlow1993, thanks for the info - could you please share an example of what your lambda handler returns?

The error message seems to suggest you might be returning a class instance, which the SDK doesn't know how to serialize.

[cbarlow1993]

Yes, you are right. There is a class used format the response. If I just return a standard JSON payload, it does work correctly.

Is there a way to support class responses?

return new HttpResponseV2({
      statusCode: StatusCode.CREATED,
      additionalBodyData: new View(savedPayout),
    });

[dreamorosi]

Hi @cbarlow1993, thanks for confirming - that's most definitely the issue.

Technically speaking the underlying AWS SDK function we are using to serialize values before writing them to DynamoDB can support this type of conversion by enabling the convertClassInstanceToMap option.

When the convertClassInstanceToMap is enabled, class instances are serialized into a DynamoDB Map type.

For example, having this class:

class MyClass {
  public foo: string;
  public bar: string;

  constructor(props: { foo: string; bar: string }) {
    this.foo = props.foo;
    this.bar = props.bar;
  }

  public greet() {
    console.log(`hello ${this.foo}`);
  }
}

it would do this:

import { marshall } from '@aws-sdk/util-dynamodb';

const instance = new MyClass({ foo: 'foo', bar: 'bar' });
const marshalled = marshall(
  { id: 'abcd', value: instance },
  { convertClassInstanceToMap: true }
);

// { id: { S: 'abcd' }, value: { M: { foo: { S: 'foo' }, bar: { S: 'bar' } } } }

This would make sure the write operation is successful, which is nice, however it would break the idempotency further down the line.

The issue with this conversion is that it's a one-way lossy conversion, meaning that it's not possible to rehydrate or otherwise reconstruct the class instance from the map record stored in DynamoDB.

Building on top of the previous example, suppose we queried the value and unmarshalled it (aka convert it back from DynamoDB attribute to native JavaScript objects), the result would be this:

import { unmarshall } from '@aws-sdk/util-dynamodb';

const restored = unmarshall({ id: { S: 'abcd' }, value: { M: { foo: { S: 'foo' }, bar: { S: 'bar' } } } });

// { id: 'abcd', value: { foo: 'foo', bar: 'bar' } }

As you can see the restored item is now a plain object and no longer an instance of the original class. This is because we have lost any information about the class when serializing it.

For this project specifically, I think this would be an undesirable behavior since the one of the key features of the Idempotency utility is to:

Ensure Lambda handler returns the same result when called with the same payload

If we allowed class instances to be serialized, we would break this contract since we wouldn't be able to return the same stored payload on subsequent idempotent invocations/requests.

This is also a known limitation of the utility that is called out in the docs (although it could perhaps be called out more prominently):

---

Now, back to your use case, I don't think this is what you'd want either. I don't know what your HttpResponseV2 and View classes do, but given that you've mentioned that this is an API Gateway response I'm assuming that at some point either of the two will serialize the response to a mime type supported by the service, since you can't return a class instance.

If that's the case, if possible, I'd like to learn more about how/when this serialization happens so we can try to find a solution for your use case that would allow you to continue using our feature. Either way I think we'll have to make some changes to support this use case, so far I think allowing you to pass your own serializer function similar to how the Python implementation of Powertools for AWS does, and/or supporting response hooks (#2887) would probably help you.