The typings on `marshall` are too narrow

[vivmaha]

Describe the bug
marshall from @aws-sdk/util-dynamodb only accepts data in the form of

{
    [key: string]: NativeAttributeValue;
}

This narrow typing prevents you from using marshall with:

1. typescript interfaces. If you use a typescript type it works.
2. arrays. The underlying convertToAttr properly supports this, so why not expose it up?

The v2 sdk's DocumentClient handled both of these cases, so I'm guessing marshall should do the same if parity is a factor.

SDK version number

1.0.0-rc.2

Is the issue in the browser/Node.js/ReactNative?
Node.js

Details of the browser/Node.js/ReactNative version
12

To Reproduce (observed behavior)

interface MyType = {
  foo: string;
  nested: {
    baz: string;
  };
};

const mySimpleObject: MyType = {
  foo: "foo",
  nested: {
    baz: "baz",
  },
};

const x = marshall(mySimpleObject);
^^^^ Argument of type 'MyType' is not assignable to parameter of type '{ [key: string]: NativeAttributeValue; }'.
  Index signature is missing in type 'MyType'.ts(2345)

Expected behavior
I should be able to pass in interface types into marshall. I should be able to also pass in []s.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context

[trivikr]

@vivmaha The DocumentClient in v2 had several bugs. For example, it stored a Date object as a map.
The types in marshall are narrower to avoid bugs like these.

We're exploring an option to provide premarshall configuration in #1533 (comment)

However, the interface support might not require a premarshall configuration and can be stored as a map. Marking this as feature request.

[stang-tgs]

The marshaller seems to be very selective on what it deems to be an object 😄 . It can't handle any custom classes and will fail on any named classes. Perhaps the marshaller should be doing typeof checks instead of constructor names? Or a combination of? However, this will have the same side effects as the v2 sdk. The bright side is that JS Sets exist now, so potentially that can be marshalled. JS Maps may be weird, because they don't look very different than Objects to Dynamo. 🤔

In any case, the marshaller is a little useless for custom objects at the moment; I'd say it's more of bug than feature because of the super narrowness for objects data?.constructor?.name === "Object"

[trivikr]

For reference: This TypeScript Playground shows the Type Error when passing Interface as an input to marshall function discussed in this issue.

[trivikr]

Verified that using as any works while passing interface type as an input provided that interface values confirm to NativeAttributeValue to unblock users while a solution is being investigated.

Example code

import { DynamoDB, waitForTableExists } from "@aws-sdk/client-dynamodb";
import { marshall } from "@aws-sdk/util-dynamodb";

interface TableItemIterface {
  id: string;
}

(async () => {
  const region = "us-west-2";
  const client = new DynamoDB({ region, logger: console });

  const TableName = `test-${Math.ceil(Math.random() * 10 ** 10)}`;

  const params = {
    TableName,
    AttributeDefinitions: [{ AttributeName: "id", AttributeType: "S" }],
    KeySchema: [{ AttributeName: "id", KeyType: "HASH" }],
    BillingMode: "PAY_PER_REQUEST",
  };

  await client.createTable(params);
  await waitForTableExists({ client, maxWaitTime: 60 }, { TableName });

  const itemContents: TableItemIterface = { id: "id" };
  await client.putItem({
    Item: marshall(itemContents as any),
    TableName,
  });

  await client.getItem({
    Key: marshall(itemContents as any),
    TableName,
  });

  await client.deleteTable({ TableName });
})();

Tested with v3.3.0

[trivikr]

I asked the question in StackOverflow for a solution to create a generic TypeScript interface which matches any type/interface/object with restrictions on types for values inside it.

While writing the question, I noticed that existing setup accepts types instead of interfaces as an input for marshall.

Example code

import { DynamoDB, waitForTableExists } from "@aws-sdk/client-dynamodb";
import { marshall } from "@aws-sdk/util-dynamodb";

type TableItemType = {
  id: string;
};

(async () => {
  const region = "us-west-2";
  const client = new DynamoDB({ region, logger: console });

  const TableName = `test-${Math.ceil(Math.random() * 10 ** 10)}`;

  const params = {
    TableName,
    AttributeDefinitions: [{ AttributeName: "id", AttributeType: "S" }],
    KeySchema: [{ AttributeName: "id", KeyType: "HASH" }],
    BillingMode: "PAY_PER_REQUEST",
  };

  await client.createTable(params);
  await waitForTableExists({ client, maxWaitTime: 60 }, { TableName });

  const itemContents: TableItemType = { id: "id" };
  await client.putItem({
    Item: marshall(itemContents),
    TableName,
  });

  await client.getItem({
    Key: marshall(itemContents),
    TableName,
  });

  await client.deleteTable({ TableName });
})();

[trivikr]

After going through StackOverflow Answer, and issue microsoft/TypeScript#15300 on TypeScript repo, the recommendation is to either:

• Use spread operator while passing interface object to marshall.
• Use Type instead of Interface.

Verified that spread operator works:

Example code

import { DynamoDB, waitForTableExists } from "@aws-sdk/client-dynamodb";
import { marshall } from "@aws-sdk/util-dynamodb";

interface TableItemType {
  id: string;
}

(async () => {
  const region = "us-west-2";
  const client = new DynamoDB({ region, logger: console });

  const TableName = `test-${Math.ceil(Math.random() * 10 ** 10)}`;

  const params = {
    TableName,
    AttributeDefinitions: [{ AttributeName: "id", AttributeType: "S" }],
    KeySchema: [{ AttributeName: "id", KeyType: "HASH" }],
    BillingMode: "PAY_PER_REQUEST",
  };

  await client.createTable(params);
  await waitForTableExists({ client, maxWaitTime: 60 }, { TableName });

  const itemContents: TableItemType = { id: "id" };
  await client.putItem({
    Item: marshall({ ...itemContents }),
    TableName,
  });

  await client.getItem({
    Key: marshall({ ...itemContents }),
    TableName,
  });

  await client.deleteTable({ TableName });
})();

Verified that TypeScript will still throw error is Interface contains values which are not allowed.
TypeScript Playground

[stang-tgs]

FYI - in our ported code, we already are using the spread operator before marshalling, to sidestep the current limitations of the marshaller not marshalling named types. However, note that the spread operator will not solve the case where there are nested types. However, #1533 should be able to solve the nesting issue for named types, if the marshaller allows an option to marshall v2 style.

[trivikr]

note that the spread operator will not solve the case where there are nested types.

Verified that the spread operator will not work on nested interfaces.

Example code

import { DynamoDB, waitForTableExists } from "@aws-sdk/client-dynamodb";
import { marshall } from "@aws-sdk/util-dynamodb";

interface NameInterface {
  firstname: string;
  lastname: string;
}

interface TableItemInterface {
  id: string;
  name: NameInterface;
}

(async () => {
  const region = "us-west-2";
  const client = new DynamoDB({ region });

  const TableName = `test-${Math.ceil(Math.random() * 10 ** 10)}`;

  const params = {
    TableName,
    AttributeDefinitions: [{ AttributeName: "id", AttributeType: "S" }],
    KeySchema: [{ AttributeName: "id", KeyType: "HASH" }],
    BillingMode: "PAY_PER_REQUEST",
  };

  await client.createTable(params);
  await waitForTableExists({ client, maxWaitTime: 60 }, { TableName });

  const itemContents: TableItemInterface = {
    id: "id",
    name: { firstname: "John", lastname: "Doe" },
  };
  await client.putItem({
    // TS error: Type 'NameInterface' is not assignable to type 'NativeAttributeValue'.
    Item: marshall({ ...itemContents }),
    TableName,
  });

  await client.getItem({
    Key: marshall({ id: itemContents.id }),
    TableName,
  });

  await client.deleteTable({ TableName });
})();

[trivikr]

The solution given in microsoft/TypeScript#15300 (comment) works with simple interfaces, but not nested ones.
TypeScript Playground

We can still implement it to allow passing non-nested interfaces as an input to marshall function.

Verified that the solution works with types - normal as well as nested.
TypeScript Playground

[trivikr]

An option to support nested interfaces would be to advise users to extend their interfaces from a generic interface like this:

import { NativeAttributeValue, INativeAttributeValue } from "@aws-sdk/util-dynamodb";

interface IPersonName extends INativeAttributeValue<string> {
  firstname: string;
  lastname: string;
}
interface IPersonNested extends INativeAttributeValue<NativeAttributeValue> {
  id: string;
  name: IPersonName;
}

where:

interface INativeAttributeValue<T> {
  [key: string]: T
}

TypeScript playground

This way, the customer code would be guaranteed to work with marshall function in @aws-sdk/util-dynamodb
What do you think @stang-tgs @vivmaha ?

[trivikr]

A draft PR to accept interface as an input to marshall is posted at #1952