Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Help importing and building #288

Open
wz2b opened this issue Nov 8, 2022 · 14 comments
Open

Help importing and building #288

wz2b opened this issue Nov 8, 2022 · 14 comments

Comments

@wz2b
Copy link

wz2b commented Nov 8, 2022

I'm having trouble building using this:

"dependencies": {
    "@aws-sdk/client-dynamodb": "^3.204.0",
    "@aws-sdk/lib-dynamodb": "^3.204.0",
    "@typedorm/common": "^1.15.1",
    "@typedorm/core": "^1.15.1",
    "reflect-metadata": "^0.1.13",
},
"devDependencies": {
    "@aws-sdk/smithy-client": "^3.201.0",
    "@aws-sdk/types": "^3.201.0",
}

It insisted I install smithy-client, which I didn't note in the docs. To make a connection, I'm doing this:

import {createConnection} from '@typedorm/core';
import {DocumentClientV3} from '@typedorm/document-client';
import {DynamoDBClient} from '@aws-sdk/client-dynamodb';

const documentClient = new DocumentClientV3(new DynamoDBClient({}));
const connection = createConnection({
    entities: [WashnetEquipment],
    documentClient: documentClient, // <-- When documentClient is not provided, TypeDORM defaults to use the DocumentClientV2
});

I get two errors I don't know how to resolve:

$ tsc
node_modules/@typedorm/document-client/cjs/src/types/document-client-types.d.ts:1:8 - error TS1192: Module '"C:/work/myproject/node_modules/aws-sdk/index"' has no default export.

1 import AWS, { DynamoDB } from 'aws-sdk';
         ~~~

node_modules/@typedorm/document-client/cjs/src/types/document-client-types.d.ts:2:8 - error TS1192: Module '"C:/work/myproject/node_modules/@aws-sdk/client-dynamodb/dist-types/index"' has no default export.

2 import DynamoDBClientV3 from '@aws-sdk/client-dynamodb';
         ~~~~~~~~~~~~~~~~

For the first error, I'm not sure why it needs to bring that dependency in in the first place if I'm doing V3. For the second error, that's correct, the client isn't called DynamoDBClientV3, it's just called DynamoDBClient.

I'm not sure what's going on here, is it a versioning problem? I don't really want to bring aws-sdk at all. If I don't, I get some warnings:

warning " > @typedorm/core@1.15.1" has unmet peer dependency "aws-sdk@^2.799.0".
warning " > @typedorm/document-client@1.15.1" has unmet peer dependency "aws-sdk@^2.799.0".

even if I do bring aws-sdk in at 2.799 or the most recent version I still get the error that it doesn't export "AWS"

I'm not sure how to clear this up.

@wz2b
Copy link
Author

wz2b commented Nov 8, 2022

I tried this without using the document client from typedorm, not sure if this should work or not:

import {DynamoDBClient} from '@aws-sdk/client-dynamodb';
import {DynamoDBDocumentClient} from "@aws-sdk/lib-dynamodb";
import {createConnection} from '@typedorm/core';

const client = new DynamoDBClient({});
const documentClient = DynamoDBDocumentClient.from(client);
const connection = createConnection({
    entities: [WashnetEquipment],
    documentClient: documentClient,
});

but regardless, I still get the same errors from document-client-types.d.ts 1.15.1:

import AWS, { DynamoDB } from 'aws-sdk';
TS1192: Module has no default export.

import DynamoDBClientV3 from '@aws-sdk/client-dynamodb';
TS1192: Module has no default export.

@whimzyLive
Copy link
Member

Hey @wz2b ,

Document client from typedorm/document-client is required when working with SDK v3.

here is a working example of using Typedorm with typescript, can you see if there is something missing in your case??

@wz2b
Copy link
Author

wz2b commented Nov 9, 2022

I had @typedorm/document-client in there explicitly but it seemed to be present anyway so I took it out. I put it back, but that didn't seem to make a difference. The two main problems are in document-client-types, actually:

import AWS, { DynamoDB } from 'aws-sdk';
import DynamoDBClientV3 from '@aws-sdk/client-dynamodb';

'aws-sdk' shouldn't be imported at all, as I'm using v3 ... and it exports a bunch of stuff and doesn't have a default export. And @aws-sdk/client-dynamodb doesn't export DynamoDBClientV3 - in fact, I can't find that symbol anywhere. Where does DynamoDBClientV3 come from?

I'm building this project as part of a CDK stack. What has me concerned is that some other (perhaps transitive) dependency is being noticed and telling it to try to import aws-sdk.

Let me take a closer look at the sample you referenced.

{
  "name": "stack-aws",
  "version": "0.1.0",
  "license": "UNLICENSED",
  "bin": {
    "braun-aws": "bin/braun-aws.js"
  },
  "scripts": {
    "build": "tsc",
    "watch": "tsc -w",
    "type-check": "tsc --project tsconfig.json --pretty --noEmit",
    "test": "jest",
    "cdk": "cdk"
  },
  "devDependencies": {
    "@aws-cdk/aws-apigatewayv2-alpha": "^2.50.0-alpha.0",
    "@aws-cdk/aws-apigatewayv2-authorizers-alpha": "^2.50.0-alpha.0",
    "@aws-cdk/aws-apigatewayv2-integrations-alpha": "^2.50.0-alpha.0",
    "@aws-cdk/aws-iot-actions-alpha": "^2.50.0-alpha.0",
    "@aws-cdk/aws-iot-alpha": "^2.50.0-alpha.0",
    "@aws-cdk/aws-iotevents-alpha": "^2.50.0-alpha.0",
    "@aws-cdk/aws-kinesisfirehose-alpha": "^2.50.0-alpha.0",
    "@aws-sdk/smithy-client": "^3.201.0",
    "@aws-sdk/types": "^3.201.0",
    "@middy/core": "^3.6.2",
    "@types/aws-lambda": "^8.10.108",
    "@types/jest": "^27.5.2",
    "@types/node": "10.17.27",
    "@types/prettier": "2.6.0",
    "aws-cdk": "2.47.0",
    "aws-cdk-lib": "2.50.0",
    "aws-sdk": "^2.1249.0",
    "esbuild": "0",
    "jest": "^27.5.1",
    "ts-jest": "^27.1.4",
    "ts-node": "^10.9.1",
    "typescript": "4.8.4"
  },
  "dependencies": {
    "@aws-lambda-powertools/commons": "^1.4.0",
    "@aws-lambda-powertools/logger": "^1.4.0",
    "@aws-lambda-powertools/metrics": "^1.4.0",
    "@aws-lambda-powertools/tracer": "^1.4.0",
    "@aws-sdk/client-dynamodb": "^3.204.0",
    "@aws-sdk/client-iot": "^3.204.0",
    "@aws-sdk/lib-dynamodb": "^3.204.0",
    "@typedorm/common": "^1.15.1",
    "@typedorm/core": "^1.15.1",
    "@typedorm/document-client": "^1.15.1",
    "aws-jwt-verify": "^3.1.0",
    "constructs": "^10.0.0",
    "node-fetch": "^3.2.10",
    "reflect-metadata": "^0.1.13",
    "source-map-support": "^0.5.21"
  },
  "peerDependencies": {}
}

@wz2b
Copy link
Author

wz2b commented Nov 9, 2022

I have a few differencse in tsconfig.json could that be a problem?

    "module": "commonjs",

is the one that has me a little concerned because I'm not sure how the magic works to support both v2 and v3

@wz2b
Copy link
Author

wz2b commented Nov 9, 2022

Just for fun, I cloned
https://github.com/typedorm/typedorm-examples
without modifying it at all, into a clean directory not at the same path, then:

$ yarn install && cd examples/aws-sdk-v3-typescript-webpack && yarn build

C:\work\gabraun\braun-aws\src\sites>yarn build
yarn run v1.22.11
$ tsc
node_modules/@typedorm/document-client/cjs/src/types/document-client-types.d.ts:1:8 - error TS1192: Module '"C:/work/gabraun/braun-aws/node_modules/aws-sdk/index"' has no default export.

1 import AWS, { DynamoDB } from 'aws-sdk';
         ~~~

node_modules/@typedorm/document-client/cjs/src/types/document-client-types.d.ts:2:8 - error TS1192: Module '"C:/work/gabraun/braun-aws/src/sites/node_modules/@aws-sdk/client-dynamodb/dist-types/index"' has no default export.

2 import DynamoDBClientV3 from '@aws-sdk/client-dynamodb';
         ~~~~~~~~~~~~~~~~


Found 2 errors in the same file, starting at: node_modules/@typedorm/document-client/cjs/src/types/document-client-types.d.ts:1

error Command failed with exit code 2.

I didn't think it would make a difference that I'm using yarn rather than npm, but just to be sure, I wiped the project and started again with npm, and it did the same. That makes sense since the errors are just coming from tsc.

I also tried building at the root ... so three commands:

$ git clone https://github.com/typedorm/typedorm-examples
$ npm install
$ npm run compile


examples/aws-sdk-v2-typescript-webpack/entities/organisation.entity.ts:15:14 - error TS1219: Experimental support for decorators is a feature that is subject to change in a future release. Set the 'experimentalDecorators' option in your 'tsconfig' or 'jsconfig' to remove this warning.

15 export class Organisation {
                ~~~~~~~~~~~~

examples/aws-sdk-v2-typescript-webpack/entities/organisation.entity.ts:19:3 - error TS1219: Experimental support for decorators is a feature that is subject to change in a future release. Set the 'experimentalDecorators' option in your 'tsconfig' or 'jsconfig' to remove this warning.

19   id!: string;
     ~~

examples/aws-sdk-v2-typescript-webpack/entities/organisation.entity.ts:22:3 - error TS1219: Experimental support for decorators is a feature that is subject to change in a future release. Set the 'experimentalDecorators' option in your 'tsconfig' or 'jsconfig' to remove this warning.

22   name!: string;
     ~~~~

examples/aws-sdk-v3-typescript-webpack/entities/organisation.entity.ts:15:14 - error TS1219: Experimental support for decorators is a feature that is subject to change in a future release. Set the 'experimentalDecorators' option in your 'tsconfig' or 'jsconfig' to remove this warning.

15 export class Organisation {
                ~~~~~~~~~~~~

examples/aws-sdk-v3-typescript-webpack/entities/organisation.entity.ts:19:3 - error TS1219: Experimental support for decorators is a feature that is subject to change in a future release. Set the 'experimentalDecorators' option in your 'tsconfig' or 'jsconfig' to remove this warning.

19   id!: string;
     ~~

examples/aws-sdk-v3-typescript-webpack/entities/organisation.entity.ts:22:3 - error TS1219: Experimental support for decorators is a feature that is subject to change in a future release. Set the 'experimentalDecorators' option in your 'tsconfig' or 'jsconfig' to remove this warning.

22   name!: string;
     ~~~~

node_modules/@aws-sdk/util-dynamodb/dist-types/models.d.ts:19:59 - error TS2304: Cannot find name 'Blob'.

19 export declare type NativeAttributeBinary = ArrayBuffer | Blob | Buffer | DataView | File | Int8Array | Uint8Array | Uint8ClampedArray | Int16Array | Uint16Array | Int32Array | Uint32Array | Float32Array | Float64Array | BigInt64Array | BigUint64Array;
                                                             ~~~~

node_modules/@aws-sdk/util-dynamodb/dist-types/models.d.ts:19:86 - error TS2304: Cannot find name 'File'.

19 export declare type NativeAttributeBinary = ArrayBuffer | Blob | Buffer | DataView | File | Int8Array | Uint8Array | Uint8ClampedArray | Int16Array | Uint16Array | Int32Array | Uint32Array | Float32Array | Float64Array | BigInt64Array | BigUint64Array;
                                                                                        ~~~~


Found 8 errors in 3 files.

Errors  Files
     3  examples/aws-sdk-v2-typescript-webpack/entities/organisation.entity.ts:15
     3  examples/aws-sdk-v3-typescript-webpack/entities/organisation.entity.ts:15
     2  node_modules/@aws-sdk/util-dynamodb/dist-types/models.d.ts:19

@wz2b
Copy link
Author

wz2b commented Nov 9, 2022

Sorry for the monolog here. Working through this, I'm trying first to track down where thie symbol DynamoDBClientV3 comes from, as one of the two errors is trying to import that. In all of Github there are only a handful, less if you eliminate things not in the exact case:

https://github.com/search?q=DynamoDBClientV3&type=code

which didn't reveal much, so then I did this:

https://github.com/search?q=DynamoDBClientV3&type=issues

which pointed me to this:

#265

that I think is my problem:

https://www.typescriptlang.org/tsconfig#allowSyntheticDefaultImports

I don't understand these comments: #265 (comment) and #265 (comment)

but a few conclusions here:

  • allowSyntheticDefaultImports does seem to have fixed the problem
  • I am indeed using esbuild, despite the above comments, it seems to compile - am I in for problems because of this?

The reason I am using esbuild is because I'm working within the CDK environment. If you don't use esbuild but you're using CDK, then it needs docker to work, which makes things slower and makes it virtually impossible to build on windows (I flip back and forth between windows and linux depending on what I'm doing).

@wz2b
Copy link
Author

wz2b commented Nov 9, 2022

I probably should have said when I started all this, my goal here is to build a lambda using typedorm within a CDK stack build. That said, there may be other issues beyond the above. Example:

2022-11-09T17:59:52.277Z	undefined	ERROR	Uncaught Exception 	
{
    "errorType": "MissingReflectMetadata",
    "errorMessage": "Could not reflect metadata of type design:type, did you forget to enable \"emitDecoratorMetadata\" on compiler options?",
    "name": "MissingReflectMetadata",
    "message": "Could not reflect metadata of type design:type, did you forget to enable \"emitDecoratorMetadata\" on compiler options?"
}

but I definitely did set emitDecoratorMetadata to true. Does this look right?

const EquipmentTable = new Table({
    name: "equipment-table",  /* this is the actual table name in dynamodb */
    partitionKey: "siteId",
    sortKey: "equipmentPath"
});

@Entity<MyEquipment >(
    {
        table: EquipmentTable,
        name: 'equipment',
        primaryKey: {
            partitionKey: '{{siteId}}',
            sortKey: '{{systemId}}#{{machineId}}',
        }
    }
)
export class MyEquipment {
    @Attribute()
    siteId: string;

    @Attribute()
    systemId: string | undefined;

    @Attribute()
    machineId: string | undefined;
}

The generated output of tsc includes:

var __decorate = (this && this.__decorate) || function (decorators, target, key, desc) {
    var c = arguments.length, r = c < 3 ? target : desc === null ? desc = Object.getOwnPropertyDescriptor(target, key) : desc, d;
    if (typeof Reflect === "object" && typeof Reflect.decorate === "function") r = Reflect.decorate(decorators, target, key, desc);
    else for (var i = decorators.length - 1; i >= 0; i--) if (d = decorators[i]) r = (c < 3 ? d(r) : c > 3 ? d(target, key, r) : d(target, key)) || r;
    return c > 3 && r && Object.defineProperty(target, key, r), r;
};

MyEquipment= __decorate([
    (0, common_1.Entity)({
        table: EquipmentTable,
        name: 'equipment',
        primaryKey: {
            partitionKey: '{{siteId}}',
            sortKey: '{{systemId}}#{machineId}',
        }
    })
], MyEquipment);

so I think emitDecoratorMetadata did its job. Not sure what the problem is.

@michael-pont
Copy link
Contributor

@wz2b I can second the MissingReflectMetadata error

Wed Nov 09 23:16:05 UTC 2022 : Endpoint response body before transformations: {"errorType":"MissingReflectMetadata","errorMessage":"Could not reflect metadata of type design:type, did you forget to enable \"emitDecoratorMetadata\" on compiler options?","trace":["MissingReflectMetadata: Could not reflect metadata of type design:type, did you forget to enable \"emitDecoratorMetadata\" on compiler options?","    at /var/task/index.js:1312:13","    at __decorateClass (/var/task/index.js:35:24)","    at lib/entities/base.ts (/var/task/index.js:5639:5)","    at __init (/var/task/index.js:9:56)","    at lib/entities/user.ts (/var/task/index.js:45205:5)","    at __init (/var/task/index.js:9:56)","    at Object.<anonymous> (/var/task/index.js:94918:15)","    at Module._compile (node:internal/modules/cjs/loader:1105:14)","    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)","    at Module.load (node:internal/modules/cjs/loader:981:32)"]}
Wed Nov 09 23:16:05 UTC 2022 : Lambda execution failed with status 200 due to customer function error: Could not reflect metadata of type design:type, did you forget to enable "emitDecoratorMetadata" on compiler options?. Lambda request id: 33284bec-72e9-4d2f-820a-574cc44bc720

I just migrated a project from AWS SAM to CDK and am trying to deploy lambda functions that have Typedorm code.
I am using aws sdk v3 and the latest typedorm package versions. My 3 entity class files all have import "reflect-metadata"; at the top and my tsconfig has "emitDecoratorMetadata": true, "experimentalDecorators": true,. My lambda function has bundling: { preCompilation: true, // Emit decorator metadata fix goes here, },

Here is my package.json

{
  "name": "cdk-esbuild",
  "version": "0.1.0",
  "bin": {
    "cdk-esbuild": "bin/cdk-esbuild.js"
  },
  "scripts": {
    "build": "tsc",
    "watch": "tsc -w",
    "test": "jest",
    "cdk": "cdk"
  },
  "devDependencies": {
    "@aws-sdk/types": "^3.201.0",
    "@slack/bolt": "3.12.2",
    "@types/aws-lambda": "8.10.108",
    "@types/http-errors": "2.0.1",
    "@types/jest": "^29.2.1",
    "@types/node": "^18.11.9",
    "@types/prettier": "^2.7.1",
    "@types/source-map-support": "^0.5.6",
    "aws-cdk": "2.50.0",
    "esbuild": "0.15.13",
    "jest": "^29.2.2",
    "ts-jest": "^29.0.3",
    "ts-node": "^10.9.1",
    "typescript": "^4.7.4"
  },
  "dependencies": {
    "@aws-crypto/client-node": "^3.1.1",
    "@aws-sdk/client-cognito-identity-provider": "^3.204.0",
    "@aws-sdk/client-dynamodb": "^3.204.0",
    "@aws-sdk/client-lambda": "^3.204.0",
    "@aws-sdk/client-ssm": "^3.205.0",
    "@aws-sdk/lib-dynamodb": "^3.204.0",
    "@aws-sdk/util-dynamodb": "^3.204.0",
    "@middy/core": "^3.2.0",
    "@middy/event-normalizer": "^3.2.0",
    "@middy/http-error-handler": "^3.2.0",
    "@middy/http-event-normalizer": "^3.2.0",
    "@middy/http-header-normalizer": "^3.2.0",
    "@middy/http-json-body-parser": "^3.2.0",
    "@middy/http-partial-response": "^3.2.0",
    "@middy/http-response-serializer": "^3.2.0",
    "@middy/http-security-headers": "^3.2.0",
    "@slack/web-api": "^6.7.2",
    "@typedorm/common": "^1.15.1",
    "@typedorm/core": "^1.15.1",
    "@typedorm/document-client": "^1.15.1",
    "aws-cdk-lib": "^2.50.0",
    "axios": "^0.27.2",
    "constructs": "^10.1.147",
    "dayjs": "^1.11.5",
    "http-errors": "^2.0.0",
    "pino": "^7.11.0",
    "pino-lambda": "^4.0.0",
    "reflect-metadata": "^0.1.13",
    "slack-block-builder": "^2.6.0",
    "source-map-support": "0.5.21",
    "tsscmp": "^1.0.6"
  }
}

@whimzyLive I have seen your comments in aws-cdk issues regarding emitDecoratorMetadata and esbuild bundling lambdas so perhaps you have a better idea if we are configuring something wrong with regards to esbuild.

Looking forward to your insights
Michael

@wz2b
Copy link
Author

wz2b commented Nov 10, 2022

Yeah it doesn't seem to work right in the CDK environment. I asked this question on Repost to try to see if there's a way to have a lambda that's part of the project but has its own tsconfig, package.json, etc. I think that might be the better approach, I'm just not sure how to make it work.

@whimzyLive
Copy link
Member

whimzyLive commented Nov 12, 2022

Hey @wz2b amd @michael-pont

Yes, it is correct that the TypeDORM does not build in CDK environment out of the box, and the main reason behind that is to do with esBuild - the default bundler used by the CDK team for transpiling TS sources.

TypeDORM relies on reflection to get run-time types information which is an experimental feature in Typescript and not since the ESBuild is pretty new they have still not implemented it correctly. In contrast, other mature bundlers like webpack handle that flawlessly. Also just to be clear that this is not the issue with TypeDORM infact there are many other libs also affected by this behaviour of esbuild (which relies on reflection) like Nestjs, TypeORM.

So when the ESbuild is used to transpile code in CDK environment it drops all the type information from the final bundle which is required for the TypeDORM to work properly.
Until the ESBuild supports experimental metadata, you will have to rely on tsc or some other mature bundlers to correctly preserve reflection information in the final bundle.

@whimzyLive
Copy link
Member

The obvious workaround for getting the TypeDORM to work with CDK is to always rely on tsc the default typescript compiler to transpile ts -> js and then let ESBuild bundle js.

If one of you can set up an example of using TypeDORM with CDK under https://github.com/typedorm/typedorm-examples, I'll can help with adding in a workaround, so it is there for reference for anyone else that might be facing the same problem.

@michael-pont
Copy link
Contributor

@whimzyLive thanks. I submitted a PR here typedorm/typedorm-examples#1

@michael-pont
Copy link
Contributor

Hi @whimzyLive , have you been able to take a look at the PR?

@joshstrange
Copy link

joshstrange commented Dec 20, 2022

@michael-pont Am I missing something or does your PR not use esbuild? I see it as a devDependencies but I don't see it used anywhere. It looks like you are just using tsc to build your code.

I'm running into similar issues with trying to get TypeDORM to play nice with esbuild (using SST, "The Serverless Stack").

EDIT: I finally got this working. If anyone else is trying to use TypeDORM with SST do the following (paths are relative to root of project, NOT root of services folder:

  1. Create the follow file in /config/esbuild.js:
const {esbuildDecorators} = require("@anatine/esbuild-decorators");

module.exports = [
    esbuildDecorators({
        tsconfig: './services/tsconfig.json', // <- This was the missing piece for me, without this nothing worked
    }),
];
  1. Install @anatine/esbuild-decorators in the root of your project
npm install @anatine/esbuild-decorators
  1. Modify any place you reference functions in your stack(s) (Api, Queue consumers, Table streams, etc) to use the esbuild.js file

Api Example.

Before:

const api = new Api(stack, 'api', {
    routes: {
        'GET /': 'functions/lambda.handler'
    }
});

After:

const api = new Api(stack, 'api', {
    defaults: {
        function: {
            bundle: {
                esbuildConfig: {
                    plugins: 'config/esbuild.js'
                }
            }
        }
    },
    routes: {
        'GET /': 'functions/lambda.handler'
    }
});

For SEO purposes

  • Here was the error I was getting "ERROR Runtime.UnhandledPromiseRejection: TypeError: Class extends value undefined is not a constructor or null"

  • SST The Serverless Stack

  • esbuild emitDecoratorMetadata

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants