[FEATURE] Make Smithy Build/Model a first class package

[codypenta]

Describe the feature

Can we pull Smithy Build and Model out of OpenAPIGateway and make it it's own project?

Use Case

Ive been using SmithyAPIGatewayTSProject to generate typescript API. Instead of generating code from an OpenAPI spec, I wanted to generate code from Smithy directly instead. The current solution has been:

• Just generate and ignore/delete the generated code for OpenAPI
• Use the generated SDKs instead

I keep running into a chicken and egg problem:

• Opinionated CDK Construct uses generated SDKs
• SDKs are generated within the same project

nx tries to find and build all subdirectories with a package.json. So what ends up happening is that the CDK construct tries to build before the model builds. You can get around this with some projen-fu by generated SDKs, manually running yarn install to install the net new dependencies the SDK needs, commenting out files, adding dependencies, uncommenting files, etc, etc etc.

It would be easier if the flow was:

packages/
|--model        # contains model and generated code
|--construct   # contains CDK constructs that depends on model
|--anything else down the road # depends on model

This way, all packages can depend on model, that way, whenever I run npx nx run-many --target build --all. nx will know to always build model first.

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

PDK version used

0.12.4

What languages will this feature affect?

No response

Environment details (OS name and version, etc.)

MacOS

[cogwirrel]

This sounds like a reasonable feature! Breaking it up into individual project types which you can use independently would be useful, though I think it's still important to keep the existing interface so that users can choose to use the more opinionated option to get up and running quickly if they're happy with it :)

Interesting though that you're experiencing an issue where the cdk construct is building before the model builds - the dependency should be set up automatically here: https://github.com/aws/aws-prototyping-sdk/blob/mainline/packages/open-api-gateway/src/project/open-api-gateway-ts-project.ts#L249-L250 - I believe NX should honour the dependency graph so long as each package is in the yarn workspace.

[codypenta]

Agreed on the first statement, maybe "pull out" wasn't the best terminology as not everybody wants to use Smithy code generation and would like to stick with the Open API opinionated construct.

For the second one, I think this has to do with projen and nx conflicting with each other for some reason and need to dive deeper into it. For example, cloning a repo and getting started works fine:

1. Clone the repo, then cd into it
2. yarn install
3. npx projen
4. npx nx run-many --target build --all

Running npx nx run-many --target build --all usually works as well. But there's certain oddities, like for example when I add a dependency to a new SmithyApiGatewayTsProject it intermittently fails with the same error discuss on another issue "failed to get @prototype/ssdk" because it's looking for the public repository first before locally (even when the package is in the workspace). It's not till I do some combination of yarn install at the root and npx nx run-many --target build --all again then will projen work.

The only thing I have seen is how node_modules are laid out. Everything works fine when the root nx monorepo has almost all of the node_modules at root (including the autogenerated SDK) and all subprojects usually have a node_modules that are just symlinks. But for some reason (still trying to determine reproducible steps), the subpackage api will instead download it's own node_modules (like it's the only repo that exist, which is why I think it has something to do with projen). It's not till I run a yarn install at the root that all of the sub packages node_modules are "hoisted" up to the nx monorepo root.

Note: I say almost all, because the generated SDK node_modules are not hoisted and shared all the way up to root. It instead just takes the entire directory (it's node modules and all) and hoisted it to root as a single unit. Not really a bad thing, just a weird thing

[NukaCody]

Reproducing steps (sent a link to an example repo):

1. Add a dependency to the project, then run projen
   1. An unexpected error occurred: "https://registry.yarnpkg.com/@prototype%2fsdk: Not found"
2. Run yarn install at the root
   1. Then run projen again and it works

[NukaCody]

The above error seems to be a result of an underlying limitation of projen and not the prototyping SDK. When projen runs the default task (I.e. synthesizes and download dependencies). It looks like it essentially 'cd's into the directory to perform the operations. Because the cwd is no longer the root monorepo, it doesn't know that the other package exist locally and keeps trying to go out to npm to find it.

Hoisting all of my dependencies to the top (in monorepo) solves the issue.

May be easier to orchestrate when Smithy code generation becomes its own package

[github-actions]

This issue is now marked as stale because it hasn't seen activity for a while. Add a comment or it will be closed soon. If you wish to exclude this issue from being marked as stale, add the "backlog" label.

[github-actions]

Closing this issue as it hasn't seen activity for a while. Please add a comment @mentioning a maintainer to reopen. If you wish to exclude this issue from being marked as stale, add the "backlog" label.

[cogwirrel]

This is now addressed by type-safe-api! This supersedes open-api-gateway and provides a more modular directory structure. It also exports TypeSafeApiModelProject which will create a project which contains the Smithy or OpenAPI model. The build task generates an openapi spec- the smithy one can be further customised to add more projections however (eg TypeScript server SDK).

The generated spec can then be used with the TypeSafeRestApi CDK construct if you want to deploy the api gateway resources, but the integrations won’t be type safe :). This should be enough to use the TypeScript server SDK instead of the handle wrappers provided by PDK :)

Note that you may need to add a TypeScriptProject for the shell of the TSSSDK that’s parented by the monorepo, and include a preCompile task to copy the generated code from “smithy-output” into its source directory- that way the project will be available at “install” time so yarn etc will find it, but the code is generated at build time :)