Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add missing configuration/features to Smithy build plugin #356

Open
robin-aws opened this issue May 9, 2024 · 2 comments
Open

Add missing configuration/features to Smithy build plugin #356

robin-aws opened this issue May 9, 2024 · 2 comments
Assignees
@justplaz

Comments

@robin-aws
Copy link
Contributor

robin-aws commented May 9, 2024 •
edited

The number and complexity of CLI arguments are getting out of hand. We already have an alternate interface into the code generation logic as a Smithy build plugin, which supports configuration via a smithy-build.json. It also supports standard Smithy features such as projections because it hooks into the full Smithy model building pipeline.

This task is to add the missing necessary configuration knobs to our smithy-build.json configuration handler so that we can migrate off the CLI.

(More specific design for how these options should map to smithy-build.json configuration pending...)
@robin-aws robin-aws mentioned this issue May 9, 2024
Merged
robin-aws added a commit that referenced this issue May 10, 2024
@robin-aws
feat: Add support for smithy.test#smokeTests for Java (#355)
47f3345 
Issue #, if available:

Fixes #298

Description of changes:

This trait is ideal for filling a testing gap in our TestModels. Because our strategy for mapping Smithy shapes to Dafny types enforces constraints statically, it's not possible to express invalid input without lying to Dafny through axioms that aren't actually true, or in some cases (such as @required) not possible at all. smokeTests instances generate tests that specify inputs at the Smithy level using document types (i.e. more or less JSON valued arguments to the trait), which generate target language tests using the customer-facing interfaces such as builders to create input, and can therefore express invalid input as well.

The key new component in the implementation is the ModeledShapeValue.shapeValue method, which creates a code block in Java expressing the given Shape value. Similar logic already appears in other smithy code generators, although mostly for HTTP Protocol Compliance Tests instead. I mainly imitated the structure of smithy-python's logic, but was also inspired by smithy-typescript and smithy-rs.

Because this feature introduces a new category of generated code (Smithy-generated tests) this PR also adds an --output-java-test option with SmithyDafnyMakefile.mk support. We will likely need to add a similar parameter for the other languages. It may make more sense to migrate to smithy-build.json first though: #356.

It also adds ./gradlew test as an additional step for make test_java, since running the Java @tests is distinct from executing the transpiled dafny test program.

Finally, it also improves the CLI interface to print out validation events even if there are only warnings. Besides providing previously hidden but useful information in general, this provides a way to warn against using smokeTests outside of test models - since it doesn't/can't support @reference shapes it's not terribly useful as a testing method for local services themselves.
@robin-aws
Copy link
Contributor Author

Note the scope of this issue is only for CLI options that are relevant in --aws-sdk mode, since we only have the dafny-client-plugin plugin so far. Some options will only be needed for the dafny-library-plugin we have yet to create: #153

@robin-aws
Copy link
Contributor Author

robin-aws commented May 14, 2024 •
edited

Here is some more detail on the scope of this task vs. future ones.

One key difference between the CLI and the Smithy build plugin is how they use input and output. The CLI has options like --library-root pointing to the source code directory, and options like --output-java that allow it to write code anywhere on disk. The Smithy CLI/Gradle plugin only gives Smithy build plugins parsed/resolved/validated Smithy models as input, and expects output to only be written to a build directory like build/sources/dafny-client-plugin.

I believe the best way to bridge this gap is with an additional shared Gradle plugin that can be executed after invoking the smithy-gradle-plugin task to build a model. This plugin would provide the connection between the contents of a directory that the Smithy build plugin outputs like <project-root>/build/sources/dafny-client-plugin/runtimes/java/smithy-generated and where that code is checked in, i.e. <project-root>/runtimes/java/smithy-generated. The --patch-files-dir and --update-patch-files features of the CLI would also move to this plugin.

The plugin would have three different tasks (or at least modes of a single task):

  • assert-same - apply patch files, then assert that the two locations have the same content
  • update - apply patch files, then copy from the build directory on top of the checked-in directory
  • update-patch-files - don't apply patch file, update them to match the difference between the build directory and the checked-in copy

It looks like this is the only missing "feature" to make this work. For the record here is the analysis I did of the current set of CLI options:

CLI options that need new smithy-build.json equivalents

  • --patch-files-dir/--update-patch-files
    • Should be handled by post-processing

CLI options that already have smithy-build.json equivalents

  • --model
    • Maps to sources
  • --namespace
    • Maps roughly to service: the Smithy build plugin tells the code generation logic to generate for all shapes in the namespace of the identified service
  • --library-root
    • Always the plugin artifacts output directory (i.e. build/sources/dafny-client-codegen)
  • --dafny-version
    • Maps to plugins.dafny-client-codegen.dafnyVersion
  • --include-dafny
    • Maps to plugins.dafny-client-codegen.includeDafnyFile
  • --properties-file
    • Always set to <output directory>/project.properties
  • --output-dafny/java/dotnet/...
    • Target languages are selected via a plugins.dafny-client-codegen.targetLanguages list instead
    • Output locations are fixed for the Smithy build plugin, see above about post-processing
  • --aws-sdk
    • Implicitly always true for dafny-client-plugin
  • --java-aws-sdk-version
    • Intentionally not supported: defaults to V2, V1 not used

CLI options not yet relevant (because they will apply to dafny-library-codegen instead)

  • --dependent-model
  • --local-service-test

@justplaz justplaz mentioned this issue May 29, 2024
Merged
robin-aws pushed a commit that referenced this issue Jun 11, 2024
@justplaz
chore: add polymorphDafny and polymorphDotnet tasks to Smithy Build P…
b7cec71 
…lugin-based SQS TestModel (#395)

*Issue #, if available:* #356 (related)

*Description of changes:* This PR allows the Smithy Build Plugin-based SQS TestModel to be built and structured similarly to the Makefile/CLI based models used elsewhere. This enables some Smithy features which are not supported by the CLI, such as [Projections](https://smithy.io/2.0/guides/smithy-build-json.html#projections). 

Instead of running `make polymorph_dafny` and `make polymorph_dotnet`, run `./gradlew polymorphDafny` and `./gradlew polymorphDotnet` to generate types for the respective languages. A Makefile is included, as it is needed to run `make transpile_net` and `make test_net`, which work as expected when the above commands are executed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@justplaz justplaz

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@robin-aws @justplaz