Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Adopt SmithyDafnyMakefile.mk, progress towards fixing nightly build #797

Merged
merged 42 commits into from
Apr 19, 2024

Conversation

robin-aws
Copy link
Contributor

@robin-aws robin-aws commented Mar 7, 2024

Description of changes:

Replaces nearly all of the SharedMakefile.mk with the common smithy-dafny/SmithyDafnyMakefile.mk makefile, just retaining configuration variables specific to this repo (such as the path to the smithy-dafny submodule). Uses the new features in that makefile and smithy-dafny itself to make the projects forwards-compatible with the latest Dafny nightly prerelease, and hence will MOSTLY fix the nightly build once merged. "Mostly" because I still need to fix some externs to make them use the pattern that avoids the Java TypeDescriptor differences between Dafny versions, but that can be fixed in a follow-up PR.

Highlights of the changes:

  • Apply the same workflow changes as chore: Forwards compatibility with Dafny > 4.2 (including pending 4.5) aws-cryptographic-material-providers-library#195 to use smithy-dafny to regenerate code, either to check that the output matches what's checked in (in a new separate codegen workflow) or to be compatible with newer versions of Dafny in the nightly build (in existing workflows).
    • In this case we also have to locally update the MPL submodule to the latest, so that we can pick up the forwards-compatible changes to that repo, and regenerate code transitively. Generating code is unfortunately getting quite expensive, especially since the shared makefile logic isn't sophisticated enough to avoid regenerating the same code multiple times.
    • Because the code in this repo wasn't formatted already, but applying newer smithy-dafny code generation automatically formats, all the generated has trivial layout changes.
    • Also extracted a manual patch.
    • ~Applied a lot of explicit client downcasting to account for the change in smithy-dafny ~ (the change in smithy-dafny was undone so this isn't necessary any more)
  • Converted Beacon.CheckBytesToHex() from a "test lemma" to a dynamic test, because on the latest Dafny the verification got even more expensive, and this didn't seem to introduce any real additional risk.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@robin-aws robin-aws changed the title Workflow and makefile changes chore: Adopt SmithyDafnyMakefile.mk, fix nightly build Mar 7, 2024
@robin-aws
Copy link
Contributor Author

Closing temporarily to avoid pointless CI, while I verify this works for Dafny nightly as well

@robin-aws robin-aws closed this Mar 8, 2024
@robin-aws robin-aws reopened this Mar 8, 2024
…amodb-java into robin-aws/use-smithy-dafny-makefile

# Conflicts:
#	DynamoDbEncryption/dafny/DynamoDbEncryption/Model/AwsCryptographyDbEncryptionSdkDynamoDbTypes.dfy
#	DynamoDbEncryption/dafny/DynamoDbEncryption/src/Index.dfy
#	DynamoDbEncryption/dafny/DynamoDbEncryptionTransforms/Model/AwsCryptographyDbEncryptionSdkDynamoDbTransformsTypes.dfy
#	DynamoDbEncryption/dafny/DynamoDbEncryptionTransforms/src/Index.dfy
#	DynamoDbEncryption/dafny/DynamoDbEncryptionTransforms/test/TestFixtures.dfy
#	DynamoDbEncryption/dafny/DynamoDbItemEncryptor/Model/AwsCryptographyDbEncryptionSdkDynamoDbItemEncryptorTypes.dfy
#	DynamoDbEncryption/dafny/DynamoDbItemEncryptor/src/Index.dfy
#	DynamoDbEncryption/dafny/StructuredEncryption/Model/AwsCryptographyDbEncryptionSdkStructuredEncryptionTypes.dfy
#	DynamoDbEncryption/dafny/StructuredEncryption/src/AwsCryptographyDbEncryptionSdkStructuredEncryptionOperations.dfy
#	DynamoDbEncryption/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/dbencryptionsdk/dynamodb/DynamoDbEncryption.java
#	DynamoDbEncryption/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/dbencryptionsdk/dynamodb/itemencryptor/DynamoDbItemEncryptor.java
#	DynamoDbEncryption/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/dbencryptionsdk/dynamodb/itemencryptor/ToDafny.java
#	DynamoDbEncryption/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/dbencryptionsdk/dynamodb/itemencryptor/ToNative.java
#	DynamoDbEncryption/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/dbencryptionsdk/dynamodb/itemencryptor/model/ParsedHeader.java
#	DynamoDbEncryption/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/dbencryptionsdk/dynamodb/transforms/DynamoDbEncryptionTransforms.java
#	DynamoDbEncryption/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/dbencryptionsdk/structuredencryption/StructuredEncryption.java
#	DynamoDbEncryption/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/dbencryptionsdk/structuredencryption/ToDafny.java
#	DynamoDbEncryption/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/dbencryptionsdk/structuredencryption/ToNative.java
#	DynamoDbEncryption/runtimes/java/src/main/smithy-generated/software/amazon/cryptography/dbencryptionsdk/structuredencryption/model/ParsedHeader.java
#	DynamoDbEncryption/runtimes/net/Generated/DynamoDbEncryption/TypeConversion.cs
#	DynamoDbEncryption/runtimes/net/Generated/DynamoDbEncryptionTransforms/TypeConversion.cs
#	DynamoDbEncryption/runtimes/net/Generated/DynamoDbItemEncryptor/TypeConversion.cs
#	SharedMakefile.mk
#	TestVectors/dafny/DDBEncryption/src/JsonConfig.dfy
@robin-aws robin-aws marked this pull request as ready for review March 27, 2024 16:30
@robin-aws robin-aws requested a review from a team as a code owner March 27, 2024 16:30
…amodb-java into robin-aws/use-smithy-dafny-makefile

# Conflicts:
#	DynamoDbEncryption/dafny/DynamoDbEncryptionTransforms/Model/AwsCryptographyDbEncryptionSdkDynamoDbTransformsTypes.dfy
#	DynamoDbEncryption/dafny/DynamoDbEncryptionTransforms/src/DdbMiddlewareConfig.dfy
#	Makefile
#	SharedMakefile.mk
@robin-aws robin-aws changed the title chore: Adopt SmithyDafnyMakefile.mk, fix nightly build chore: Adopt SmithyDafnyMakefile.mk, progress towards fixing nightly build Apr 18, 2024
Copy link
Contributor

@ajewellamz ajewellamz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@robin-aws robin-aws merged commit 785481c into main Apr 19, 2024
29 checks passed
@robin-aws robin-aws deleted the robin-aws/use-smithy-dafny-makefile branch April 19, 2024 16:02
josecorella pushed a commit to josecorella/aws-database-encryption-sdk-dynamodb that referenced this pull request Apr 23, 2024
## [3.4.0](v3.3.0...v3.4.0) (2024-04-23)

### Features

* enforce Dafny formatting ([aws#865](https://github.com/josecorella/aws-database-encryption-sdk-dynamodb/issues/865)) ([dfc0dbd](dfc0dbd))

### Maintenance

* add verify test for test vectors ([aws#897](https://github.com/josecorella/aws-database-encryption-sdk-dynamodb/issues/897)) ([6c980e7](6c980e7))
* Adopt SmithyDafnyMakefile.mk, progress towards fixing nightly build ([aws#797](https://github.com/josecorella/aws-database-encryption-sdk-dynamodb/issues/797)) ([785481c](785481c)), closes [/github.com/aws/pull/797/files#diff-692e2b06d124c9775e2fcd9cd9dbd10e0c8ea470e08174ed0b258b0301622581R182](https://github.com/josecorella//github.com/aws/aws-database-encryption-sdk-dynamodb/pull/797/files/issues/diff-692e2b06d124c9775e2fcd9cd9dbd10e0c8ea470e08174ed0b258b0301622581R182)
* **deps:** bump actions/setup-dotnet from 3 to 4 in /.github/workflows ([aws#943](https://github.com/josecorella/aws-database-encryption-sdk-dynamodb/issues/943)) ([f5d9748](f5d9748))
* **deps:** bump io.github.gradle-nexus.publish-plugin ([aws#903](https://github.com/josecorella/aws-database-encryption-sdk-dynamodb/issues/903)) ([04c6cc4](04c6cc4))
* **deps:** bump org.projectlombok:lombok ([aws#838](https://github.com/josecorella/aws-database-encryption-sdk-dynamodb/issues/838)) ([56f1cd1](56f1cd1))
* **deps:** bump rrainn/dynamodb-action in /.github/workflows ([aws#932](https://github.com/josecorella/aws-database-encryption-sdk-dynamodb/issues/932)) ([16e4d7b](16e4d7b))
* point to the correct readme ([aws#845](https://github.com/josecorella/aws-database-encryption-sdk-dynamodb/issues/845)) ([b950b4a](b950b4a))
* repair json file names ([aws#846](https://github.com/josecorella/aws-database-encryption-sdk-dynamodb/issues/846)) ([3ca955a](3ca955a))
* test "dotnet pack" in CI ([aws#851](https://github.com/josecorella/aws-database-encryption-sdk-dynamodb/issues/851)) ([75e44d0](75e44d0))
josecorella pushed a commit that referenced this pull request Apr 30, 2024
## [3.4.0](v3.3.0...v3.4.0) (2024-04-30)

### Features

* enforce Dafny formatting ([#865](#865)) ([dfc0dbd](dfc0dbd))
* more test vectors ([#959](#959)) ([3ca15af](3ca15af))

### Maintenance

* add verify test for test vectors ([#897](#897)) ([6c980e7](6c980e7))
* Adopt SmithyDafnyMakefile.mk, progress towards fixing nightly build ([#797](#797)) ([785481c](785481c)), closes [/github.com//pull/797/files#diff-692e2b06d124c9775e2fcd9cd9dbd10e0c8ea470e08174ed0b258b0301622581R182](https://github.com/aws//github.com/aws/aws-database-encryption-sdk-dynamodb/pull/797/files/issues/diff-692e2b06d124c9775e2fcd9cd9dbd10e0c8ea470e08174ed0b258b0301622581R182)
* **CI/CD:** add semantic release automation ([#949](#949)) ([3f22abc](3f22abc))
* **deps:** bump actions/setup-dotnet from 3 to 4 in /.github/workflows ([#943](#943)) ([f5d9748](f5d9748))
* **deps:** bump aws-actions/configure-aws-credentials ([#954](#954)) ([90d7d78](90d7d78))
* **deps:** bump io.github.gradle-nexus.publish-plugin ([#903](#903)) ([04c6cc4](04c6cc4))
* **deps:** bump org.projectlombok:lombok ([#838](#838)) ([56f1cd1](56f1cd1))
* **deps:** bump rrainn/dynamodb-action in /.github/workflows ([#932](#932)) ([16e4d7b](16e4d7b))
* **docs:** mention sign_and_include in javadoc for keyid supplier ([#966](#966)) ([2796693](2796693))
* point to the correct readme ([#845](#845)) ([b950b4a](b950b4a))
* repair json file names ([#846](#846)) ([3ca955a](3ca955a))
* test "dotnet pack" in CI ([#851](#851)) ([75e44d0](75e44d0))
* **test:** add tests for attribute names that seem structured ([#964](#964)) ([c4c0886](c4c0886))
* Update MPL to 1.3.0 ([#972](#972)) ([3d8acae](3d8acae))
josecorella added a commit that referenced this pull request May 1, 2024
* chore(release): 3.4.0 [skip ci]

## [3.4.0](v3.3.0...v3.4.0) (2024-04-30)

### Notes
#### .NET
- [#797](#797) ([785481c](785481c)) Enforces User input Constraints at Type Conversion.

Prior to this fix, unset Integers defaulted to `0`, and unset Booleans defaulted to `false`.

Now, all required fields MUST be set or a Runtime Exception will be thrown.

This particularly effects Searchable Encryption's 
`ConstructorPart`, who's required field previously
would have defaulted to false.
Any configuration ever created for Searchable Encryption can be re-created with the fix, but they may look different.

### Features

*  **feat(.NET):** Validate user input #797 (785481c)


### Maintenance

* **format:** enforce Dafny formatting ([#865](#865)) ([dfc0dbd](dfc0dbd))
* **test:** more test vectors ([#959](#959)) ([3ca15af](3ca15af))
* **CI** add verify test for test vectors ([#897](#897)) ([6c980e7](6c980e7))
* **CI/CD:** add semantic release automation ([#949](#949)) ([3f22abc](3f22abc))
* **deps:** bump actions/setup-dotnet from 3 to 4 in /.github/workflows ([#943](#943)) ([f5d9748](f5d9748))
* **deps:** bump aws-actions/configure-aws-credentials ([#954](#954)) ([90d7d78](90d7d78))
* **deps(Java):** bump io.github.gradle-nexus.publish-plugin ([#903](#903)) ([04c6cc4](04c6cc4))
* **deps(Java):** bump org.projectlombok:lombok ([#838](#838)) ([56f1cd1](56f1cd1))
* **deps:** bump rrainn/dynamodb-action in /.github/workflows ([#932](#932)) ([16e4d7b](16e4d7b))
* **docs:** mention sign_and_include in javadoc for keyid supplier ([#966](#966)) ([2796693](2796693))
* **docs:** point to the correct readme ([#845](#845)) ([b950b4a](b950b4a))
* **fix:** repair json file names ([#846](#846)) ([3ca955a](3ca955a))
* **test(.NET):** "dotnet pack" in CI ([#851](#851)) ([75e44d0](75e44d0))
* **test:** add tests for attribute names that seem structured ([#964](#964)) ([c4c0886](c4c0886))
* **deps(Java & .NET):** Update MPL to 1.3.0 ([#972](#972)) ([3d8acae](3d8acae))

Co-authored-by: semantic-release-bot <semantic-release-bot@martynus.net>
Co-authored-by: Tony Knapp <5892063+texastony@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants