Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider Adding State Check Equivalents to Plan Checks #266

Closed
bendbennett opened this issue Jan 8, 2024 · 1 comment
Closed

Consider Adding State Check Equivalents to Plan Checks #266

bendbennett opened this issue Jan 8, 2024 · 1 comment
Labels
enhancement New feature or request
Milestone

Comments

@bendbennett
Copy link
Contributor

bendbennett commented Jan 8, 2024

terraform-plugin-testing version

v.1.6.0

Use cases

The plancheck package within terraform-plugin-testing has plan checks which use tfjson.Plan. This provides structured data which is easily traversed and interrogated for the purposes of, for instance, verifying that a specific resource attribute, or output value has a known type and value.

There is an equivalent tfjson.State which is available, and would provide an similarly straightforward way to traverse and interrogate the state for the purposes of verifying, for instance, that a specific resource attribute, or output value has a known type and value.

Attempted solutions

Currently, values in state can be checked by configuring the TestStep.Check field and using either predefined TestCheckFunc functions (e.g., TestCheckNoResourceAttr) or by writing custom TestCheckFunc functions. Checks that use TestCheckFunc functions are constrained by the flatmap syntax that is required to access state values, and by the fact that all values are represented as strings.

Proposal

The proposal is to:

  • Add, where relevant, state checks that are equivalent to the previously implemented plan checks.
  • Add predefined/built-in state checks that provide equivalent functionality to that provided by the predefined TestCheckFunc functions (e.g., TestCheckNoResourceAttr).

Further Considerations

The deprecation of the predefined TestCheckFunc functions (e.g., TestCheckNoResourceAttr) has been separated from the implementation of state checks, and will now be considered in a subsequent set of changes:

@bendbennett bendbennett added the enhancement New feature or request label Jan 8, 2024
@bendbennett bendbennett added this to the v1.7.0 milestone Jan 8, 2024
bendbennett added a commit that referenced this issue Jan 11, 2024
bendbennett added a commit that referenced this issue Jan 11, 2024
  * Configuring when state checks are executed.
  * Testing that state checks are executed.
bendbennett added a commit that referenced this issue Jan 11, 2024
bendbennett added a commit that referenced this issue Jan 11, 2024
bendbennett added a commit that referenced this issue Jan 15, 2024
…and `ExpectKnownOutputValueAtPath` (#248)

* Add KnownValue interface and types (#243)

* Add ExpectKnownValue plan check (#243)

* Handling different permutations for equality checking of interface type and value, and known value type and value (#243)

* Adding tests for missing resource, and attribute value null (#243)

* Adding plan checks for known output value and known output value at path (#243)

* Adding documentation (#243)

* Adding changelog entries (#243)

* Adding TerraformVersionChecks (#243)

* Modifying to handle numerical values returned as json.Number for tfjson.Plan (#243)

* Renaming known value constructors (#243)

* Refactoring to Check interface (#243)

* Linting (#243)

* Modifying known value check error messages and tests (#243)

* Updating tests for ExpectKnownValue, ExpectKnownOutputValue and ExpectKnownOutputValueAtPaath (#243)

* Adding changelog entry to note the switch to using json.Number for numerical value representation in the plan (#243)

* Remove reference to state checks (#243)

* Moving concepts under title and removing reference to Framework types (#243)

* Updating Go doc comments to clarify usage of partial equality and remove references to KnownValue interface (#243)

* Modifying known-values.mdx page description (#243)

* Restructuring and updating references to knownvalue.Check (#243)

* Adding individual docs pages for each type of known value check (#243)

* Removing references to num elements (#243)

* Removing references to state (#243)

* Adding docs page for custom known value checks (#243)

* Fixing error message (#243)

* Refactoring to accomodate custom known value checks in ExpectKnownValue, ExpectKnownOutputValue and ExpectKnownOutputValueAtPath (#243)

* Apply suggestions from code review

Co-authored-by: Brian Flad <bflad417@gmail.com>

* Unexporting types that implement known value check (#266)

* Document usage of 512-bit precision in the number known value check (#266)

* Adding attribute or output path to error message (#266)

* Replacing alias in example code (#266)

* Rename file (#266)

* Renamed list, map, and set element length checks to <List|Map|Set>SizeExact (#243)

* Removing ObjectAttributesExact (#243)

* Renaming known value check types (#243)

---------

Co-authored-by: Brian Flad <bflad417@gmail.com>
bendbennett added a commit that referenced this issue Jan 15, 2024
bendbennett added a commit that referenced this issue Jan 15, 2024
  * Configuring when state checks are executed.
  * Testing that state checks are executed.
bendbennett added a commit that referenced this issue Jan 15, 2024
bendbennett added a commit that referenced this issue Jan 15, 2024
bendbennett added a commit that referenced this issue Jan 16, 2024
bendbennett added a commit that referenced this issue Jan 22, 2024
bendbennett added a commit that referenced this issue Jan 22, 2024
…d `ExpectNullOutputValueAtPath` plan checks (#266)
bendbennett added a commit that referenced this issue Jan 22, 2024
bendbennett added a commit that referenced this issue Jan 22, 2024
bendbennett added a commit that referenced this issue Jan 25, 2024
)

* Adding StateCheck interface (#266)

  * Configuring when state checks are executed.
  * Testing that state checks are executed.

* Adding validation to ensure state checks are only defined for config (apply) tests (#266)

* Adding ExpectKnownValue state check (#266)

* Adding ExpectKnownOutputValue state check (#266)

* Adding ExpectKnownOutputValueAtPath state check (#266)

* Modifying ExpectKnown<Value|OutputValue|OutputValueAtPath> to allow for checking of null values (#266)

* Adding ExpectSensitiveValue state check (#266)

* Adding documentation for state checks and null known value check type (#266)

* Adding to the documentation for the custom known value check (#266)

* Adding changelog entries (#266)

* Refactoring to use updated known value check types (#266)

* Correcting documentation for revised naming of known value check types (#266)

* Renaming nul known value check (#266)

* Fixing tests (#266)

* Adding address and path to state check errors (#266)

* Fixing navigation (#266)

* Fixing changelog entries

* Modifying ExpectKnown<Value|OutputValue|OutputValueAtPath> to handle null checking (#266)

* Deprecating ExpectNullOutputValue and ExpectNullOutputValueAtPath plan checks (#266)

* Adding return statements (#266)

* Adding change log entry for deprecation of `ExpectNullOutputValue` and `ExpectNullOutputValueAtPath` plan checks (#266)

* Modifying return value of nullExact.String() (#266)

* Renaming variable (#266)

* Adding comment for Terraform v1.4.6 (#266)

* Adding further tests for null exact known value type check (#266)

* Linting (#266)

* Renaming BoolExact to Bool, and NullExact to Null (#266)

* Removing ConfigStateChecks type (#266)

* Move execution of ConfigStateChecks (#266)
bendbennett added a commit that referenced this issue Mar 5, 2024
…`ExpectKnownOutputValue`, and `ExpectKnownOutputValueAtPath` plan and state checks (#276)

* Adding StateCheck interface (#266)

  * Configuring when state checks are executed.
  * Testing that state checks are executed.

* Adding validation to ensure state checks are only defined for config (apply) tests (#266)

* Adding ExpectKnownValue state check (#266)

* Adding ExpectKnownOutputValue state check (#266)

* Adding ExpectKnownOutputValueAtPath state check (#266)

* Modifying ExpectKnown<Value|OutputValue|OutputValueAtPath> to allow for checking of null values (#266)

* Adding ExpectSensitiveValue state check (#266)

* Adding documentation for state checks and null known value check type (#266)

* Adding to the documentation for the custom known value check (#266)

* Adding changelog entries (#266)

* Refactoring to use updated known value check types (#266)

* Correcting documentation for revised naming of known value check types (#266)

* Renaming nul known value check (#266)

* Fixing tests (#266)

* Adding address and path to state check errors (#266)

* Fixing navigation (#266)

* Adding ExpectNoValueExists check to replace TestCheckNoResourceAttr

* Adding ExpectMatchingValues check to replace TestCheckResourceAttrPair

* Adding BoolPointerExact to illustrate a possible replacement for TestCheckResourceAttrPtr

* Adding ExpectValueExists state check to replace TestCheckResourceAttrSet

* Fixing bool pointer exact error message

* Adding ExpectContains state check to replace TestCheckTypeSetElemAttrPair

* Removing unneeded bool pointer exact check value type

* Adding docs for new built-in state checks

* Adding deprecations

* Fixing doc

* Adding changelog entries

* Fixing changelog entries

* Linting

* Modifying ExpectKnown<Value|OutputValue|OutputValueAtPath> to handle null checking (#266)

* Deprecating ExpectNullOutputValue and ExpectNullOutputValueAtPath plan checks (#266)

* Updating docs

* Adding further Go docs to illustrate migration from built-in TestCheckFunc(s), and include example usage of state checks

* Adding return statements (#266)

* Adding change log entry for deprecation of `ExpectNullOutputValue` and `ExpectNullOutputValueAtPath` plan checks (#266)

* Modifying return value of nullExact.String() (#266)

* Renaming variable (#266)

* Adding comment for Terraform v1.4.6 (#266)

* Adding further tests for null exact known value type check (#266)

* Linting (#266)

* Moving code examples to Go doc example files

* Adding deprecation note to website docs for ExpectNullOutputValue and ExpectNullOutputValueAtPath

* Adding return statements

* Renaming variables

* Modify wording of error message

* Remove unneeded file

* Renaming BoolExact to Bool, and NullExact to Null (#266)

* Removing ConfigStateChecks type (#266)

* Move execution of ConfigStateChecks (#266)

* Replacing usage of ConfigStateChecks, BoolExact and NullExact types

* Removing ExpectValueExists state check

  * Adding known value not null check type

* Adding Terraform version check

* Removing ExpectContains state check

* Removing ExpectMatchingValues state check

* Updating docs

* Apply suggestions from code review

Co-authored-by: Brian Flad <bflad417@gmail.com>

* Removing ExpectNoValueExists

* Adding knownvalue.StringRegularExpression

* Adding `CheckStateFunc` type and test for illustrating replacement of TestCheckResourceAttrPtr

* Updating docs regarding usage of preferring direct testing of resource attributes rather than using output values

* Modify docs to highlight caveats of using ExpectKnownOutputValueAtPath plan check with versions of Terraform <1.3.0

* Fixing test

* Updating docs

* Removing deprecation of built-in TestCheckFunc implementations

* Updating tests

* Remove CheckStateFunc type

* Updating docs

* Adding changelog

* Fixing links

* Adding changelog entries

* Removing nolint directives

* Updating docs and renaming following code review

* Updating docs following code review

---------

Co-authored-by: Brian Flad <bflad417@gmail.com>
Co-authored-by: Austin Valle <austinvalle@gmail.com>
Copy link

github-actions bot commented Apr 7, 2024

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 7, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant