Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decode change values with marks #28687

Merged
merged 7 commits into from
May 13, 2021
Merged

Decode change values with marks #28687

merged 7 commits into from
May 13, 2021

Conversation

jbardin
Copy link
Member

@jbardin jbardin commented May 12, 2021

When decoding change values, the stored marks should be applied to the decoded value, as this is the direct inverse of the Encode operation. The missing marks were not noticed in most cases because they would be correctly re-evaluated on demand, however some cases like module output values can rely on the change source alone, resulting in the marks being lost during apply.

Many call sites which were aware that the marks were missing would manually apply the marks after decoding. After removing the duplicate marking and verifying tests, Decode call sites were again statically located and checked for mark handling. Any locations found that were relying on the absence of marks now must manually strip the marks before processing the values.

Fixes #28671

Marks stored in a plans.ChangeSrc were not decoded along with the
stored values. This was working in many cases by evaluation correctly
re-evaluating the marks, but this cannot happen in all cases.
When an output change was decoded, we were decoding the raw value rather
than the ChangeSrc, which lost any encoded marks for that value.
Make sure that this function can handle any unexpectedly marked values.
The only remaining caller of this function is in the diff formatter,
which uses it to suppress meaningless diffs created by legacy providers.
Marks are not needed when transcoding changes from the internal msgpack
format to json.
@jbardin jbardin requested a review from a team May 12, 2021 16:03
@jbardin jbardin added the 0.15-backport If you add this label to a PR before merging, backport-assistant will open a new PR once merged label May 12, 2021
@NestorRV
Copy link

NestorRV commented May 13, 2021

Is this going to be released within 0.15.4? @jbardin

Copy link
Member

@alisdair alisdair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This definitely feels clearer to me. One suggestion/abstract worry inline. Will test locally then ✅

// We drop the marks from the change, as decoding is only an
// intermediate step to re-encode the values as json
changeV.Before, _ = changeV.Before.UnmarkDeep()
changeV.After, _ = changeV.After.UnmarkDeep()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than mutating the struct, it might be clearer to bind the results of the UnmarkDeep calls to locals inside the if changeV.[Before|After] != cty.NilVal` blocks below. Those are already messily referring back to the change source to retrieve the marks.

One other worry here is that I think calling UnmarkDeep on cty.NilVal might crash. I'm not completely sure that this is the case, but moving the unmarking inside the if blocks feels safer.

(Same thought for the other changes in this package.)

Copy link
Member Author

@jbardin jbardin May 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified that UnmarkDeep is a noop with a NilVal, and opted to overwrite the fields here to leave the rest of the code as unchanged as possible.

I also don't particularly like mutating the value, but since the changeV values have no way of leaking out from this package, I went with the smaller change.

Copy link
Member

@alisdair alisdair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified that all my sensitive manual test cases continue to work as expected!

@jbardin jbardin merged commit ef88c54 into main May 13, 2021
@jbardin jbardin deleted the jbardin/sensitive-changes branch May 13, 2021 16:44
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 13, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
0.15-backport If you add this label to a PR before merging, backport-assistant will open a new PR once merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Always changing plan with sensitive values
3 participants