Exclude some profile fields from Jinja rendering when they are not valid Jinja. #7630
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
peterallenwebb
requested review from
QMalcolm,
aranke and
VersusFacit
and removed request for
a team
aranke
reviewed
May 15, 2023
|
|
||
| try: | ||
| rendered = super().render_value(value, keypath) | ||
| except Exception as ex: |
There was a problem hiding this comment.
Catch a more specific exception here.
There was a problem hiding this comment.
I would except I do not know what will be in the exception message here, and I'm concerned that the risk of it containing sensitive information that would be logged could outweigh the value of a narrower catch.
There was a problem hiding this comment.
Is the issue that we don't have any idea of the Exception? Or is it that we expect a few specific ones (let's say ExceptionA and ExceptionB for simplicity) and also want to make sure that we catch anything? If it's the latter, then would this be a compromise:
try:
rendered = super().render_value(value, keypath)
except ExceptionA:
# do some ExceptionA stuff
except ExceptionB:
# do some ExceptionB stuff
except Exception as ex:
# this shouldn't happen so throw a dbt exception, such as DbtCompilerExceptionThere was a problem hiding this comment.
I like @mikealfare's suggestion here
| def test_no_jinja_for_password(self, project, profiles_root): | ||
| with open(pathlib.Path(profiles_root, "profiles.yml"), "w") as profiles_yml: | ||
| profiles_yml.write( | ||
| """test: |
There was a problem hiding this comment.
Can you store this string as a constant at the top to make it easier to read?
| """ | ||
| ) | ||
| result = dbtRunner().invoke(["parse"]) | ||
| assert result.success # for now, at least, just ensure success |
There was a problem hiding this comment.
Is there a failure case we can test here?
There was a problem hiding this comment.
I'll add an additional check that scans the log messages for the password string.
gshank
approved these changes
May 16, 2023
|
@gshank I added a test along the lines you suggested. |
dbeatty10
reviewed
May 17, 2023
| @@ -0,0 +1,6 @@ | |||
| kind: Fixes | |||
| body: Exclude password fields from Jinja rendering. | |||
There was a problem hiding this comment.
One idea for updating the changelog entry to reflect the latest approach:
Suggested change
| body: Exclude password fields from Jinja rendering. | |
| body: Fall back if rendering the password field fails. |
peterallenwebb
changed the title
peterallenwebb
added a commit
that referenced
this pull request
May 17, 2023
…lid Jinja. (#7630) * CT-2583: Exclude some profile fields from Jinja rendering. * CT-2583: Add functional test. * CT-2583: Change approach to password jinja detection * CT-2583: Extract string constant and add additional checks * CT-2583: Improve unit test coverage
peterallenwebb
added a commit
that referenced
this pull request
May 18, 2023
* Exclude some profile fields from Jinja rendering when they are not valid Jinja. (#7630) * CT-2583: Exclude some profile fields from Jinja rendering. * CT-2583: Add functional test. * CT-2583: Change approach to password jinja detection * CT-2583: Extract string constant and add additional checks * CT-2583: Improve unit test coverage * CT-2583: Update changelog entry to reflect new approach
2 tasks
emmyoop
pushed a commit
that referenced
this pull request
Sep 8, 2023
…lid Jinja. (#7630) * CT-2583: Exclude some profile fields from Jinja rendering. * CT-2583: Add functional test. * CT-2583: Change approach to password jinja detection * CT-2583: Extract string constant and add additional checks * CT-2583: Improve unit test coverage
emmyoop
pushed a commit
that referenced
this pull request
Sep 8, 2023
…lid Jinja. (#7630) * CT-2583: Exclude some profile fields from Jinja rendering. * CT-2583: Add functional test. * CT-2583: Change approach to password jinja detection * CT-2583: Extract string constant and add additional checks * CT-2583: Improve unit test coverage
emmyoop
pushed a commit
that referenced
this pull request
Sep 8, 2023
…lid Jinja. (#7630) * CT-2583: Exclude some profile fields from Jinja rendering. * CT-2583: Add functional test. * CT-2583: Change approach to password jinja detection * CT-2583: Extract string constant and add additional checks * CT-2583: Improve unit test coverage
peterallenwebb
added a commit
that referenced
this pull request
Sep 19, 2023
* Exclude some profile fields from Jinja rendering when they are not valid Jinja. (#7630) * CT-2583: Exclude some profile fields from Jinja rendering. * CT-2583: Add functional test. * CT-2583: Change approach to password jinja detection * CT-2583: Extract string constant and add additional checks * CT-2583: Improve unit test coverage * Remove unneeded unit test. --------- Co-authored-by: Peter Webb <peter.webb@dbtlabs.com>
peterallenwebb
added a commit
that referenced
this pull request
Sep 19, 2023
* Exclude some profile fields from Jinja rendering when they are not valid Jinja. (#7630) * CT-2583: Exclude some profile fields from Jinja rendering. * CT-2583: Add functional test. * CT-2583: Change approach to password jinja detection * CT-2583: Extract string constant and add additional checks * CT-2583: Improve unit test coverage * Exclude some profile fields from Jinja rendering when they are not valid Jinja. (#7630) * CT-2583: Exclude some profile fields from Jinja rendering. * CT-2583: Add functional test. * CT-2583: Change approach to password jinja detection * CT-2583: Extract string constant and add additional checks * CT-2583: Improve unit test coverage * Remove unneeded unit test. --------- Co-authored-by: Peter Webb <peter.webb@dbtlabs.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
resolves #7629
resolves #8345
Main branch PR for #7629, which will also require backports.
Description
Excludes some profile fields from Jinja rendering.
Checklist
changie newto create a changelog entry