Function to double quote (or custom character) surround all column names (sat and mas macros) #91
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Also an example of usage in the eff_sat.sql macro
- Currently the framework doesn't allow unusual column names to pass through
- The data for the unittests need to be updated - The associated test .sql files that call the macro for each of the unittest need to be created too
- Added a new test for when the "columns" parameter is an empty string - Reformatted the macro; now it uses dbtvault.is_something() to check if "columns" is not empty
AB#1981
AB#1981
AB#1981
AB#1981
AB#1981
AB#1981
AB#1981
Additional unit tests, update escape_column_name and sat macros
To allow for internal macro escape_column_name(column)
DVAlexHiggs
approved these changes
Jan 5, 2022
This was
linked to
issues
Jan 5, 2022
DVAlexHiggs
added a commit
that referenced
this pull request
Dec 17, 2022
…mes (sat and mas macros) (#91) * Macro to escape column names (Norbert's code) Also an example of usage in the eff_sat.sql macro * Example of usage in the eff_sat.sql macro * WIP: Added new sat fixture tables + new test for the escape column macro - Currently the framework doesn't allow unusual column names to pass through * Created a unittest file for the escape_col_name macro - The data for the unittests need to be updated - The associated test .sql files that call the macro for each of the unittest need to be created too * WIP: Added new tests + their files * Added more tests for escape_column_name macro * Extended test coverage for escape_column_name macro + minor reformatting - Added a new test for when the "columns" parameter is an empty string - Reformatted the macro; now it uses dbtvault.is_something() to check if "columns" is not empty * submod * Move example usage to eff_sat_escape_col_name.sql * Fix for dirty status * Move example fixture into fixtures_sat.py * Removed submodule dbtvault-docs * WIP update unit tests AB#1981 * Update unit tests and increase test coverage * More unit tests AB#1981 * Updated unit tests for escape_column_name() macro AB#1981 * Updated unit tests for escape_column_name() macro AB#1981 * Formatting tidy up * Centralise processing for CONCAT() / CONCAT_WS() * Fix for processing of test scenario staging rank column lists * Apply escape_column_name() macro to hub, link, t_link, xts AB#1981 * Apply escape_column_name() macro to stage macro AB#1981 * Column name protection for period and rank load AB#1981 * Testing the stage macro with a 'bad' column name AB#1981 * Bug fix AB#1981 * Update sat macro AB#1981 * Update eff sat macro AB#1981 * Update ma sat macro AB#1981 * Testing stage macro again AB#1981 * Updated fixture for stage testing AB#1981 * Minor * Updated fixture for stage testing AB#1981 * Update pit and bridge macros AB#1981 * Delete obsolete fixture * PR issue: missing fixtures and tests * PR issue: update unit tests' expected data with escaped column names * PR issue: update unit tests' expected data with escaped column names * Bug fix for PR failure * Resolve issues with dictionary style hashdiff column metadata Additional unit tests, update escape_column_name and sat macros * Rename macro as escape_column_names(columns) To allow for internal macro escape_column_name(column) * Tidy up after merge branch 'develop' Co-authored-by: Tim Wilson <tim-wilson@data-vault.com> Co-authored-by: Norbert Acatrinei <norbert.acatrinei@business-thinking.com> Co-authored-by: Alex Higgs <alex.higgs@data-vault.co.uk> Co-authored-by: Alex Higgs <alex.higgs@data-vault.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Provide function to double quote (or custom character) surround all column names in compiled SQL code,
now working with sat and mas macros.
Pull request 89
Issue 23
Issue 28
AB#1981