Issue #2955 : Warn when a referenced database connection does not exist - #8245
Merged
Conversation
…ot exist Warn on save and Verify when a transform or action names a relational database connection that is not in project metadata. Saving is never blocked. Names that still contain a variable after resolving the current environment are skipped, and the check does not open a JDBC connection.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2955
Sergio originally asked for a warning when a transform is assigned a database connection that is not in the project metadata, while still being able to save. This implements that, and extends it so the same check covers actions as well as transforms, without patching every plugin's
check()method.What the user sees
When Validate database connections when saving is on (the default), saving a pipeline or workflow that references a missing RDBMS connection shows a warning:
Verify (F7) reports the same findings. Saving is never blocked. Hop does not open a JDBC connection for this check.
Connection names that still contain a variable after the current environment is applied (
'${CONNECTION}','db_${ENV}', …) are skipped, because the name cannot be decided at design time.How it works
HopMetadataPropertyWalkercollects every@HopMetadataProperty(hopMetadataPropertyType = RDBMS_CONNECTION)string, including nested lists (Check DB connections, Table Compare).ReferencedDatabaseConnectionCheckerresolves the name, skips unresolved variable tokens, and looks it up in metadata (serializer.exists).ReferencedConnectionSaveValidatorandMessageDialogWithToggle.A few plugins stored a
connectionstring withoutRDBMS_CONNECTIONand would have been invisible to the walker. Those annotations are added on Wait for SQL, MySQL/MSSQL bulk load, SQL File Output, and Snowflake Warehouse Manager.Tests
StringUtil.containsVariableTokenHopMetadataPropertyWalkerTest(nested lists, two connection fields, unannotated fields ignored)ReferencedDatabaseConnectionCheckerTest(missing/present names, unresolved and resolved variables, empty connection, nested lists, pipeline transform, workflow action)UI compiled with an isolated display. Full
mvn clean install apache-rat:checkwas not run locally.Docs
Configuration perspective, create pipeline, create workflow, and RDBMS connection pages, with screenshots of the option and the save warning.
addresses #123), if applicable.