UN-2808 [FIX] Added support for {column_name}_v2 columns in destination DB connectors#1538
Merged
Merged
Conversation
Contributor
Summary by CodeRabbit
Walkthroughget_columns_and_values (WRITE_JSON_TO_A_SINGLE_COLUMN) now computes a has_v2_col flag from Changes
Sequence Diagram(s)sequenceDiagram
participant Caller as Caller
participant DBUtils as get_columns_and_values
participant TableInfo as table_info source
Caller->>DBUtils: request with column_mode=WRITE_JSON_TO_A_SINGLE_COLUMN, data, single_column_name
DBUtils->>TableInfo: read table_info (may be None/empty or contain columns)
alt table_info is None or empty or contains single_column_name_v2
DBUtils-->>DBUtils: has_v2_col = true
DBUtils-->>Caller: set values[single_column] = data (or {"result": data} if string)
DBUtils-->>Caller: set values[single_column_v2] = same payload
else table_info provided and lacks single_column_name_v2
DBUtils-->>DBUtils: has_v2_col = false
DBUtils-->>Caller: set values[single_column] = data (or {"result": data} if string)
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests
Tip 👮 Agentic pre-merge checks are now available in preview!Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.
Please see the documentation for more information. Example: reviews:
pre_merge_checks:
custom_checks:
- name: "Undocumented Breaking Changes"
mode: "warning"
instructions: |
Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).Please share your feedback with us on this Discord post. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
kirtimanmishrazipstack
requested review from
chandrasekharan-zipstack and
pk-zipstack
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
backend/workflow_manager/endpoint_v2/destination.py (1)
374-375: Lower log level and avoid leaking engine/schema details.Info-level logging of engine repr and full table_info can be noisy and may expose internals. Prefer debug and log minimal, non-sensitive bits.
-logger.info( - f"destination connector engine: {engine} with table_info: {table_info}" -) +logger.debug( + "destination connector engine: %s with table_info keys: %s", + type(engine).__name__, + list(table_info.keys()) if isinstance(table_info, dict) else table_info, +)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to Reviews > Disable Cache setting
Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting
📒 Files selected for processing (2)
backend/workflow_manager/endpoint_v2/database_utils.py(1 hunks)backend/workflow_manager/endpoint_v2/destination.py(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build
pk-zipstack
approved these changes
Sep 16, 2025
chandrasekharan-zipstack
approved these changes
Sep 16, 2025
chandrasekharan-zipstack
left a comment
chandrasekharan-zipstack
left a comment
Contributor
There was a problem hiding this comment.
I've approved this PR to unblock it but I don't have much context here, do check the code rabbit comment I unresolved
…ack/unstract into UN-2808-db-connector-column-v2-fix
Contributor
|
|
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
backend/workflow_manager/endpoint_v2/database_utils.py (3)
167-173: DRY up column-presence checks with a tiny helper (optional).The
has_*_colpredicates repeat the same pattern. A helper improves readability and keeps the logic in one place.Apply within this block:
- has_v2_col = ( - (table_info is None) - or any(k.lower() == v2_col_name.lower() for k in table_info) - if table_info - else True - ) + has_v2_col = _has_col(table_info, v2_col_name)Add this helper inside
get_columns_and_values(or as a@staticmethod) and optionally use it for metadata/error/status too:def _has_col(table_info: dict[str, str] | None, col_name: str) -> bool: if not table_info: return True col_lc = col_name.lower() try: iterable = table_info.keys() except AttributeError: iterable = table_info return any((k or "").lower() == col_lc for k in iterable)
197-205: Docstring arg name mismatch (nit).“connector_cls” in the docstring should be “conn_cls”.
- Args: - connector_cls: DB connection class + Args: + conn_cls: DB connection class
268-268: Typo in log message (nit).“sucessfully” → “successfully”.
- logger.debug(f"sucessfully inserted into table {table_name} with: {sql} query") + logger.debug(f"successfully inserted into table {table_name} with: {sql} query")
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to Reviews > Disable Cache setting
Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting
📒 Files selected for processing (1)
backend/workflow_manager/endpoint_v2/database_utils.py(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build
🔇 Additional comments (1)
backend/workflow_manager/endpoint_v2/database_utils.py (1)
167-173: Guarded writes to “_v2” look correct — backward compatible.Conditionally populating
<single_column_name>_v2based ontable_infoprevents inserts from failing on unmigrated tables while still creating the column on first-write (when schema is unknown/empty). Nice fix.
- Please confirm: in our code paths, an empty
table_inforeliably indicates “new table to be created” rather than “schema fetch failed/insufficient privileges”. If the latter can occur, we might inadvertently attempt to write a non-existent_v2column.Also applies to: 177-182
kirtimanmishrazipstack
deleted the
UN-2808-db-connector-column-v2-fix
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
{column_name}_v2will be added to all destination connectorsWhy
How
Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)
Database Migrations
Env Config
Relevant Docs
Related Issues or PRs
Dependencies Versions
Notes on Testing
Screenshots
Checklist
I have read and understood the Contribution Guidelines.