Skip to content

[Solutions Analyzer] Fix azuredeploy wrapper producing duplicate connector IDs#14184

Merged
oshezaf merged 1 commit into
Azure:masterfrom
jochrin-ms:fix/solutions-analyzer-azuredeploy-dedup
May 5, 2026
Merged

[Solutions Analyzer] Fix azuredeploy wrapper producing duplicate connector IDs#14184
oshezaf merged 1 commit into
Azure:masterfrom
jochrin-ms:fix/solutions-analyzer-azuredeploy-dedup

Conversation

@jochrin-ms

Copy link
Copy Markdown

Summary

Fixes a connector overcount bug in Solutions Analyzer for CCP v2 solutions where azuredeploy wrapper files can produce synthetic connector IDs that duplicate real definition entries.

Root Cause

  • Non-literal IDs in azuredeploy wrapper files are converted to generated IDs from title
  • In CCP v2, wrapper and definition describe the same connector, producing duplicates

Fix

  • Add two-phase deduplication in map_solutions_connectors_tables.py
  • Track non-azuredeploy literal IDs by (parent directory, connector title)
  • Skip azuredeploy entries when ID is generated and a matching non-azuredeploy definition exists

Validation

  • Reproduced with 1Password: removes synthetic duplicate only
  • connectors.csv: 593 -> 592
  • solutions_with_connectors.csv: 606 -> 605
  • solutions_connectors_tables_mapping.csv: 1397 -> 1396
  • solutions.csv unchanged

…ate connector ID

CCP v2 connector folders contain two files:
  1. *_DataConnectorDefinition.json - the actual definition with a literal id
  2. azuredeploy_*_poller_connector.json - ARM deployment wrapper with ARM variable id

Both describe the same connector, but find_connector_objects() generates a
synthetic ID from the title for the azuredeploy file (since the ARM variable
cannot be resolved), creating a phantom second connector entry.

Example - 1Password:
  Definition file: id = '1PasswordCCPDefinition' (literal)
  Azuredeploy file: id = '1Password(Serverless)' (synthetic from title)
  Result: 3 connectors reported instead of correct 2

Fix: Track connector titles from non-azuredeploy definition files per
directory. When an azuredeploy file produces a connector with id_generated=True
and the same (directory, title) was already registered from a definition file,
skip the duplicate entry and log an azuredeploy_duplicate_skipped issue.

Impact (verified via test_dedup_azuredeploy_connectors.py):
  - connectors.csv: 593 -> 592 (1Password only)
  - solutions_with_connectors.csv: 606 -> 605 (1Password only)
  - solutions_connectors_tables_mapping.csv: 1397 -> 1396 (1Password only)
  - solutions.csv: unchanged
  - No other solution affected across entire repo

Test output confirming single-solution impact:
  Duplicate azuredeploy connector IDs found:     1
  connectors.csv rows removed:                   1
  solutions_with_connectors.csv rows removed:    1
  mapping CSV rows removed:                      1
  Solutions with corrected connector_count:       1
  1Password connector_count: 3 -> 2
  Removed phantom connector: 1Password(Serverless)
@jochrin-ms jochrin-ms requested a review from a team as a code owner May 2, 2026 14:08
@jochrin-ms

Copy link
Copy Markdown
Author

Root cause: in CCP v2 connector folders, azuredeploy wrapper files can emit generated IDs when the ARM id expression is non-literal. That generated ID can describe the same logical connector already defined in the DataConnectorDefinition file, causing duplicate connector rows and overcounting (reproduced with 1Password).

Fix implemented in map_solutions_connectors_tables.py:

  • Track non-azuredeploy connector definitions with literal IDs by (parent directory, connector title)
  • When processing azuredeploy files, skip entries where id_generated=true and a matching non-azuredeploy definition already exists in the same directory

Validation:

  • 1Password synthetic duplicate removed
  • connectors.csv: 593 -> 592
  • solutions_with_connectors.csv: 606 -> 605
  • solutions_connectors_tables_mapping.csv: 1397 -> 1396
  • solutions.csv unchanged

Scope and safety:

  • Filter applies only to generated IDs from azuredeploy wrappers
  • No impact to literal IDs or unrelated solutions

@jochrin-ms

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="Microsoft"

@v-maheshbh v-maheshbh self-assigned this May 4, 2026
@jochrin-ms

jochrin-ms commented May 4, 2026 via email

Copy link
Copy Markdown
Author

@oshezaf oshezaf assigned oshezaf and unassigned v-maheshbh May 4, 2026
@oshezaf oshezaf merged commit 60738f3 into Azure:master May 5, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants