Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GraphQL Queries - Entities with self-joining relationships get expected responses #2138

Merged
merged 87 commits into from
May 10, 2024

Conversation

seantleonard
Copy link
Contributor

@seantleonard seantleonard commented Mar 27, 2024

Self Join

Why this change?

When users defined self-joining relationships between entities (SourceEntity == TargetEntity), DAB couldn't properly process the relationship at request-time and returned null/unexpected results due to incorrectly generated sqltext.

What is this change?

Primary Changes

The primary changes:

  • An additional member of SqlMetadataProvider named RelationshipToFkDefinition (Dictionary<EntityRelationshipKey, ForeignKeyDefinition>)
  • Two new ForeignKeyDefinition fields:
    • ReferencedEntityRole -> indicates whether the entity in a self-joined relationship is the "Target" or "Source" entity.
    • ReferencingEntityRole -> indicates whether the entity in a self-joined relationship is the "Target" or "Source" entity.

Example dab-config.json -> in-engine data structure

Below is an example translation between a dab-config.json relationship and the RelationshipToFkDefinition:

dab-config.json

"relationships": {
  "parent_account": {
    "cardinality": "one",
    "target.entity": "dbo_DimAccount",
    "source.fields": [
      "ParentAccountKey"
    ],
    "target.fields": [
      "AccountKey"
    ]
  },
  "child_accounts": {
    "cardinality": "many",
    "target.entity": "dbo_DimAccount",
    "source.fields": [
      "AccountKey"
    ],
    "target.fields": [
      "ParentAccountKey"
    ]
  }
}

in-engine data structure

The following table illustrates how the dictionary RelationshipToFkDefinition is populated.

EntityRelationshipKey {EntityName-RelationshipName} FKD - Referenced Entity (Column) [Role] FKD Referencing Entity (Column) [Role]
dbo_DimAccount - parent_account dbo_DimAccount (AccountKey) [Target] dbo_DimAccount (ParentAccountKey) [Source]
dbo_DimAccount - child_accounts dbo_DimAccount (AccountKey) [Source] dbo_DimAccount (ParentAccountKey) [Target]

The enhanced metadata collection listed above enables DAB to pick out

  1. The correct ForeignKeyDefinition created to represent a config relationship
  2. Which self-join relationship columns to set for leftColumnNames (Source Fields) and rightColumnNames (Target Fields).
    • The ForeignKeyDefinition object (prior to this pr) only tracked "referencing" and "referenced" columns and DAB didn't know whether to map
      Referencing - Source; Referenced - Target; OR Referencing - Target; Referenced - Source;
    • With this PR, DAB now determines the correct mapping which is critical to creating correct join statements in the sqltext.

DAB startup uses the relationship cardinality value (one, many) to determine whether

  1. The source entity is the "referenced" or "referencing" entity.
  2. The target entity is the "referenced" or "referencing" entity.
Cardinality: one (many:one; one:one)

Example: books(Many) - publisher(One) where books.publisher_id is referencing publisher.id

Referencing Entity Referenced Entity
Source Entity Target Entity
Cardinality: many (one:many)

Example: publisher(One)-books(Many) where publisher.id is referenced by books.publisher_id

Referencing Entity Referenced Entity
Target Entity Source Entity

Example GraphQL Query

In a GraphQL query, the top level entity is always the source entity and the nested entity is the target entity.

  • Before this PR, DAB didn't know whether the source entity was the referenced or referencing entity.
    • Now DAB knows and finds out via querying:
  • dbo_DimAccount - parent_account (entity - relationshipName) -> top level entity in query is Source and nested entity is Target
    • ForeignKeyDefinition.ResolveSourceColumns() -> check ReferencingEntityRole to return expected columns (referencing/referenced)
    • ForeignKeyDefinition.ResolveTargetColumns() -> checks ReferencedEntityRole to return expected columns (referencing/referenced)
// BaseSqlQueryStructure.AddJoinPredicatesForSelfJoinedEntity(...)
            if (MetadataProvider.RelationshipToFkDefinition.TryGetValue(key: fkLookupKey, out ForeignKeyDefinition? fkDef))
            {
                subQuery.Predicates.AddRange(
                        CreateJoinPredicates(
                            leftTableAlias: SourceAlias,
                            leftColumnNames: fkDef.ResolveSourceColumns(),
                            rightTableAlias: subqueryTargetTableAlias,
                            rightColumnNames: fkDef.ResolveTargetColumns()));
            }

The following example requires DAB to utilize both entries in the RelationshipToFkDefinition table above:

query DboDimAccountsByPk
 {
  dbo_DimAccount_by_pk(AccountKey: 2) {
    AccountKey
    ParentAccountKey
    child_accounts {
      items {
        AccountKey
      }
    }
    parent_account{
      AccountKey
    }
  }
}
  • parent_account field configured with relationship cardinatlity (one) which means:
    • Source Entity: Referencing
    • Target Entity: Referenced
  • child_accounts field configured with relationship cardinality (many) which means:
    • Source Entity: Referenced
    • Target Entity: Referencing

How was this tested?

-[] Unit tests
-[x] Integration tests

Sample Request(s)

GraphQL Parent and Child Self-Join with tsql

sample data

AccountKey ParentAccountKey
1 NULL
2 1
3 2
4 2
            query queryAccountAndParent{
                dbo_DimAccounts(first: 3, filter: {ParentAccountKey: { isNull: false}}, orderBy: { AccountKey: ASC}) {
                    items {
                        AccountKey
                        ParentAccountKey
                      parent_account {
                            AccountKey
                        }
                        child_accounts{
                          items{
                            AccountKey
                            ParentAccountKey
                          }
                        }
                    }
                }
            }
 SELECT TOP 3 
 [table0].[AccountKey] AS [AccountKey], 
 [table0].[ParentAccountKey] AS [ParentAccountKey], 
 JSON_QUERY ([table1_subq].[data]) AS [parent_account], 
 JSON_QUERY (COALESCE([table3_subq].[data], '[]')) AS [child_accounts] 
 FROM [dbo].[DimAccount] AS [table0] 
 OUTER APPLY (
	SELECT TOP 1 [table1].[AccountKey] AS [AccountKey] 
	FROM [dbo].[DimAccount] AS [table1] 
	WHERE [table1].[AccountKey] = [table0].[ParentAccountKey] 
	ORDER BY [table1].[AccountKey] ASC FOR JSON PATH, INCLUDE_NULL_VALUES,WITHOUT_ARRAY_WRAPPER) 
AS [table1_subq]([data]) 
OUTER APPLY (
	SELECT TOP 100 [table3].[AccountKey] AS [AccountKey], 
	[table3].[ParentAccountKey] AS [ParentAccountKey] 
	FROM [dbo].[DimAccount] AS [table3] 
	WHERE [table3].[ParentAccountKey] = [table0].[AccountKey] 
	ORDER BY [table3].[AccountKey] ASC FOR JSON PATH, INCLUDE_NULL_VALUES) 
AS [table3_subq]([data]) WHERE [table0].[ParentAccountKey] IS NOT NULL 
ORDER BY [table0].[AccountKey] ASC FOR JSON PATH, INCLUDE_NULL_VALUES
{
  "data": {
    "dbo_DimAccounts": {
      "items": [
        {
          "AccountKey": 2,
          "ParentAccountKey": 1,
          "parent_account": {
            "AccountKey": 1
          },
          "child_accounts": {
            "items": [
              {
                "AccountKey": 3,
                "ParentAccountKey": 2
              },
              {
                "AccountKey": 4,
                "ParentAccountKey": 2
              }
            ]
          }
        },
        {
          "AccountKey": 3,
          "ParentAccountKey": 2,
          "parent_account": {
            "AccountKey": 2
          },
          "child_accounts": {
            "items": []
          }
        },
        {
          "AccountKey": 4,
          "ParentAccountKey": 2,
          "parent_account": {
            "AccountKey": 2
          },
          "child_accounts": {
            "items": []
          }
        }
      ]
    }
  }
}

ayush3797 and others added 30 commits December 20, 2023 15:47
Co-authored-by: Shyam Sundar J  <shyamsundarj@microsoft.com>
Co-authored-by: Shyam Sundar J  <shyamsundarj@microsoft.com>
Co-authored-by: Shyam Sundar J  <shyamsundarj@microsoft.com>
@seantleonard
Copy link
Contributor Author

/azp run

Copy link
Contributor

@Aniruddh25 Aniruddh25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posting comments so far. Few more files to go.

Copy link
Contributor

@severussundar severussundar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@seantleonard
Copy link
Contributor Author

/azp run

Copy link
Contributor

@Aniruddh25 Aniruddh25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending to look at QueryStructures.

…from databaseResolvedFkDefinition into configResolvedFkDefinition because configResolvedFkDefinition was missing that metadata, but did have the metadata ( the "more metadata" you called out) for relationshipName/sourceEntityName.
@seantleonard
Copy link
Contributor Author

/azp run

@seantleonard
Copy link
Contributor Author

/azp run

@seantleonard
Copy link
Contributor Author

/azp run

… the security advisory now correctly scopes in 7.0.7 as patched.
@seantleonard
Copy link
Contributor Author

/azp run

@seantleonard
Copy link
Contributor Author

/azp run

@seantleonard seantleonard enabled auto-merge (squash) May 10, 2024 22:29
@seantleonard seantleonard merged commit ab6f77a into main May 10, 2024
7 checks passed
@seantleonard seantleonard deleted the dev/sean/selfReferencingRelationship branch May 10, 2024 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants