Skip to content

Commit

Permalink
Make relationship fields in referenced entities nullable (#1958)
Browse files Browse the repository at this point in the history 
## Why make this change?

- Closes #1747 
- Records in referenced entities may not always be related to the
referencing entity. E.g. in a `book-websiteplacement` relationship, not
all `books` (referenced entity) may have a `websiteplacement`
(referencing) yet. This change is to ensure we get all records of the
referenced entity including the ones that DON'T have any relationships.

## What is this change?

- Previously, we used to rely on the nullability of the referenced
fields to determine whether the relationship field in the referenced
entity should be nullable or not. While this is applicable when we are
considering the relationship fields of a referencing entity, it is
actually restricting when used for a referenced entity.
- For example, the referencing entity (BookWebsitePlacement) should have
a nullable `books` (relationship field) based on whether the foreign key
`book_id` in `book_website_placements` is nullable or not -> indicating
whether a website placement MUST have a book or not.
- On the other hand, the referenced entity `books` should always have a
NULLABLE `websiteplacement` relationship field in order to include those
books that don't have any `websiteplacement` published yet. Relying on
the nullability of the `id` - the referenced field in the
book->book_website_placement foreign key would make the relationship
NON-NULLABLE but this restricts inclusion of those books that don't have
any website placements hence the `error: "Cannot return null for
non-nullable field"`. We need to make the relationship field nullable in
such cases.

- The source entity could be both the referencing and referenced entity
in case of missing foreign keys in the db or self referencing
relationships.
- Use the nullability of referencing columns to determine the
nullability of the relationship field only if
1. there is exactly one relationship where source is the referencing
entity.
DAB doesn't support multiple relationships at the moment.
and
2. when the source is not a referenced entity in any of the
relationships.
- Identifies the scenario where the relationship field is from the
referenced entity and always sets its nullability to `true`.

## How was this tested?

- [X] Integration Tests - existing data already had this bug, needed to
modify the query to expose it.
- Ran the query in #1747 to verify this fix solved that issue.

## Sample Request(s)

```GraphQL
{
  books {
    items {
      id
      title
      websiteplacement {
        price
      }
    }
  }
}
```

BEFORE:

![image](https://github.com/Azure/data-api-builder/assets/3513779/17602247-db9b-4b61-9e42-cccf527306b5)

AFTER FIX:
There are no more errors and note that we now actually return `null` as
the relationship field value(here, `websiteplacement`) when querying the
referenced entity (here, `book`) which doesn't have any relationship
with the referencing entity.

![image](https://github.com/Azure/data-api-builder/assets/3513779/eec87cc0-8341-44bd-98d9-3bcf7fd4bc40)

## NOTE
The issue is only exposed in a 1:1 relationship. This is because the
only other scenario for querying a referenced entity is a 1:many
relationship. And for records in the referenced entity(e.g. publisher)
which are NOT related to the referencing entity(e.g. book), we
explicitly check for nulls and return an empty array. See here:
https://github.com/Azure/data-api-builder/blob/dacf3bccbe0a36da51776bf4afd1b4a0d4348ff4/src/Core/Resolvers/SqlQueryEngine.cs#L123

E.g.
```GraphQL Query for 1:many relationship
{ publishers {
    items {
      id
      name
      books {
        items {
          title
        }
      }
      }
    }
}
```

Expected Response

![image](https://github.com/Azure/data-api-builder/assets/3513779/556cee8f-80d4-4fd3-b365-9046ec31326e)


NOTE:
When DAB supports multiple relationships, nullability of each
relationship field should be determined based on foreign keys associated
with each relationship.

---------

Co-authored-by: Sean Leonard <sean.leonard@microsoft.com>
  • Loading branch information
@Aniruddh25 @seantleonard
Aniruddh25 and seantleonard committed Jan 31, 2024
1 parent e681aa5 commit 5fe65f8
Show file tree
Hide file tree
Showing 7 changed files with 160 additions and 190 deletions.
118 changes: 82 additions & 36 deletions src/Service.GraphQLBuilder/Sql/SchemaConverter.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -121,42 +121,7 @@ public static class SchemaConverter
string targetEntityName = relationship.TargetEntity.Split('.').Last();
Entity referencedEntity = entities[targetEntityName];

bool isNullableRelationship = false;

if (// Retrieve all the relationship information for the source entity which is backed by this table definition
sourceDefinition.SourceEntityRelationshipMap.TryGetValue(entityName, out RelationshipMetadata? relationshipInfo)
&&
// From the relationship information, obtain the foreign key definition for the given target entity
relationshipInfo.TargetEntityToFkDefinitionMap.TryGetValue(targetEntityName,
out List<ForeignKeyDefinition>? listOfForeignKeys))
{
ForeignKeyDefinition? foreignKeyInfo = listOfForeignKeys.FirstOrDefault();

// Determine whether the relationship should be nullable by obtaining the nullability
// of the referencing(if source entity is the referencing object in the pair)
// or referenced columns (if source entity is the referenced object in the pair).
if (foreignKeyInfo is not null)
{
RelationShipPair pair = foreignKeyInfo.Pair;
// The given entity may be the referencing or referenced database object in the foreign key
// relationship. To determine this, compare with the entity's database object.
if (pair.ReferencingDbTable.Equals(databaseObject))
{
isNullableRelationship = sourceDefinition.IsAnyColumnNullable(foreignKeyInfo.ReferencingColumns);
}
else
{
isNullableRelationship = sourceDefinition.IsAnyColumnNullable(foreignKeyInfo.ReferencedColumns);
}
}
else
{
throw new DataApiBuilderException(
message: $"No relationship exists between {entityName} and {targetEntityName}",
statusCode: HttpStatusCode.InternalServerError,
subStatusCode: DataApiBuilderException.SubStatusCodes.GraphQLMapping);
}
}
bool isNullableRelationship = FindNullabilityOfRelationship(entityName, databaseObject, targetEntityName);

INullableTypeNode targetField = relationship.Cardinality switch
{
Expand Down Expand Up @@ -276,5 +241,86 @@ public static IValueNode CreateValueNodeFromDbObjectMetadata(object metadataValu

return arg;
}

/// <summary>
/// Given the source entity name, its underlying database object and the targetEntityName,
/// finds if the relationship field corresponding to the target should be nullable
/// based on whether the source is the referencing or referenced object or both.
/// </summary>
/// <exception cref="DataApiBuilderException">Raised no relationship exists between the source and target
/// entities.</exception>
private static bool FindNullabilityOfRelationship(
string entityName,
DatabaseObject databaseObject,
string targetEntityName)
{
bool isNullableRelationship = false;
SourceDefinition sourceDefinition = databaseObject.SourceDefinition;
if (// Retrieve all the relationship information for the source entity which is backed by this table definition
sourceDefinition.SourceEntityRelationshipMap.TryGetValue(entityName, out RelationshipMetadata? relationshipInfo)
&&
// From the relationship information, obtain the foreign key definition for the given target entity
relationshipInfo.TargetEntityToFkDefinitionMap.TryGetValue(targetEntityName,
out List<ForeignKeyDefinition>? listOfForeignKeys))
{
// DAB optimistically adds entries to 'listOfForeignKeys' representing each relationship direction
// between a pair of entities when 1:1 or many:1 relationships are defined in the runtime config.
// Entries which don't have a matching corresponding foreign key in the database
// will have 0 referencing/referenced columns. So, we need to filter out these
// invalid entries. Non-zero referenced columns indicate valid matching foreign key definition in the
// database and hence only those can be used to determine the directionality.

// Find the foreignkeys in which the source entity is the referencing object.
IEnumerable<ForeignKeyDefinition> referencingForeignKeyInfo =
listOfForeignKeys.Where(fk =>
fk.ReferencingColumns.Count > 0
&& fk.ReferencedColumns.Count > 0
&& fk.Pair.ReferencingDbTable.Equals(databaseObject));

// Find the foreignkeys in which the source entity is the referenced object.
IEnumerable<ForeignKeyDefinition> referencedForeignKeyInfo =
listOfForeignKeys.Where(fk =>
fk.ReferencingColumns.Count > 0
&& fk.ReferencedColumns.Count > 0
&& fk.Pair.ReferencedDbTable.Equals(databaseObject));

// The source entity should at least be a referencing or referenced db object or both
// in the foreign key relationship.
if (referencingForeignKeyInfo.Count() > 0 || referencedForeignKeyInfo.Count() > 0)
{
// The source entity could be both the referencing and referenced entity
// in case of missing foreign keys in the db or self referencing relationships.
// Use the nullability of referencing columns to determine
// the nullability of the relationship field only if
// 1. there is exactly one relationship where source is the referencing entity.
// DAB doesn't support multiple relationships at the moment.
// and
// 2. when the source is not a referenced entity in any of the relationships.
if (referencingForeignKeyInfo.Count() == 1 && referencedForeignKeyInfo.Count() == 0)
{
ForeignKeyDefinition foreignKeyInfo = referencingForeignKeyInfo.First();
isNullableRelationship = sourceDefinition.IsAnyColumnNullable(foreignKeyInfo.ReferencingColumns);
}
else
{
// a record of the "referenced" entity may or may not have a relationship with
// any other record of the referencing entity in the database
// (irrespective of nullability of the referenced columns)
// Setting the relationship field to nullable ensures even those records
// that are not related are considered while querying.
isNullableRelationship = true;
}
}
else
{
throw new DataApiBuilderException(
message: $"No relationship exists between {entityName} and {targetEntityName}",
statusCode: HttpStatusCode.InternalServerError,
subStatusCode: DataApiBuilderException.SubStatusCodes.GraphQLMapping);
}
}

return isNullableRelationship;
}
}
}
2 changes: 2 additions & 0 deletions src/Service.Tests/GraphQLRequestExecutor.cs
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Net.Http.Json;
Expand Down Expand Up @@ -67,6 +68,7 @@ internal static class GraphQLRequestExecutor
if (graphQLResult.TryGetProperty("errors", out JsonElement errors))
{
// to validate expected errors and error message
Console.WriteLine($"GraphQL error: {errors}");
return errors;
}

Expand Down
64 changes: 25 additions & 39 deletions src/Service.Tests/SqlTests/GraphQLQueryTests/DwSqlGraphQLQueryTests.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -68,46 +68,32 @@ FROM GQLmappings
[TestMethod]
public async Task OneToOneJoinQuery()
{
string msSqlQuery = @"
SELECT
TOP 1 [table0].[id] AS [id],
JSON_QUERY ([table1_subq].[data]) AS [websiteplacement]
FROM
[books] AS [table0]
OUTER APPLY (
SELECT
TOP 1 [table1].[id] AS [id],
[table1].[price] AS [price],
JSON_QUERY ([table2_subq].[data]) AS [books]
FROM
[book_website_placements] AS [table1]
OUTER APPLY (
SELECT
TOP 1 [table2].[id] AS [id]
FROM
[books] AS [table2]
WHERE
[table1].[book_id] = [table2].[id]
ORDER BY
[table2].[id] Asc FOR JSON PATH,
INCLUDE_NULL_VALUES,
WITHOUT_ARRAY_WRAPPER
) AS [table2_subq]([data])
WHERE
[table1].[book_id] = [table0].[id]
ORDER BY
[table1].[id] Asc FOR JSON PATH,
INCLUDE_NULL_VALUES,
WITHOUT_ARRAY_WRAPPER
) AS [table1_subq]([data])
WHERE
[table0].[id] = 1
ORDER BY
[table0].[id] Asc FOR JSON PATH,
INCLUDE_NULL_VALUES,
WITHOUT_ARRAY_WRAPPER";
string dwSqlQuery = @"
SELECT COALESCE('[' + STRING_AGG('{' + N'""id"":' + ISNULL(STRING_ESCAPE(CAST([id] AS NVARCHAR(MAX)), 'json'),
'null') + ',' + N'""title"":' + ISNULL('""' + STRING_ESCAPE([title], 'json') + '""', 'null') + ',' +
N'""websiteplacement"":' + ISNULL([websiteplacement], 'null') +
'}', ', ') + ']', '[]')
FROM (
SELECT TOP 100 [table0].[id] AS [id],
[table0].[title] AS [title],
([table1_subq].[data]) AS [websiteplacement]
FROM [dbo].[books] AS [table0]
OUTER APPLY (
SELECT STRING_AGG('{' + N'""price"":' + ISNULL(STRING_ESCAPE(CAST([price] AS NVARCHAR(MAX)), 'json'),
'null') + '}', ', ')
FROM (
SELECT TOP 1 [table1].[price] AS [price]
FROM [dbo].[book_website_placements] AS [table1]
WHERE [table0].[id] = [table1].[book_id]
AND [table1].[book_id] = [table0].[id]
ORDER BY [table1].[id] ASC
) AS [table1]
) AS [table1_subq]([data])
WHERE 1 = 1
ORDER BY [table0].[id] ASC
) AS [table0]";

await OneToOneJoinQuery(msSqlQuery);
await OneToOneJoinQuery(dwSqlQuery);
}

/// <summary>
Expand Down
15 changes: 7 additions & 8 deletions src/Service.Tests/SqlTests/GraphQLQueryTests/GraphQLQueryTestBase.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -556,24 +556,23 @@ public async Task QueryAgainstSPWithOnlyTypenameInSelectionSet(string dbQuery)
[TestMethod]
public async Task OneToOneJoinQuery(string dbQuery)
{
string graphQLQueryName = "book_by_pk";
string graphQLQueryName = "books";
string graphQLQuery = @"query {
book_by_pk(id: 1) {
id
websiteplacement {
books {
items {
id
title
websiteplacement {
price
books {
id
}
}
}
}
}";

JsonElement actual = await base.ExecuteGraphQLRequestAsync(graphQLQuery, graphQLQueryName, isAuthenticated: false);
string expected = await GetDatabaseResultAsync(dbQuery);

SqlTestHelper.PerformTestEqualJsonStrings(expected, actual.ToString());
SqlTestHelper.PerformTestEqualJsonStrings(expected, actual.GetProperty("items").ToString());
}

/// <summary>
Expand Down
54 changes: 17 additions & 37 deletions src/Service.Tests/SqlTests/GraphQLQueryTests/MsSqlGraphQLQueryTests.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -83,43 +83,23 @@ FROM GQLmappings
public async Task OneToOneJoinQuery()
{
string msSqlQuery = @"
SELECT
TOP 1 [table0].[id] AS [id],
JSON_QUERY ([table1_subq].[data]) AS [websiteplacement]
FROM
[books] AS [table0]
OUTER APPLY (
SELECT
TOP 1 [table1].[id] AS [id],
[table1].[price] AS [price],
JSON_QUERY ([table2_subq].[data]) AS [books]
FROM
[book_website_placements] AS [table1]
OUTER APPLY (
SELECT
TOP 1 [table2].[id] AS [id]
FROM
[books] AS [table2]
WHERE
[table1].[book_id] = [table2].[id]
ORDER BY
[table2].[id] Asc FOR JSON PATH,
INCLUDE_NULL_VALUES,
WITHOUT_ARRAY_WRAPPER
) AS [table2_subq]([data])
WHERE
[table1].[book_id] = [table0].[id]
ORDER BY
[table1].[id] Asc FOR JSON PATH,
INCLUDE_NULL_VALUES,
WITHOUT_ARRAY_WRAPPER
) AS [table1_subq]([data])
WHERE
[table0].[id] = 1
ORDER BY
[table0].[id] Asc FOR JSON PATH,
INCLUDE_NULL_VALUES,
WITHOUT_ARRAY_WRAPPER";
SELECT TOP 100 [table0].[id] AS [id]
,[table0].[title] AS [title]
,JSON_QUERY([table1_subq].[data]) AS [websiteplacement]
FROM [dbo].[books] AS [table0]
OUTER APPLY (
SELECT TOP 1 [table1].[price] AS [price]
FROM [dbo].[book_website_placements] AS [table1]
WHERE [table1].[book_id] = [table0].[id]
ORDER BY [table1].[id] ASC
FOR JSON PATH
,INCLUDE_NULL_VALUES
,WITHOUT_ARRAY_WRAPPER
) AS [table1_subq]([data])
WHERE 1 = 1
ORDER BY [table0].[id] ASC
FOR JSON PATH
,INCLUDE_NULL_VALUES";

await OneToOneJoinQuery(msSqlQuery);
}
Expand Down
30 changes: 11 additions & 19 deletions src/Service.Tests/SqlTests/GraphQLQueryTests/MySqlGraphQLQueryTests.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -107,30 +107,22 @@ public async Task MultipleResultQueryWithVariables()
public async Task OneToOneJoinQuery()
{
string mySqlQuery = @"
SELECT JSON_OBJECT('id', `subq11`.`id`, 'websiteplacement', `subq11`.`websiteplacement`)
AS `data`
SELECT COALESCE(JSON_ARRAYAGG(JSON_OBJECT('id', `subq7`.`id`, 'title', `subq7`.`title`, 'websiteplacement',
`subq7`.`websiteplacement`)), JSON_ARRAY()) AS `data`
FROM (
SELECT `table0`.`id` AS `id`,
`table0`.`title` AS `title`,
`table1_subq`.`data` AS `websiteplacement`
FROM `books` AS `table0`
LEFT OUTER JOIN LATERAL(SELECT JSON_OBJECT('id', `subq10`.`id`, 'price', `subq10`.`price`, 'books',
`subq10`.`books`) AS `data` FROM (
SELECT `table1`.`id` AS `id`,
`table1`.`price` AS `price`,
`table2_subq`.`data` AS `books`
LEFT OUTER JOIN LATERAL(SELECT JSON_OBJECT('price', `subq6`.`price`) AS `data` FROM (
SELECT `table1`.`price` AS `price`
FROM `book_website_placements` AS `table1`
LEFT OUTER JOIN LATERAL(SELECT JSON_OBJECT('id', `subq9`.`id`) AS `data` FROM (
SELECT `table2`.`id` AS `id`
FROM `books` AS `table2`
WHERE `table1`.`book_id` = `table2`.`id`
ORDER BY `table2`.`id` asc LIMIT 1
) AS `subq9`) AS `table2_subq` ON TRUE
WHERE `table0`.`id` = `table1`.`book_id`
ORDER BY `table1`.`id` asc LIMIT 1
) AS `subq10`) AS `table1_subq` ON TRUE
WHERE `table0`.`id` = 1
ORDER BY `table0`.`id` asc LIMIT 100
) AS `subq11`
WHERE `table1`.`book_id` = `table0`.`id`
ORDER BY `table1`.`id` ASC LIMIT 1
) AS `subq6`) AS `table1_subq` ON TRUE
WHERE 1 = 1
ORDER BY `table0`.`id` ASC LIMIT 100
) AS `subq7`
";

await OneToOneJoinQuery(mySqlQuery);
Expand Down

0 comments on commit 5fe65f8

Please sign in to comment.