JsonNode/JsonObject not differentiating between missing property and null-value

[WhatzGames]

Description

The [string]-Operator of JsonNode states the following:

       /// <summary>
        ///   Gets or sets the element with the specified property name.
        ///   If the property is not found, <see langword="null"/> is returned.
        /// </summary>
        /// <param name="propertyName">The name of the property to return.</param>
        /// <exception cref="ArgumentNullException">
        ///   <paramref name="propertyName"/> is <see langword="null"/>.
        /// </exception>
        /// <exception cref="InvalidOperationException">
        ///   The current <see cref="JsonNode"/> is not a <see cref="JsonObject"/>.
        /// </exception>

The result though is not only null if the property is not found, but also if a value seems to be explicitly set to null.
This leads me to say, that one can not differentiate between the two cases.

Reproduction Steps

using System.Text.Json.Nodes;

const string jsonAbsent = "{}";

const string jsonPresent = @"{
    ""value"": null
}";

var nodeAbsent = JsonNode.Parse(jsonAbsent);
var nodePresent = JsonNode.Parse(jsonPresent);

var valueAbsent = nodeAbsent!["value"];
var valuePresent = nodePresent!["value"];

//expected: True
//result: False
Console.WriteLine(valueAbsent != valuePresent);

Expected behavior

for the property-access to return a null on a property not included in the JsonObject, but returning a JsonValue containing null in the case if the property has been explicitly set to null.

Actual behavior

Both cases return null.

Regression?

As far as I know, it has been the default behavior.

Known Workarounds

I could not find any workarounds.

Configuration

• Which version of .NET is the code running on?
  .NET 6.0.103 & 6.0.201
• What OS and version, and what distro if applicable?
  Microsoft Windows 10 (21H2 / Build 19044.1586)
• What is the architecture (x64, x86, ARM, ARM64)?
  x64
• Do you know whether it is specific to that configuration?
  No
• If you're using Blazor, which web browser(s) do you see this issue in?
  Not used

Other information

Probably would need an extra condition for null-values while parsing the node.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-text-json
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

The [string]-Operator of JsonNode states the following:

       /// <summary>
        ///   Gets or sets the element with the specified property name.
        ///   If the property is not found, <see langword="null"/> is returned.
        /// </summary>
        /// <param name="propertyName">The name of the property to return.</param>
        /// <exception cref="ArgumentNullException">
        ///   <paramref name="propertyName"/> is <see langword="null"/>.
        /// </exception>
        /// <exception cref="InvalidOperationException">
        ///   The current <see cref="JsonNode"/> is not a <see cref="JsonObject"/>.
        /// </exception>

The result though is not only null if the property is not found, but also if a value seems to be explicitly set to null.
This leads me to say, that one can not differentiate between the two cases.

Reproduction Steps

using System.Text.Json.Nodes;

const string jsonAbsent = "{}";

const string jsonPresent = @"{
    ""value"": null
}";

var nodeAbsent = JsonNode.Parse(jsonAbsent);
var nodePresent = JsonNode.Parse(jsonPresent);

var valueAbsent = nodeAbsent!["value"];
var valuePresent = nodePresent!["value"];

Console.WriteLine(valueAbsent != valuePresent);

Expected behavior

for the property-access to return a null value on a property not included in the JsonObject, but returning a JsonValue containing null in the case if the property has been explicitly set to null.

Actual behavior

Both cases return null.

Regression?

As far as I know, it has been the default behavior.

Known Workarounds

I could not find any workarounds.

Configuration

• Which version of .NET is the code running on?
  .NET 6.0.103 & 6.0.201
• What OS and version, and what distro if applicable?
  Microsoft Windows 10 (21H2 / Build 19044.1586)
• What is the architecture (x64, x86, ARM, ARM64)?
  x64
• Do you know whether it is specific to that configuration?
  No
• If you're using Blazor, which web browser(s) do you see this issue in?
  Not used

Other information

Probably would need an extra condition for null-values while parsing the node.

Author:	WhatzGames
Assignees:	-
Labels:	area-System.Text.Json, untriaged
Milestone:	-

[KalleOlaviNiemitalo]

Mentioned in https://github.com/dotnet/designs/blob/40794be63ecd8b35e9596412050a84dedd575b99/accepted/2020/serializer/WriteableDomAndDynamic.md#missing-vs-null. It should be possible to distinguish by calling JsonObject.TryGetPropertyValue or JsonObject.ContainsKey.

returning a JsonValue containing null

AFAICT, JsonValue does not support constructing a JsonValue containing null. The JsonValue.Create methods return null if given null.

[eiriktsarpalis]

Thanks for the answer @KalleOlaviNiemitalo. Closing as by-design behavior.

[gregsdennis]

Can this be reopened, please? (If I need to open a new issue, I can.)

In cases where JsonNodes are returned from methods, it remains impossible to determine if the node exists with a null value or doesn't exist.

The "as designed" behavior is insufficient to express this.

I have a lot of consideration around this in going through the same design decisions years ago with Manatee.Json. Ultimately I decided it was better to define a static value to represent JSON null and distinguish it from .Net null.

Having this distinction makes so many operations considerably easier.

Moreover this is counterintuitive:

var obj = JsonNode.Parse(@"{
    ""value"": null
}") as JsonObject;

var value = obj["value"];  // value == null

value.GetPath();  // should return "$.value"

This throws a NullReferenceException, even through I can plainly see that the value property exists.

[eiriktsarpalis]

I understand the concern, however I suspect that at this point changing JsonValue to have a baked in representation for JSON null would be a breaking change for many.

cc @steveharter for a second opinion

[gregsdennis]

To further the point, this has forced me to write an API for my library as one that follows the TryParse pattern:

public bool TryEvaluate(JsonNode source, out JsonNode? result) { ... }

rather than one that simply returns a JsonNode:

public JsonNode? Evaluate(JsonNode source) { ... }

It's not impossible to work around this design decision, but it's really uncomfortable. I'd much rather be able to do the latter.