[SR-15781] 5.5 Linux only: for empty dictionaries, JSONDecoder & JSONEncoder don't roundtrip

[weissi]

Previous ID	SR-15781
Radar	rdar://problem/88197530
Original Reporter	@weissi
Type	Bug
Status	Resolved
Resolution	Done

Environment

5.5 (swift-5.5-RELEASE)

Additional Detail from JIRA

Votes	0
Component/s	Foundation
Labels	Bug, Linux
Assignee	@millenomi
Priority	Medium

md5: c76d56c48a3626a6da4c743bc9592a1b

Issue Description:

This is a very severe bug that will affect any Swift software that encodes any dictionary whose keys aren't String or Int.

JSONDecoder will error on values produces by JSONEncoder for the empty Dictionary.

Consider this repro:

import Foundation


struct Something: Codable {
   struct Key: Codable, Hashable {
     var x: String
   }

   var dict: [Key: String]
}

let enc1 = try JSONEncoder().encode(Something(dict: [
.init(x: "a"): "a",
]))
let enc2 = try JSONEncoder().encode(Something(dict: [:
]))

do {
    print(String(decoding: enc1, as: UTF8.self))
    print(try JSONDecoder().decode(Something.self, from: enc1))
    print(String(decoding: enc2, as: UTF8.self))
    print(try JSONDecoder().decode(Something.self, from: enc2))
} catch {
    print("ERRROR: ", error)
}

Darwin

$ swift test.swift 
{"dict":[{"x":"a"},"a"]}
Something(dict: [test.Something.Key(x: "a"): "a"])
{"dict":[]}
Something(dict: [:])

which is correct. Note how the empty dictionary gets encoded as [] in JSON.

Linux Swift 5.4

$ jw-docker-swift-5.4 swift test.swift 
{"dict":[{"x":"a"},"a"]}
Something(dict: [test.Something.Key(x: "a"): "a"])
{"dict":[]}
Something(dict: [:])

same result, correct.

Linux 5.5

$ jw-docker-swift-5.5 swift test.swift 
{"dict":[{"x":"a"},"a"]}
Something(dict: [test.Something.Key(x: "a"): "a"])
{"dict":{}}
ERRROR:  typeMismatch(Swift.Array<Foundation.JSONValue>, Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "dict", intValue: nil)], debugDescription: "Expected to decode Array<JSONValue> but found a dictionary instead.", underlyingError: nil))

note how

1. the encoding of the empty dictionary becomes {} and not [] anymore.
2. how the decoder can't decode it

Linux main (from today)

$ jw-docker-swift-main swift test.swift 
docker.io/swiftlang/swift:nightly-main
{"dict":[{"x":"a"},"a"]}
Something(dict: [test.Something.Key(x: "a"): "a"])
{"dict":{}}
ERRROR:  typeMismatch(Swift.Array<Foundation.JSONValue>, Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "dict", intValue: nil)], debugDescription: "Expected to decode Array<JSONValue> but found a dictionary instead.", underlyingError: nil))

$ jw-docker-swift-main swift -version
docker.io/swiftlang/swift:nightly-main
Swift version 5.6-dev (LLVM 7b20e61dd04138a, Swift 9438cf6b2e83c5f)
Target: x86_64-unknown-linux-gnu

is also affected

[weissi]

@swift-ci create

[weissi]

The shortest repro for this bug is probably

  8> String(decoding: JSONEncoder().encode(Dictionary<Double, String>()), as: UTF8.self) 
$R6: String = "{}"

note: encoded as JSON dictionary.

whereas

  9> String(decoding: JSONEncoder().encode(Dictionary<Double, String>(uniqueKeysWithValues: [(1, "")])), as: UTF8.self) 
$R7: String = "[1,\"\"]"

note: encoded as JSON array.

[weissi]

Oh, this is the bug I think:

https://github.com/apple/swift-corelibs-foundation/blob/main/Sources/Foundation/JSONEncoder.swift#L449-L450

        case let object as [String: Encodable]: // this emits a warning, but it works perfectly
            return try self.wrapObject(object, for: nil)

this isn't right in Swift (maybe that's what the warning's about)?

The empty Array/Dictionary is always the same value irregardless of the type (emptyArrayStorage/emptyDictionaryStorage):

Obviously I can't go from a real Double:Double to a String:Double

$R0: [String : Double]? = 0 key/value pairs
  2> (Dictionary<Double,Double>(uniqueKeysWithValues: [(1, 1)]) as Any) as? [String: Double] 
$R1: [String : Double]? = nil

BUT: I can if it's empty

  3> (Dictionary<Double,Double>(uniqueKeysWithValues: []) as Any) as? [String: Double] 
$R2: [String : Double]? = 0 key/value pairs

CC @fabianfett/@millenomi

[millenomi]

See #3138 and linked for fix.