[SR-6631] Swift JSONEncoder is dependent to the locale decimal point policy encoding

[swift-ci]

Previous ID	SR-6631
Radar	None
Original Reporter	b (JIRA User)
Type	Bug

Attachment: Download

Environment

The bug is confirmed on Darwin (but should affect Linux too).

Additional Detail from JIRA

Votes	1
Component/s	Foundation
Labels	Bug
Assignee	None
Priority	Medium

md5: 3eaf41e50aaf89f052741cedf5637529

is duplicated by:

• SR-6642 JSONEncoder on Linux should not produce incorrect JSON when setlocale() is invoked

Issue Description:

When using a locale that does not use "." as decimal separator, the JSONEncoder & JSONSerialization encodes the Double value with the localized decimal separator.

The bug is easy to reproduce sample code :

import Foundation

// Required to call setLocale
import Darwin

// Let's set to french
setlocale(LC_ALL,"fr_FR")
 
struct Shot:Codable{
    let seconds:Double
}
 
let shot = Shot(seconds: 1.1)
 
do{
    let data = try JSONEncoder().encode(shot)
    if let json =  String(data:data, encoding:.utf8){
        // the result is : "{"seconds":1,1000000000000001}"
        // should be : "{"seconds":1.1000000000000001}"
        // The decimal separator should not be "," but "."
        print(json)
    }
}catch{
    print("\(error)")
}
 
exit(EX_OK)
 

Here is my proposed fix in `JSONSerialization.swift` line 308 . Specifying `kCFNumberFormatterDecimalSeparator` is required. `

    private lazy var _numberformatter: CFNumberFormatter = {
        let formatter: CFNumberFormatter
        formatter = CFNumberFormatterCreate(nil, CFLocaleCopyCurrent(), kCFNumberFormatterNoStyle)
        CFNumberFormatterSetProperty(formatter, kCFNumberFormatterMaxFractionDigits, NSNumber(value: 15))
       CFNumberFormatterSetProperty(formatter, kCFNumberFormatterDecimalSeparator,"."._cfObject)
        CFNumberFormatterSetFormat(formatter, "0.###############"._cfObject)
        return formatter
    }()

Gonna try to submit this fix.

[swift-ci]

Comment by Benoit Pereira da Silva (JIRA)

The problem occurs when calling JSONSerialization.data(withJSONObject: dictionary)

//  main.swift
//  doubleEncoding
//
//  Created by Benoit Pereira da silva on 02/12/2017.
//  Copyright © 2017 Pereira da Silva. All rights reserved.
//

import Foundation

// Let's set to french
setlocale(LC_ALL,"fr_FR")

do{
    let dictionary = ["decimalValue":1.1]
    let data = try JSONSerialization.data(withJSONObject: dictionary)
    if let json =  String(data:data, encoding:.utf8){
        // the result is : "{"decimalValue":1,1000000000000001}"
        // The decimal separator should not be "," but "."
        print(json)
    }
}catch{
    print("\(error)")
}

[swift-ci]

Comment by Benoit Pereira da Silva (JIRA)

BUT The problem does not occur within swift-corelibs-foundation test when adding the following test `swift-corelibs-foundation/testJSONSerialization.swift`

 func test_serializeFloatingPointWithANonDotDecimalSeparatorLocale() {
        // We use a locale that is using "," as decimal separator.
        setlocale(LC_ALL,"fr_FR")

        let dictionary = ["decimalValue":1.1]
        do{
            let data = try JSONSerialization.data(withJSONObject: dictionary)
            if let json =  String(data:data, encoding:.utf8){
                XCTAssert(!json.contains(","), "Expected result: \"{\"decimalValue\":1.1000000000000001}\" found: \"\(json)\"" )
            }else{
                XCTFail("Failed during string encoding")
            }
        }catch{
            XCTFail("Failed during serialization")
        }
    }

[gparker42]

That CFLocaleCopyCurrent() looks suspicious. I would expect the number formatter used by JSON parsing to start with an explicit parse-friendly locale such as CFLocalCreate(nil, "en_US_POSIX").

[swift-ci]

Comment by Benoit Pereira da Silva (JIRA)

Greg

1- This problem seems not to be in swift-corelibs-foundation I ve been adding a test in `swift-corelibs-foundation/testJSONSerialization.swift` that is not failing, while my CLI sample fails.

Philippe Hausler has written on swift-corelibs-dev@swift.org : " Building a CLI tool will use the objective-c Foundation in the system so that wont use your freshly built swift-corelibs-foundation. So that means we have a bug in the objc side if this is actually happening on Darwin (which is a completely different issue…) That should be a radar against Foundation and I definitely think that may very well be a bug… "

2- That's why i've been opening a radar https://bugreport.apple.com/web/?problemID=36107307

[swift-ci]

Comment by Benoit Pereira da Silva (JIRA)

Philippe Hausler emitted the hypothesis that the bug https://bugs.swift.org/browse/SR-6631 was a bug in Darwin Objc Foundations. He was right!

The following Objc Implementation is faulting:

 #import <Foundation/Foundation.h>

int main(int argc, const char * argv[]) {
    @autoreleasepool {
        // Let's set to french
        setlocale(LC_ALL,"fr_FR");
        NSDictionary *dictionary = @{@"decimalValue":@1.1};
        NSError* error = nil;
        NSData* data = [NSJSONSerialization dataWithJSONObject:dictionary options:kNilOptions error:&error];
        NSString * json = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
        NSLog(@"%@",json);
    }
    return 0;
}

Result :

2017-12-19 09:08:49.657316+0100 doubleEncodingObjc[2016:82301]{"decimalValue":1,1000000000000001}