Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SR-9232] Optionals with nil value do not get encoded to JSON #3594

Open
swift-ci opened this issue Nov 13, 2018 · 14 comments
Open

[SR-9232] Optionals with nil value do not get encoded to JSON #3594

swift-ci opened this issue Nov 13, 2018 · 14 comments

Comments

@swift-ci
Copy link
Contributor

@swift-ci swift-ci commented Nov 13, 2018

Previous ID SR-9232
Radar None
Original Reporter Peter de Kraker (JIRA User)
Type Bug
Additional Detail from JIRA
Votes 22
Component/s Foundation
Labels Bug, Codable
Assignee bendjones (JIRA)
Priority Medium

md5: d40e1037406253fafe704bc8c33c89d9

Issue Description:

The implementation of Codable and JSONEncoder in Swift 4 do automatically discard optional fields that have nil as value.

I can understand that this is an option, but there is no option to be set and that makes it a big problem for working with JSON API's that expect nulls for nil fields.

I don't really understand why this default is chosen, since removing nulls from JSON is easier afterwards, then adding them manually by needing to write the encode() method for each class.

Anyways, it would be great if an option can be provided to automatically encode nil values to null with JSONEncoder.

class Test : Codable
{
    var optional:Int?
}


let test:Test = Test()
test.optional = nil


let data = try! JSONEncoder().encode(test)
let string = String(data: data, encoding: String.Encoding.utf8)!
print(string) -> "{}"
@belkadan
Copy link

@belkadan belkadan commented Nov 13, 2018

You can always write encode(to:) yourself, so you're not prevented from doing this. A setting across all of JSONEncoder would be much less flexible.

cc @itaiferber in case he has anything to add

@swift-ci
Copy link
Contributor Author

@swift-ci swift-ci commented Nov 13, 2018

Comment by Peter de Kraker (JIRA)

Yes, I understand that it can be done manually, but in my use case, and probably for a lot of other people, I have a API with lots of inputtypes, that I just want to match with simple data classes. To manually code the encode of everyone is a lot of work, and that's where Swift could (and should 🙂 ) make it easier on the developer. Especially because removing nils afterwards is easy, adding them is not.

Why assume that API's dont take nulls in JSON? Isn't that a bit unrealistic?

@belkadan
Copy link

@belkadan belkadan commented Nov 13, 2018

The assumption isn't necessarily just that APIs don't take nulls, but also that many treat null and absent as interchangeable, and that smaller payloads are better. If Swift is both the encoder and the decoder, for example, both of these are the case.

@swift-ci
Copy link
Contributor Author

@swift-ci swift-ci commented Dec 25, 2018

Comment by Sergey Balalaev (JIRA)

I think it's would be better, if we take a option as DateEncodingStrategy something like that: OptionalEncodingStrategy.

@swift-ci
Copy link
Contributor Author

@swift-ci swift-ci commented Dec 30, 2018

Comment by Mukesh (JIRA)

Would second sof.bix (JIRA User). For example rest api's have PATCH method, where only partial body is sent and if nulls are discarded, there wouldn't be a way to make an value null once it has been entered even if it optional.

@swift-ci
Copy link
Contributor Author

@swift-ci swift-ci commented Jan 15, 2019

Comment by Jon M Snyder (JIRA)

The difference between a client of an API knowing about something and setting it to null and not including the key at all can be important in many situations. Being able to still leverage all the goodness of Encodable and just tweak this behavior would be wonderful.

@swift-ci
Copy link
Contributor Author

@swift-ci swift-ci commented Jun 19, 2019

Comment by Jens Miltner (JIRA)

I would also like to second sof.bix (JIRA User) - not always are we in control of both frontend and backend and if backend treats null values and missing values differently, it's currently a pain to have to add a custom encode(to🙂 just to encode those null values (especially if there are a lot of different structs)

@swift-ci
Copy link
Contributor Author

@swift-ci swift-ci commented Oct 4, 2019

Comment by Nicolai Cornelis (JIRA)

I was pretty surprised to find that nils simply get thrown away as if they're meaningless. This is not really a great design choice in my opinion. The main benefit of using Encodable/Decodable is that you don't have to write all of the code for serialisation. Now we have to do it anyway if we have situations where an API expects null - or where null means something different than absent. This could be "change this property to null" vs. "don't change this property".

I like Sergey's suggestion. Please take another look at this.

@swift-ci
Copy link
Contributor Author

@swift-ci swift-ci commented Oct 13, 2019

Comment by Johan Nordberg (JIRA)

I also second Sergey's OptionalEncodingStrategy suggestion.

There is a third-party JSONEncoder in FineJSON that allows you to encode Optionals as null as a workaround until that lands.

EDIT: Just in case someone else who reads this tries to implement a custom Encoder that handles nil values. This might be obvious for people more familiar with Swift's type system but it took me for a spin: You need to implement all 13 specialized versions of KeyedEncodingContainerProtocol's encodeIfPresent it's not enough to just implement the generic version like you can do with encode.

@swift-ci
Copy link
Contributor Author

@swift-ci swift-ci commented Oct 14, 2019

Comment by Ben D. Jones (JIRA)

@swift-ci sync

@swift-ci
Copy link
Contributor Author

@swift-ci swift-ci commented Dec 20, 2019

Comment by Nicolai Cornelis (JIRA)

Hello

Any updates on this? This is currently quite the headache for us.

@BrentMifsud
Copy link

@BrentMifsud BrentMifsud commented Mar 23, 2020

This is a huge issue for my team as well, as our back end uses explicit nil values for certain purposes.

@swift-ci
Copy link
Contributor Author

@swift-ci swift-ci commented Jun 11, 2020

Comment by Steven Grosmark (JIRA)

As a work-around, I've been using a property wrapper, along the lines of:

@propertyWrapper
public struct NullCodable<Wrapped: Encodable>: Encodable {
  public var wrappedValue: Wrapped?

  public func encode(to encoder: Encoder) throws {
    var container = encoder.singleValueContainer()
    switch wrappedValue {
    case .some(let value): try value.encode(to: encoder)
    case .none: try container.encodeNil()
  }
}

struct Test: Encodable {
  @NullCodable var name: String? = nil
}

JSONEncoder().encode(Test()) // -> "{\"name\": null}"

Not ideal, since each optional property needs to be marked, but it is a little easier than trying to implement a custom encoder.

I created a small package for it here: NullCodable.

@swift-ci
Copy link
Contributor Author

@swift-ci swift-ci commented Jan 21, 2022

Comment by Marcin Iwanicki (JIRA)

There are pros and cons of both cases.

Omitting optional properties results in a smaller payload. JSON format is frequently used in the networking layer as such the size of the payload matters.

I think it would be great to preserve the current behavior as default and avoid breaking changes. Adding a new property e.g. nilEncodingStrategy with options like skip (default), and includeNulls would work well, and be consistent with other options e.g. dataEncodingStrategy, dateEncodingStrategy.
As an alternative, it could possibly leverage the existing outputFormatting property with a new flag e.g. withNulls.

@swift-ci swift-ci transferred this issue from apple/swift-issues Apr 25, 2022
@shahmishal shahmishal transferred this issue from apple/swift May 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants