New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Codable firestore #838

Closed
wants to merge 30 commits into
base: master
from

Conversation

Projects
None yet
10 participants
@alickbass
Copy link

alickbass commented Feb 22, 2018

Hi guys! So this is a WIP PR to solve #627. It is based on #815.

In general, code is almost 1-to-1 port of JSONEncoder from Foundation and PlistEncoder with some minor changes to fit it to Firestore. Most of the code is just boilerplate and the most interesting part is in

func unbox<T : Decodable>(_ value: Any, as type: T.Type) throws -> T?

in FirestoreDecoder and

func box_<T : Encodable>(_ value: T) throws -> NSObject?

in FirestoreEncoder.

I also created internal types Firestore.Encoder and Firestore.Decoder to able to test without needing DocumentReference and DocumentSnapshot. Everything is internal now without any public methods, so that we can already start testing and deciding if it's the right approach without the pain of approving the public API

//сс @wilhuff @paulb777

wilhuff and others added some commits Feb 20, 2018

Clean up generated Firestore_SwiftTests_iOS test target
  * Move Example/Firestore_SwiftTests_iOS to Swift/Tests
  * Delete generated test case
  * Update Info.plist location to match
Add Firestore_SwiftTests_iOS to the Podfile
Also corresponding xcodeproj changes
@googlebot

This comment has been minimized.

Copy link

googlebot commented Feb 22, 2018

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.


  • If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.
  • If your company signed a CLA, they designated a Point of Contact who decides which employees are authorized to participate. You may need to contact the Point of Contact for your company and ask to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the project maintainer to go/cla#troubleshoot. The email used to register you as an authorized contributor must be the email used for the Git commit.
  • In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

@googlebot googlebot added the cla: no label Feb 22, 2018

@alickbass

This comment has been minimized.

Copy link
Author

alickbass commented Feb 22, 2018

I signed it!

@googlebot

This comment has been minimized.

Copy link

googlebot commented Feb 22, 2018

CLAs look good, thanks!

@googlebot googlebot added cla: yes and removed cla: no labels Feb 22, 2018

@paulb777 paulb777 added the Firestore label Feb 23, 2018

@wilhuff
Copy link
Member

wilhuff left a comment

I've taken a pass through the decoder and note that there is a ton of effectively duplicate code. It's very hard to tell what's unique about each overload in here.

I've made some suggestions for how to cut down on this.

@@ -0,0 +1,38 @@
//

This comment has been minimized.

@wilhuff

wilhuff Feb 24, 2018

Member

All files need a copyright notice to be in this repo. You can copy from the top of any existing file and adjust the date or use this:

/*
 * Copyright 2018 Google
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

This comment has been minimized.

@alickbass
import FirebaseFirestore

enum FirestoreDecodingError: Error {
case decodingIsNotSupported

This comment has been minimized.

@wilhuff

wilhuff Feb 24, 2018

Member

Google's coding style requires two space indents. Rather than fix this by hand please wait for PR #847 to land and run that here to fix this.

import FirebaseFirestore

/**
* A protocol describing the encodable properties of a FirebaseFirestore.

This comment has been minimized.

@wilhuff

wilhuff Feb 24, 2018

Member

This is the encodable properties of a FieldValue, no?

This comment has been minimized.

@alickbass
@@ -0,0 +1,38 @@
//

This comment has been minimized.

@wilhuff

wilhuff Feb 24, 2018

Member

I just re-read our internal style guide and realized that these files should really be Class+Protocol.swift. In this case DocumentReference+Codable.swift. Sorry to lead you astray with the other PR :-(.

This comment has been minimized.

@alickbass
import FirebaseFirestore

enum FirestoreDecodingError: Error {

This comment has been minimized.

@wilhuff

wilhuff Feb 24, 2018

Member

These don't have much to do with DocumentReference. Perhaps put these in an Errors.swift?

This comment has been minimized.

@alickbass
throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: decoder.codingPath + [_FirestoreKey(index: currentIndex)], debugDescription: "Unkeyed container is at end."))
}

decoder.codingPath.append(_FirestoreKey(index: currentIndex))

This comment has been minimized.

@wilhuff

wilhuff Feb 24, 2018

Member

If this (and the defer) were at the top of function then we wouldn't have to append to to the codingPath to compose the error message for the !isAtEnd case.

defer { decoder.codingPath.removeLast() }

guard let decoded = try decoder.unbox(container[currentIndex], as: Bool.self) else {
throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: decoder.codingPath + [_FirestoreKey(index: currentIndex)], debugDescription: "Expected \(type) but found null instead."))

This comment has been minimized.

@wilhuff

wilhuff Feb 24, 2018

Member

Also decoder.codingPath already has _FirestoreKey(index: currentIndex) appended to the end of it; this will duplicate the index in the coding path.

guard !self.isAtEnd else {
throw DecodingError.valueNotFound(KeyedDecodingContainer<NestedKey>.self,
DecodingError.Context(codingPath: self.codingPath,
debugDescription: "Cannot get nested keyed container -- unkeyed container is at end."))

This comment has been minimized.

@wilhuff

wilhuff Feb 24, 2018

Member

Despite the different error message, this could probably also use expectNotAtEnd. I'm not sure there's any special value in giving a slightly better error message for cases that seem extremely unlikely.

This comment has been minimized.

@alickbass
}

let value = self.container[self.currentIndex]
guard !(value is NSNull) else {

This comment has been minimized.

@wilhuff

wilhuff Feb 24, 2018

Member

Could use requireValue.

This comment has been minimized.

@alickbass

alickbass Feb 27, 2018

Author

We cannot really use requireValue here as it expects the Optional and we are checking for the NSNull

This comment has been minimized.

@alickbass
debugDescription: "Cannot get keyed decoding container -- found null value instead."))
}

guard let dictionary = value as? [String : Any] else {

This comment has been minimized.

@wilhuff

wilhuff Feb 24, 2018

Member

This through the bit that constructs the KeyedDecodingContainer seems as if it can be a method we factor out on the top-level decoder.

  public func container<Key>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> {
    return try container(of: storage.topContainer, keyedBy: type)
  }

  public func container<Key>(of value: Any, keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> {
    guard !(value is NSNull) else {
      throw DecodingError.valueNotFound(KeyedDecodingContainer<Key>.self,
                                        DecodingError.Context(codingPath: codingPath,
                                                              debugDescription: "Cannot get keyed decoding container -- found null value instead."))
    }

    guard let dictionary = value as? [String: Any] else {
      let context = DecodingError.Context(codingPath: codingPath, debugDescription: "Not a dictionary")
      throw DecodingError.typeMismatch([String: Any].self, context)
    }

    let container = _FirestoreKeyedDecodingContainer<Key>(referencing: self, wrapping: dictionary)
    return KeyedDecodingContainer(container)
  }

alickbass added some commits Feb 26, 2018

@googlebot

This comment has been minimized.

Copy link

googlebot commented Feb 27, 2018

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this State. It's up to you to confirm consent of the commit author(s) and merge this pull request when appropriate.

@googlebot googlebot added cla: no and removed cla: yes labels Feb 27, 2018

@alickbass

This comment has been minimized.

Copy link
Author

alickbass commented Feb 27, 2018

@wilhuff I merged master into this branch... And now I am not sure what should I do :) Should I create another PR against master?

@wilhuff

This comment has been minimized.

Copy link
Member

wilhuff commented Feb 28, 2018

@wilhuff wilhuff changed the base branch from wilhuff/codable to master Mar 1, 2018

@wilhuff

This comment has been minimized.

Copy link
Member

wilhuff commented Mar 1, 2018

The base PR from which this started has since been merged to master. I've retargeted this PR against master and now the diff is clean again.

@wilhuff wilhuff added cla: yes and removed cla: no labels Mar 3, 2018

@wilhuff
Copy link
Member

wilhuff left a comment

Decoder changes are great. I have minor naming feedback (see specific callouts) and a larger question about how to handle values beyond the range of the intended storage type. JSONDecoder seems to want to fail, but in other context Firestore just does what a native cast would do.

@@ -49,6 +49,16 @@
ReferencedContainer = "container:Firestore.xcodeproj">
</BuildableReference>
</TestableReference>
<TestableReference

This comment has been minimized.

@wilhuff

wilhuff Mar 3, 2018

Member

The base state from which you branched was broken. I've since fixed this, but unfortunately these diffs remain. For all the .xcscheme files, make sure your upstream branch is up-to-date and then git checkout the master version of these files and commit them. After pushing you shouldn't see any additions to project files.

@@ -0,0 +1,16 @@
//

This comment has been minimized.

@wilhuff

wilhuff Mar 3, 2018

Member

Copyright notice.

I realize this isn't super important to you, but this is the only way that Google allows me to continue to work in the open with you, so it's very important to me :-).

}

@available(swift 4.0.0)
fileprivate class _FirestoreDecoder : Decoder {

This comment has been minimized.

@wilhuff

wilhuff Mar 3, 2018

Member

Is Firestore.DocumentDecoder a possibility? I'm still not clear on exactly what the nested struct within Firestore is for. To me this seems like it should be the public interface on the extension of DocumentSnapshot.

That is to say DocumentDecoder becomes internal, and we create an extension of DocumentSnapshot that uses it (as above) to implement the public API for decoding to an object of a Decodable type.

}

@available(swift 4.0.0)
fileprivate struct _FirestoreDecodingStorage {

This comment has been minimized.

@wilhuff

wilhuff Mar 3, 2018

Member

Could this be named DocumentDecodingStorage?

My basic concern is that "Firestore decoding" is too general. For example, we could, in the future decode whole query snapshots, in which case a "Firestore decoder" is insufficient.

}

@available(swift 4.0.0)
fileprivate struct _FirestoreKeyedDecodingContainer<K : CodingKey> : KeyedDecodingContainerProtocol {

This comment has been minimized.

@wilhuff

wilhuff Mar 3, 2018

Member

DocumentKeyedDecodingContainer?

}

public func decode(_ type: Bool.Type, forKey key: Key) throws -> Bool {
let entry = try require(key: key)

This comment has been minimized.

@wilhuff

wilhuff Mar 3, 2018

Member

This looks substantially better. Thank you for working through it.

}

@available(swift 4.0.0)
fileprivate struct _FirestoreUnkeyedDecodingContainer : UnkeyedDecodingContainer {

This comment has been minimized.

@wilhuff

wilhuff Mar 3, 2018

Member

DocumentUnkeyedDecodingContainer?

return false
}

/* FIXME: If swift-corelibs-foundation doesn't change to use NSNumber, this code path will need to be included and tested:

This comment has been minimized.

@wilhuff

wilhuff Mar 3, 2018

Member

How are these FIXMEs to be addressed? I assume it comes from the implementation to which you referred.

Do you think there a way to add a check to make sure that we're not messing this up? I'm more concerned with correctness than speed so an extra else case here (after the two cases that are realistically possible) doesn't bother me.


let int8 = number.int8Value
guard NSNumber(value: int8) == number else {
throw DecodingError.dataCorrupted(DecodingError.Context(codingPath: self.codingPath, debugDescription: "Parsed JSON number <\(number)> does not fit in \(type)."))

This comment has been minimized.

@wilhuff

wilhuff Mar 3, 2018

Member

"Parsed JSON" no longer applies. Maybe "Firestore Datasnapshot"?

// * If it was a Float, you will get back the precise value
// * If it was a Double or Decimal, you will get back the nearest approximation if it will fit
let double = number.doubleValue
guard abs(double) <= Double(Float.greatestFiniteMagnitude) else {

This comment has been minimized.

@wilhuff

wilhuff Mar 3, 2018

Member
  • @mikelehen What's the right way to respond to this based on our Android implementation?

It seems like CustomClassMapper just relies on "cast" operations (Number#floatValue): non-failing, saturating casts that would have e.g. positive values beyond ~3.4 x 10^38 saturate to Infinity when assigned to a model value containing a Float instead of a double. Isn't a similar response appropriate here?

@wilhuff
Copy link
Member

wilhuff left a comment

In general the encoder stuff is far less egregious and doesn't seem like it needs much work (aside from naming).

}

@available(swift 4.0.0)
fileprivate class _FirestoreEncoder : Encoder {

This comment has been minimized.

@wilhuff

wilhuff Mar 3, 2018

Member

The names in here should parallel the resolution on the decoder (i.e. Document*)

@phatmann

This comment has been minimized.

Copy link

phatmann commented Apr 13, 2018

Any progress on this? Would be great to have this part of the SDK.

@siyao1030

This comment has been minimized.

Copy link

siyao1030 commented May 7, 2018

Thanks for the hard work @alickbass @wilhuff!
Do you have any updates on the status of this issue/pull request? Would love to use it soon!

@jorgej-ramos

This comment has been minimized.

Copy link

jorgej-ramos commented Jul 3, 2018

A great work @alickbass @wilhuff. Thank you both.
Any updates about this PR? It's a really needed feature.

@swftvsn

This comment has been minimized.

Copy link

swftvsn commented Jul 25, 2018

We really need this for our iOS project, so please please keep up the good work @alickbass @wilhuff!
Any updates about this PR? Can I help?

@Nailer

This comment has been minimized.

Copy link

Nailer commented Nov 5, 2018

Hi guys! What's the progress on this beautiful and highly anticipated feature?

@fenixsolorzano

This comment has been minimized.

Copy link

fenixsolorzano commented Nov 8, 2018

Any update on this?

@paulb777

This comment has been minimized.

Copy link
Member

paulb777 commented Dec 13, 2018

Closing and continuing in #2178

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment