Codable firestore #838

alickbass · Feb 22, 2018

Hi guys! So this is a WIP PR to solve #627. It is based on #815.

In general, code is almost 1-to-1 port of JSONEncoder from Foundation and PlistEncoder with some minor changes to fit it to Firestore. Most of the code is just boilerplate and the most interesting part is in

func unbox<T : Decodable>(_ value: Any, as type: T.Type) throws -> T?

in FirestoreDecoder and

func box_<T : Encodable>(_ value: T) throws -> NSObject?

in FirestoreEncoder.

I also created internal types Firestore.Encoder and Firestore.Decoder to able to test without needing DocumentReference and DocumentSnapshot. Everything is internal now without any public methods, so that we can already start testing and deciding if it's the right approach without the pain of approving the public API

//сс @wilhuff @paulb777

googlebot · Feb 22, 2018

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.
If your company signed a CLA, they designated a Point of Contact who decides which employees are authorized to participate. You may need to contact the Point of Contact for your company and ask to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the project maintainer to go/cla#troubleshoot. The email used to register you as an authorized contributor must be the email used for the Git commit.
In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

alickbass · Feb 22, 2018

I signed it!

googlebot · Feb 22, 2018

CLAs look good, thanks!

wilhuff · Feb 24, 2018

wilhuff requested changes Feb 24, 2018

View changes

I've taken a pass through the decoder and note that there is a ton of effectively duplicate code. It's very hard to tell what's unique about each overload in here.

I've made some suggestions for how to cut down on this.

wilhuff · Feb 24, 2018

Firestore/Swift/Source/Codable/CodableDocumentReference.swift

@@ -0,0 +1,38 @@
+//


All files need a copyright notice to be in this repo. You can copy from the top of any existing file and adjust the date or use this:

/* * Copyright 2018 Google * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */

wilhuff · Feb 24, 2018

Firestore/Swift/Source/Codable/CodableDocumentReference.swift

+import FirebaseFirestore
+
+enum FirestoreDecodingError: Error {
+    case decodingIsNotSupported


Google's coding style requires two space indents. Rather than fix this by hand please wait for PR #847 to land and run that here to fix this.

wilhuff · Feb 24, 2018

Firestore/Swift/Source/Codable/CodableFieldValue.swift

+import FirebaseFirestore
+
+/**
+ * A protocol describing the encodable properties of a FirebaseFirestore.


This is the encodable properties of a FieldValue, no?

wilhuff · Feb 24, 2018

Firestore/Swift/Source/Codable/CodableDocumentReference.swift

@@ -0,0 +1,38 @@
+//


I just re-read our internal style guide and realized that these files should really be Class+Protocol.swift. In this case DocumentReference+Codable.swift. Sorry to lead you astray with the other PR :-(.

wilhuff · Feb 24, 2018

Firestore/Swift/Source/Codable/CodableDocumentReference.swift

+
+import FirebaseFirestore
+
+enum FirestoreDecodingError: Error {


These don't have much to do with DocumentReference. Perhaps put these in an Errors.swift?

wilhuff · Feb 24, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+            throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: decoder.codingPath + [_FirestoreKey(index: currentIndex)], debugDescription: "Unkeyed container is at end."))
+        }
+
+        decoder.codingPath.append(_FirestoreKey(index: currentIndex))


If this (and the defer) were at the top of function then we wouldn't have to append to to the codingPath to compose the error message for the !isAtEnd case.

wilhuff · Feb 24, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+        defer { decoder.codingPath.removeLast() }
+
+        guard let decoded = try decoder.unbox(container[currentIndex], as: Bool.self) else {
+            throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: decoder.codingPath + [_FirestoreKey(index: currentIndex)], debugDescription: "Expected \(type) but found null instead."))


Also decoder.codingPath already has _FirestoreKey(index: currentIndex) appended to the end of it; this will duplicate the index in the coding path.

wilhuff · Feb 24, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+        guard !self.isAtEnd else {
+            throw DecodingError.valueNotFound(KeyedDecodingContainer<NestedKey>.self,
+                                              DecodingError.Context(codingPath: self.codingPath,
+                                                                    debugDescription: "Cannot get nested keyed container -- unkeyed container is at end."))


Despite the different error message, this could probably also use expectNotAtEnd. I'm not sure there's any special value in giving a slightly better error message for cases that seem extremely unlikely.

wilhuff · Feb 24, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+        }
+
+        let value = self.container[self.currentIndex]
+        guard !(value is NSNull) else {


Could use requireValue.

We cannot really use requireValue here as it expects the Optional and we are checking for the NSNull

wilhuff · Feb 24, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+                                                                    debugDescription: "Cannot get keyed decoding container -- found null value instead."))
+        }
+
+        guard let dictionary = value as? [String : Any] else {


This through the bit that constructs the KeyedDecodingContainer seems as if it can be a method we factor out on the top-level decoder.

public func container<Key>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> { return try container(of: storage.topContainer, keyedBy: type) } public func container<Key>(of value: Any, keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> { guard !(value is NSNull) else { throw DecodingError.valueNotFound(KeyedDecodingContainer<Key>.self, DecodingError.Context(codingPath: codingPath, debugDescription: "Cannot get keyed decoding container -- found null value instead.")) } guard let dictionary = value as? [String: Any] else { let context = DecodingError.Context(codingPath: codingPath, debugDescription: "Not a dictionary") throw DecodingError.typeMismatch([String: Any].self, context) } let container = _FirestoreKeyedDecodingContainer<Key>(referencing: self, wrapping: dictionary) return KeyedDecodingContainer(container) }

googlebot · Feb 27, 2018

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this State. It's up to you to confirm consent of the commit author(s) and merge this pull request when appropriate.

alickbass · Feb 27, 2018

@wilhuff I merged master into this branch... And now I am not sure what should I do :) Should I create another PR against master?

wilhuff · Feb 28, 2018

Just keep going. If you're really concerned about the clarity of the PR stack, feel free to rebase your changes and force push to your repository. We're going to squash merge your change into canonical history so it doesn't matter too much if your changes are all that clear at the commit level.

…

On Tue, Feb 27, 2018 at 7:44 AM Oleksii Dykan ***@***.***> wrote: @wilhuff <https://github.com/wilhuff> I merged master into this branch... And now I am not sure what should I do :) Should I create another PR against master? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#838 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJxjkPzXejXuGEt0sJQM_mbBBiYa9jahks5tZCLagaJpZM4SPic9> .

wilhuff · Mar 1, 2018

The base PR from which this started has since been merged to master. I've retargeted this PR against master and now the diff is clean again.

wilhuff · Mar 3, 2018

wilhuff reviewed Mar 3, 2018

View changes

Decoder changes are great. I have minor naming feedback (see specific callouts) and a larger question about how to handle values beyond the range of the intended storage type. JSONDecoder seems to want to fail, but in other context Firestore just does what a native cast would do.

wilhuff · Mar 3, 2018

Firestore/Example/Firestore.xcodeproj/xcshareddata/xcschemes/Firestore-Example.xcscheme

@@ -49,6 +49,16 @@
               ReferencedContainer = "container:Firestore.xcodeproj">
            </BuildableReference>
         </TestableReference>
+         <TestableReference


The base state from which you branched was broken. I've since fixed this, but unfortunately these diffs remain. For all the .xcscheme files, make sure your upstream branch is up-to-date and then git checkout the master version of these files and commit them. After pushing you shouldn't see any additions to project files.

wilhuff · Mar 3, 2018

Firestore/Swift/Source/Codable/Errors.swift

@@ -0,0 +1,16 @@
+//


Copyright notice.

I realize this isn't super important to you, but this is the only way that Google allows me to continue to work in the open with you, so it's very important to me :-).

wilhuff · Mar 3, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+}
+
+@available(swift 4.0.0)
+fileprivate class _FirestoreDecoder : Decoder {


Is Firestore.DocumentDecoder a possibility? I'm still not clear on exactly what the nested struct within Firestore is for. To me this seems like it should be the public interface on the extension of DocumentSnapshot.

That is to say DocumentDecoder becomes internal, and we create an extension of DocumentSnapshot that uses it (as above) to implement the public API for decoding to an object of a Decodable type.

wilhuff · Mar 3, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+}
+
+@available(swift 4.0.0)
+fileprivate struct _FirestoreDecodingStorage {


Could this be named DocumentDecodingStorage?

My basic concern is that "Firestore decoding" is too general. For example, we could, in the future decode whole query snapshots, in which case a "Firestore decoder" is insufficient.

wilhuff · Mar 3, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+}
+
+@available(swift 4.0.0)
+fileprivate struct _FirestoreKeyedDecodingContainer<K : CodingKey> : KeyedDecodingContainerProtocol {


DocumentKeyedDecodingContainer?

wilhuff · Mar 3, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+    }
+
+    public func decode(_ type: Bool.Type, forKey key: Key) throws -> Bool {
+        let entry = try require(key: key)


This looks substantially better. Thank you for working through it.

wilhuff · Mar 3, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+}
+
+@available(swift 4.0.0)
+fileprivate struct _FirestoreUnkeyedDecodingContainer : UnkeyedDecodingContainer {


DocumentUnkeyedDecodingContainer?

wilhuff · Mar 3, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+                return false
+            }
+
+            /* FIXME: If swift-corelibs-foundation doesn't change to use NSNumber, this code path will need to be included and tested:


How are these FIXMEs to be addressed? I assume it comes from the implementation to which you referred.

Do you think there a way to add a check to make sure that we're not messing this up? I'm more concerned with correctness than speed so an extra else case here (after the two cases that are realistically possible) doesn't bother me.

wilhuff · Mar 3, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+
+        let int8 = number.int8Value
+        guard NSNumber(value: int8) == number else {
+            throw DecodingError.dataCorrupted(DecodingError.Context(codingPath: self.codingPath, debugDescription: "Parsed JSON number <\(number)> does not fit in \(type)."))


"Parsed JSON" no longer applies. Maybe "Firestore Datasnapshot"?

wilhuff · Mar 3, 2018

Firestore/Swift/Source/Codable/FirestoreDecoder.swift

+            // * If it was a Float, you will get back the precise value
+            // * If it was a Double or Decimal, you will get back the nearest approximation if it will fit
+            let double = number.doubleValue
+            guard abs(double) <= Double(Float.greatestFiniteMagnitude) else {


@mikelehen What's the right way to respond to this based on our Android implementation?

It seems like CustomClassMapper just relies on "cast" operations (Number#floatValue): non-failing, saturating casts that would have e.g. positive values beyond ~3.4 x 10^38 saturate to Infinity when assigned to a model value containing a Float instead of a double. Isn't a similar response appropriate here?

wilhuff · Mar 3, 2018

wilhuff reviewed Mar 3, 2018

View changes

In general the encoder stuff is far less egregious and doesn't seem like it needs much work (aside from naming).

wilhuff · Mar 3, 2018

Firestore/Swift/Source/Codable/FirestoreEncoder.swift

+}
+
+@available(swift 4.0.0)
+fileprivate class _FirestoreEncoder : Encoder {


The names in here should parallel the resolution on the decoder (i.e. Document*)

phatmann · Apr 13, 2018

Any progress on this? Would be great to have this part of the SDK.

siyao1030 · May 7, 2018

Thanks for the hard work @alickbass @wilhuff!
Do you have any updates on the status of this issue/pull request? Would love to use it soon!

jorgej-ramos · Jul 3, 2018

A great work @alickbass @wilhuff. Thank you both.
Any updates about this PR? It's a really needed feature.

swftvsn · Jul 25, 2018

We really need this for our iOS project, so please please keep up the good work @alickbass @wilhuff!
Any updates about this PR? Can I help?

Nailer · Nov 5, 2018

Hi guys! What's the progress on this beautiful and highly anticipated feature?

fenixsolorzano · Nov 8, 2018

Any update on this?

paulb777 · Dec 13, 2018

Closing and continuing in #2178

googlebot added the cla: no label Feb 22, 2018

googlebot added cla: yes and removed cla: no labels Feb 22, 2018

paulb777 added the Firestore label Feb 23, 2018

alickbass added some commits Feb 26, 2018

Fix copyright

Loading status checks…

889f29a

Merge master into codable-firestore

Loading status checks…

4fed07f

googlebot added cla: no and removed cla: yes labels Feb 27, 2018

wilhuff changed the base branch from wilhuff/codable to master Mar 1, 2018

wilhuff added cla: yes and removed cla: no labels Mar 3, 2018

wilhuff referenced this pull request Jul 30, 2018
Merged
Add build infrastructure for Codable support in Firestore #815

phatmann referenced this pull request Oct 6, 2018
Open
Support keyDecodingStrategy property for snake case field names #34

paulb777 referenced this pull request Dec 12, 2018
Closed
Continue Firestore Codable #2178

paulb777 closed this Dec 13, 2018

paulb777 referenced this pull request Dec 19, 2018
Open
Demonstration of WIP Firestore Codable #609

This was referenced Jan 3, 2019

Merged

Squash #2178 to continue Firestore Codable #2228

Open

Continue Firestore Codable 3 #2229

firebase/firebase-ios-sdk

Join GitHub today

Codable firestore #838

Conversation

alickbass commented Feb 22, 2018 • edited

wilhuff and others added some commits Feb 20, 2018

This comment has been minimized.

googlebot commented Feb 22, 2018

googlebot added the cla: no label Feb 22, 2018

This comment has been minimized.

alickbass commented Feb 22, 2018

This comment has been minimized.

googlebot commented Feb 22, 2018

googlebot added cla: yes and removed cla: no labels Feb 22, 2018

paulb777 added the Firestore label Feb 23, 2018

wilhuff requested changes Feb 24, 2018 View changes

wilhuff left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

alickbass added some commits Feb 26, 2018

alickbass added some commits Feb 27, 2018

This comment has been minimized.

googlebot commented Feb 27, 2018

googlebot added cla: no and removed cla: yes labels Feb 27, 2018

This comment has been minimized.

alickbass commented Feb 27, 2018

This comment has been minimized.

wilhuff commented Feb 28, 2018

wilhuff changed the base branch from wilhuff/codable to master Mar 1, 2018

This comment has been minimized.

wilhuff commented Mar 1, 2018

wilhuff added cla: yes and removed cla: no labels Mar 3, 2018

wilhuff reviewed Mar 3, 2018 View changes

wilhuff left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

wilhuff reviewed Mar 3, 2018 View changes

wilhuff left a comment

This comment has been minimized.

This comment has been minimized.

phatmann commented Apr 13, 2018

This comment has been minimized.

siyao1030 commented May 7, 2018 • edited

This comment has been minimized.

jorgej-ramos commented Jul 3, 2018

This comment has been minimized.

swftvsn commented Jul 25, 2018

wilhuff referenced this pull request Jul 30, 2018

phatmann referenced this pull request Oct 6, 2018

This comment has been minimized.

Nailer commented Nov 5, 2018

This comment has been minimized.

fenixsolorzano commented Nov 8, 2018

paulb777 referenced this pull request Dec 12, 2018

This comment has been minimized.

paulb777 commented Dec 13, 2018

paulb777 closed this Dec 13, 2018

alickbass commented Feb 22, 2018 •

edited

wilhuff requested changes Feb 24, 2018

View changes

wilhuff reviewed Mar 3, 2018

View changes

wilhuff reviewed Mar 3, 2018

View changes

siyao1030 commented May 7, 2018 •

edited