Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Codable firestore #838

Closed
wants to merge 30 commits into from
Closed

Codable firestore #838

wants to merge 30 commits into from

Conversation

@alickbass
Copy link

@alickbass alickbass commented Feb 22, 2018

Hi guys! So this is a WIP PR to solve #627. It is based on #815.

In general, code is almost 1-to-1 port of JSONEncoder from Foundation and PlistEncoder with some minor changes to fit it to Firestore. Most of the code is just boilerplate and the most interesting part is in

func unbox<T : Decodable>(_ value: Any, as type: T.Type) throws -> T?

in FirestoreDecoder and

func box_<T : Encodable>(_ value: T) throws -> NSObject?

in FirestoreEncoder.

I also created internal types Firestore.Encoder and Firestore.Decoder to able to test without needing DocumentReference and DocumentSnapshot. Everything is internal now without any public methods, so that we can already start testing and deciding if it's the right approach without the pain of approving the public API

//сс @wilhuff @paulb777

@googlebot
Copy link

@googlebot googlebot commented Feb 22, 2018

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.


  • If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.
  • If your company signed a CLA, they designated a Point of Contact who decides which employees are authorized to participate. You may need to contact the Point of Contact for your company and ask to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the project maintainer to go/cla#troubleshoot. The email used to register you as an authorized contributor must be the email used for the Git commit.
  • In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

@alickbass
Copy link
Author

@alickbass alickbass commented Feb 22, 2018

I signed it!

@googlebot
Copy link

@googlebot googlebot commented Feb 22, 2018

CLAs look good, thanks!

Copy link
Contributor

@wilhuff wilhuff left a comment

I've taken a pass through the decoder and note that there is a ton of effectively duplicate code. It's very hard to tell what's unique about each overload in here.

I've made some suggestions for how to cut down on this.

@@ -0,0 +1,38 @@
//
Copy link
Contributor

@wilhuff wilhuff Feb 24, 2018

All files need a copyright notice to be in this repo. You can copy from the top of any existing file and adjust the date or use this:

/*
 * Copyright 2018 Google
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

Copy link
Author

@alickbass alickbass Feb 28, 2018

Done

import FirebaseFirestore

enum FirestoreDecodingError: Error {
case decodingIsNotSupported
Copy link
Contributor

@wilhuff wilhuff Feb 24, 2018

Google's coding style requires two space indents. Rather than fix this by hand please wait for PR #847 to land and run that here to fix this.

import FirebaseFirestore

/**
* A protocol describing the encodable properties of a FirebaseFirestore.
Copy link
Contributor

@wilhuff wilhuff Feb 24, 2018

This is the encodable properties of a FieldValue, no?

Copy link
Author

@alickbass alickbass Feb 28, 2018

Done

@@ -0,0 +1,38 @@
//
Copy link
Contributor

@wilhuff wilhuff Feb 24, 2018

I just re-read our internal style guide and realized that these files should really be Class+Protocol.swift. In this case DocumentReference+Codable.swift. Sorry to lead you astray with the other PR :-(.

Copy link
Author

@alickbass alickbass Feb 28, 2018

Done

import FirebaseFirestore

enum FirestoreDecodingError: Error {
Copy link
Contributor

@wilhuff wilhuff Feb 24, 2018

These don't have much to do with DocumentReference. Perhaps put these in an Errors.swift?

Copy link
Author

@alickbass alickbass Feb 28, 2018

Done

throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: decoder.codingPath + [_FirestoreKey(index: currentIndex)], debugDescription: "Unkeyed container is at end."))
}

decoder.codingPath.append(_FirestoreKey(index: currentIndex))
Copy link
Contributor

@wilhuff wilhuff Feb 24, 2018

If this (and the defer) were at the top of function then we wouldn't have to append to to the codingPath to compose the error message for the !isAtEnd case.

defer { decoder.codingPath.removeLast() }

guard let decoded = try decoder.unbox(container[currentIndex], as: Bool.self) else {
throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: decoder.codingPath + [_FirestoreKey(index: currentIndex)], debugDescription: "Expected \(type) but found null instead."))
Copy link
Contributor

@wilhuff wilhuff Feb 24, 2018

Also decoder.codingPath already has _FirestoreKey(index: currentIndex) appended to the end of it; this will duplicate the index in the coding path.

guard !self.isAtEnd else {
throw DecodingError.valueNotFound(KeyedDecodingContainer<NestedKey>.self,
DecodingError.Context(codingPath: self.codingPath,
debugDescription: "Cannot get nested keyed container -- unkeyed container is at end."))
Copy link
Contributor

@wilhuff wilhuff Feb 24, 2018

Despite the different error message, this could probably also use expectNotAtEnd. I'm not sure there's any special value in giving a slightly better error message for cases that seem extremely unlikely.

Copy link
Author

@alickbass alickbass Feb 28, 2018

Done

}

let value = self.container[self.currentIndex]
guard !(value is NSNull) else {
Copy link
Contributor

@wilhuff wilhuff Feb 24, 2018

Could use requireValue.

Copy link
Author

@alickbass alickbass Feb 27, 2018

We cannot really use requireValue here as it expects the Optional and we are checking for the NSNull

Copy link
Author

@alickbass alickbass Feb 28, 2018

Done

debugDescription: "Cannot get keyed decoding container -- found null value instead."))
}

guard let dictionary = value as? [String : Any] else {
Copy link
Contributor

@wilhuff wilhuff Feb 24, 2018

This through the bit that constructs the KeyedDecodingContainer seems as if it can be a method we factor out on the top-level decoder.

  public func container<Key>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> {
    return try container(of: storage.topContainer, keyedBy: type)
  }

  public func container<Key>(of value: Any, keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> {
    guard !(value is NSNull) else {
      throw DecodingError.valueNotFound(KeyedDecodingContainer<Key>.self,
                                        DecodingError.Context(codingPath: codingPath,
                                                              debugDescription: "Cannot get keyed decoding container -- found null value instead."))
    }

    guard let dictionary = value as? [String: Any] else {
      let context = DecodingError.Context(codingPath: codingPath, debugDescription: "Not a dictionary")
      throw DecodingError.typeMismatch([String: Any].self, context)
    }

    let container = _FirestoreKeyedDecodingContainer<Key>(referencing: self, wrapping: dictionary)
    return KeyedDecodingContainer(container)
  }

@googlebot
Copy link

@googlebot googlebot commented Feb 27, 2018

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this State. It's up to you to confirm consent of the commit author(s) and merge this pull request when appropriate.

@alickbass
Copy link
Author

@alickbass alickbass commented Feb 27, 2018

@wilhuff I merged master into this branch... And now I am not sure what should I do :) Should I create another PR against master?

@wilhuff
Copy link
Contributor

@wilhuff wilhuff commented Feb 28, 2018

@wilhuff wilhuff changed the base branch from wilhuff/codable to master Mar 1, 2018
@wilhuff
Copy link
Contributor

@wilhuff wilhuff commented Mar 1, 2018

The base PR from which this started has since been merged to master. I've retargeted this PR against master and now the diff is clean again.

@wilhuff wilhuff added cla: yes and removed cla: no labels Mar 3, 2018
Copy link
Contributor

@wilhuff wilhuff left a comment

Decoder changes are great. I have minor naming feedback (see specific callouts) and a larger question about how to handle values beyond the range of the intended storage type. JSONDecoder seems to want to fail, but in other context Firestore just does what a native cast would do.

@@ -49,6 +49,16 @@
ReferencedContainer = "container:Firestore.xcodeproj">
</BuildableReference>
</TestableReference>
<TestableReference
Copy link
Contributor

@wilhuff wilhuff Mar 3, 2018

The base state from which you branched was broken. I've since fixed this, but unfortunately these diffs remain. For all the .xcscheme files, make sure your upstream branch is up-to-date and then git checkout the master version of these files and commit them. After pushing you shouldn't see any additions to project files.

@@ -0,0 +1,16 @@
//
Copy link
Contributor

@wilhuff wilhuff Mar 3, 2018

Copyright notice.

I realize this isn't super important to you, but this is the only way that Google allows me to continue to work in the open with you, so it's very important to me :-).

}

@available(swift 4.0.0)
fileprivate class _FirestoreDecoder : Decoder {
Copy link
Contributor

@wilhuff wilhuff Mar 3, 2018

Is Firestore.DocumentDecoder a possibility? I'm still not clear on exactly what the nested struct within Firestore is for. To me this seems like it should be the public interface on the extension of DocumentSnapshot.

That is to say DocumentDecoder becomes internal, and we create an extension of DocumentSnapshot that uses it (as above) to implement the public API for decoding to an object of a Decodable type.

}

@available(swift 4.0.0)
fileprivate struct _FirestoreDecodingStorage {
Copy link
Contributor

@wilhuff wilhuff Mar 3, 2018

Could this be named DocumentDecodingStorage?

My basic concern is that "Firestore decoding" is too general. For example, we could, in the future decode whole query snapshots, in which case a "Firestore decoder" is insufficient.

}

@available(swift 4.0.0)
fileprivate struct _FirestoreKeyedDecodingContainer<K : CodingKey> : KeyedDecodingContainerProtocol {
Copy link
Contributor

@wilhuff wilhuff Mar 3, 2018

DocumentKeyedDecodingContainer?

}

public func decode(_ type: Bool.Type, forKey key: Key) throws -> Bool {
let entry = try require(key: key)
Copy link
Contributor

@wilhuff wilhuff Mar 3, 2018

This looks substantially better. Thank you for working through it.

}

@available(swift 4.0.0)
fileprivate struct _FirestoreUnkeyedDecodingContainer : UnkeyedDecodingContainer {
Copy link
Contributor

@wilhuff wilhuff Mar 3, 2018

DocumentUnkeyedDecodingContainer?

return false
}

/* FIXME: If swift-corelibs-foundation doesn't change to use NSNumber, this code path will need to be included and tested:
Copy link
Contributor

@wilhuff wilhuff Mar 3, 2018

How are these FIXMEs to be addressed? I assume it comes from the implementation to which you referred.

Do you think there a way to add a check to make sure that we're not messing this up? I'm more concerned with correctness than speed so an extra else case here (after the two cases that are realistically possible) doesn't bother me.


let int8 = number.int8Value
guard NSNumber(value: int8) == number else {
throw DecodingError.dataCorrupted(DecodingError.Context(codingPath: self.codingPath, debugDescription: "Parsed JSON number <\(number)> does not fit in \(type)."))
Copy link
Contributor

@wilhuff wilhuff Mar 3, 2018

"Parsed JSON" no longer applies. Maybe "Firestore Datasnapshot"?

// * If it was a Float, you will get back the precise value
// * If it was a Double or Decimal, you will get back the nearest approximation if it will fit
let double = number.doubleValue
guard abs(double) <= Double(Float.greatestFiniteMagnitude) else {
Copy link
Contributor

@wilhuff wilhuff Mar 3, 2018

  • @mikelehen What's the right way to respond to this based on our Android implementation?

It seems like CustomClassMapper just relies on "cast" operations (Number#floatValue): non-failing, saturating casts that would have e.g. positive values beyond ~3.4 x 10^38 saturate to Infinity when assigned to a model value containing a Float instead of a double. Isn't a similar response appropriate here?

Copy link
Contributor

@wilhuff wilhuff left a comment

In general the encoder stuff is far less egregious and doesn't seem like it needs much work (aside from naming).

}

@available(swift 4.0.0)
fileprivate class _FirestoreEncoder : Encoder {
Copy link
Contributor

@wilhuff wilhuff Mar 3, 2018

The names in here should parallel the resolution on the decoder (i.e. Document*)

@phatmann
Copy link

@phatmann phatmann commented Apr 13, 2018

Any progress on this? Would be great to have this part of the SDK.

@siyao1030
Copy link

@siyao1030 siyao1030 commented May 7, 2018

Thanks for the hard work @alickbass @wilhuff!
Do you have any updates on the status of this issue/pull request? Would love to use it soon!

@jorgej-ramos
Copy link

@jorgej-ramos jorgej-ramos commented Jul 3, 2018

A great work @alickbass @wilhuff. Thank you both.
Any updates about this PR? It's a really needed feature.

@swftvsn
Copy link

@swftvsn swftvsn commented Jul 25, 2018

We really need this for our iOS project, so please please keep up the good work @alickbass @wilhuff!
Any updates about this PR? Can I help?

@Nailer
Copy link

@Nailer Nailer commented Nov 5, 2018

Hi guys! What's the progress on this beautiful and highly anticipated feature?

@fenixsolorzano
Copy link

@fenixsolorzano fenixsolorzano commented Nov 8, 2018

Any update on this?

@paulb777
Copy link
Member

@paulb777 paulb777 commented Dec 13, 2018

Closing and continuing in #2178

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

10 participants