feat: add ability to merge updated objects with original

[cbaker6]

New Pull Request Checklist

• I am not disclosing a vulnerability.
• I am creating this PR in reference to an issue.

Issue Description

The Parse Server doesn't support PATCH, but has a custom implementation of PUT which allows the client to only send a subset of keys that need updating. This custom implementation "acts" as a PATCH. See more in #242

In order for the Swift SDK to take advantage of the Parse Servers custom implementation of PUT; to act as a PATCH, developers need to make their ParseObject's conform to ParseObjectMutable and use .mutable on their ParseObject before changing any keys.

Issue: When a developer uses .mutable after conforming to ParseObjectMutable, the returned result from a save or update only returns the updated object, not the updated+original object, meaning the returned object on the client doesn't have the same key/values as the server. In order to retrieve the full updated object, the developer will have to merge the original and updated themselves or in the case of ParseUser.current and ParseInstallation.current, they have to fetch the updated object from the cloud. A related PR is #263.

If a developer isn't using .mutable, it means they are always sending all keys to the server and are essentially doing a true PUT (without upsert, as the Server doesn't support upsert), so this problem isn't relevant in those situations. Similarly, the problem also isn't relevant when creating objects using POST.

Related issue: #242

Approach

Add protocol implementations and helper methods that allow developers to specify all of their keys and allow for the original and updated objects to be merged by using updated.merge(original). This will need to be called by objects that aren't stored in the keychain by the Swift SDK. Keychain objects (those that have .current) have their changes merged by the SDK automatically. This makes the client object match the server object after an update automatically assuming the developer used the renamed mutable->mergeable property on a ParseObject before making the modification and that they implemented the new merge() method.

The following protocol properties need to be added to ALL ParseObject's by ALL developers, resulting in a breaking change:

 /**
     A JSON encoded version of this `ParseObject` before `mergeable` was called and
     properties were changed.
     - warning: This property is not intended to be set or modified by the developer.
*/
    var originalData: Data? { get set }

The following method developers CAN(not required) add to their ParseObjects if they want to take advantage of merging:

/**
      Merges two `ParseObject`'s with the resulting object consisting of all modified
     and unchanged properties.

          //: Create your own value typed `ParseObject`.
          struct GameScore: ParseObject {
              //: These are required by ParseObject
              var objectId: String?
              var createdAt: Date?
              var updatedAt: Date?
              var ACL: ParseACL?
              var originalData: Data? // New property that the compiler will enforce to be added

              //: Your own properties.
              var points: Int?

              //: Implement your own version of merge
              func merge(_ object: Self) throws -> Self {
                  var updated = try mergeParse(object)
                  if updated.shouldRestoreKey(\.points,
                                                   original: object) {
                      updated.points = object.points
                  }
                  return updated
              }
          }

      - parameter object: The original object.
      - returns: The merged object.
      - throws: An error of type `ParseError`.
      - important: It is recommend you provide an implementation of this method
      for all of your `ParseObject`'s as the developer has access to all keys of a
      `ParseObject`. You should always call `mergeParse`
      in the beginning of your implementation to handle all default Parse keys. In addition,
      use `shouldRestoreKey` to compare key modifications between objects.
 */
func merge(_ object: Self) throws -> Self

The following protocol helper methods are now available for developers to use to assist with their implementations of merge():

 /**
      Merges two `ParseObject`'s resulting in modified and unchanged Parse keys.
      - parameter object: The original installation.
      - returns: The updated installation.
      - throws: An error of type `ParseError`.
      - warning: You should only call this method and shouldn't implement it directly
      as it's already implemented for developers to use.
*/
func mergeParse(_ object: Self) throws -> Self

/**
      Determines if a `KeyPath` of the current `ParseObject` should be restored
      by comparing it to another `ParseObject`.
      - parameter original: The original `ParseObject`.
      - returns: Returns a `true` if the keyPath should be restored  or `false` otherwise.
*/
func shouldRestoreKey<W>(_ key: KeyPath<Self, W?>, original: Self) -> Bool where W: Equatable

An example of how this works can be found in the Playgrounds (notice that merge doesn't need to be called by the developer because merges happen automatically:

Parse-Swift/ParseSwift.playground/Pages/6 - Installation.xcplaygroundpage/Contents.swift

Lines 16 to 84 in d180c63

	struct Installation: ParseInstallation {
	//: These are required by `ParseObject`.
	var objectId: String?
	var createdAt: Date?
	var updatedAt: Date?
	var ACL: ParseACL?
	var score: Double?
	var originalData: Data?
	//: These are required by `ParseInstallation`.
	var installationId: String?
	var deviceType: String?
	var deviceToken: String?
	var badge: Int?
	var timeZone: String?
	var channels: [String]?
	var appName: String?
	var appIdentifier: String?
	var appVersion: String?
	var parseVersion: String?
	var localeIdentifier: String?
	//: Your custom keys
	var customKey: String?
	//: Implement your own version of merge
	func merge(_ object: Self) throws -> Self {
	var updated = try mergeParse(object)
	if updated.shouldRestoreKey(\.customKey,
	original: object) {
	updated.customKey = object.customKey
	}
	return updated
	}
	}
	/*: Save your first `customKey` value to your `ParseInstallation`.
	Performs work on background queue and returns to designated on
	designated callbackQueue. If no callbackQueue is specified it
	returns to main queue. Note that this may be the first time you
	are saving your Installation.
	*/
	let currentInstallation = Installation.current
	currentInstallation?.save { results in
	switch results {
	case .success(let updatedInstallation):
	print("Successfully saved Installation to ParseServer: \(updatedInstallation)")
	case .failure(let error):
	print("Failed to update installation: \(error)")
	}
	}
	/*: Update your `ParseInstallation` `customKey` value.
	Performs work on background queue and returns to designated on
	designated callbackQueue. If no callbackQueue is specified it
	returns to main queue.
	*/
	var installationToUpdate = Installation.current?.mutable
	installationToUpdate?.customKey = "myCustomInstallationKey2"
	installationToUpdate?.save { results in
	switch results {
	case .success(let updatedInstallation):
	print("Successfully save myCustomInstallationKey to ParseServer: \(updatedInstallation)")
	case .failure(let error):
	print("Failed to update installation: \(error)")
	}
	}

An example with ParseObject can be seen in Playgrounds:

Parse-Swift/ParseSwift.playground/Pages/1 - Your first Object.xcplaygroundpage/Contents.swift

Lines 32 to 66 in e12b88f

	//: Create your own value typed `ParseObject`.
	struct GameScore: ParseObject {
	//: These are required by ParseObject
	var objectId: String?
	var createdAt: Date?
	var updatedAt: Date?
	var ACL: ParseACL?
	var originalData: Data?
	//: Your own properties.
	var points: Int?
	//: Implement your own version of merge
	func merge(_ object: Self) throws -> Self {
	var updated = try mergeParse(object)
	if updated.shouldRestoreKey(\.points,
	original: object) {
	updated.points = object.points
	}
	return updated
	}
	}
	//: It's recommended to place custom initializers in an extension
	//: to preserve the convenience initializer.
	extension GameScore {
	init(points: Int) {
	self.points = points
	}
	init(objectId: String?) {
	self.objectId = objectId
	}
	}

I might be able to leverage the current implementation to remove the need for the developer to implement merge, but I need to think about it a little more. Basically the developer will need to give the SDK a list of all KeyPaths (for each ParseObject to remove a merge implementation requirement. Update: I wasn't able to come up with a way to make this feasible with key paths.

Notes

The current changes are non-breaking and will be released as a minor update. Later in the year, a major release will occur requiring all keys either be optional for ParseObjects or to have a default value. The current changes are breaking (weak break) and will need to be released as a major update. The compiler will require all ParseObject's to add var originalData: Data?.

In addition, all ParseObject's have mergeable by default. Essentially reverting back #243 to which prevents developers from setting up their ParseObject's incorrectly by not using optionals for all properties and allowing the SDK to be able to initialize new versions of their ParseObject's when necessary. Developers should NOT attempt to enforce properties be defined by defining their property as a non-optional type. Instead, a developer should do this by creating additional initializers (preferably in an extension) and carry out any necessary checks. Enforcing properties to be defined can also be done on the server side via schema setup, Parse Dashboard, or Cloud Code.

TODOs before merging

• Add tests
• Add entry to changelog
• Add changes to documentation (guides, repository pages, in-code descriptions)

[parse-github-assistant]

Thanks for opening this pull request!

• 🎉 We are excited about your hands-on contribution!

[cbaker6]

@dblythy @vdkdamian @Vortec4800 Since you all were part of the discussion about only saving keys of updated objects, can you take a look at this PR to provide your thoughts? Any opinions with naming, how it works, design, etc. will be helpful...

I will add more details to description later today/tomorrow.

[codecov]

Codecov Report

Merging #315 (7fa72f2) into main (3c3a6e8) will increase coverage by 0.33%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #315      +/-   ##
==========================================
+ Coverage   84.63%   84.96%   +0.33%     
==========================================
  Files         114      114              
  Lines       11850    12049     +199     
==========================================
+ Hits        10029    10238     +209     
+ Misses       1821     1811      -10     

Impacted Files	Coverage Δ
...eSwift/InternalObjects/BaseParseInstallation.swift	36.36% <ø> (ø)
Sources/ParseSwift/Types/Query.swift	92.19% <ø> (-0.10%)	⬇️
Sources/ParseSwift/API/API+Command.swift	84.64% <100.00%> (+0.83%)	⬆️
Sources/ParseSwift/Coding/ParseCoding.swift	88.31% <100.00%> (+0.15%)	⬆️
Sources/ParseSwift/Coding/ParseEncoder.swift	74.91% <100.00%> (+0.55%)	⬆️
Sources/ParseSwift/Objects/ParseInstallation.swift	82.36% <100.00%> (+1.70%)	⬆️
Sources/ParseSwift/Objects/ParseObject.swift	82.29% <100.00%> (+0.82%)	⬆️
Sources/ParseSwift/Objects/ParseRole.swift	100.00% <100.00%> (ø)
Sources/ParseSwift/Objects/ParseUser.swift	82.45% <100.00%> (+1.76%)	⬆️
...rces/ParseSwift/Protocols/ParseQueryScorable.swift	100.00% <100.00%> (ø)
... and 4 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c3a6e8...7fa72f2. Read the comment docs.

[vdkdamian]

@dblythy @vdkdamian @Vortec4800 Since you all were part of the discussion about only saving keys of updated objects, can you take a look at this PR to provide your thoughts? Any opinions with naming, how it works, design, etc. will be helpful...

I will add more details to description later today/tomorrow.

I'll take a look at this thursday.

[cbaker6]

I'm thinking of changing the name of one of the added methods, isRestoreOriginalKey -> isShouldRestoreKey

[mtrezza]

notice that merge doesn't need to be called by the developer because ParseInstallation.current merges happen automatically

Is it possible to add that behavior as default for all Parse Objects?

Later in the year, a major release will occur

That's a good reminder that we make the Swift SDK conform to the broader Parse release cycle and deprecation policy sometime this year. Together with release automation.

[cbaker6]

Is it possible to add that behavior as default for all Parse Objects?

It’s not possible in the current context because the SDK doesn’t have the original object. The developer would have pass the original ParseObject to the save method in order for it to be merged automatically.

[vdkdamian]

This definitely is an improvement to fix the current issue, but I do agree that it's making things a little complicated for some developers that may not be following the conversations we had.

If I have some time within a few weeks I'll try to help look for a better solution, but for now this definitely seems like the best way to do things.

I'm thinking of changing the name of one of the added methods, isRestoreOriginalKey -> isShouldRestoreKey

It still seems a little weird to me, what about just using shouldRestoreKey? Is there a reason why u decided to put "is" before it?

[cbaker6]

but I do agree that it's making things a little complicated for some developers that may not be following the conversations we had.

This happens with any repo. Each developer is not expected to keep up with each PR a repo opens/merges. The devs already using the SDK in it's current form and not using ParseObjectMutable probably won't care about this PR and won't need to follow along. They will just need to add var originalObject: Data? to their ParseObject's if they choose to upgrade and use the SDK as they have been doing.

The PR is only important to those who would like to use PATCH (or in the Parse Servers current context, a quirky version of PUT that acts as a PATCH). For these devs, they will want to read the README, release notes, and Playgrounds to get this functionality to add merge to their specific objects. If these devs don't do this, the SDK will act as I noted here: #315 (comment)

A developer familiar with REST will expect the Swift SDK to PUT (not the quirky PUT) in which the SDK has always handled correctly in all previous versions of the Swift SDK.

[mtrezza]

but I do agree that it's making things a little complicated for some developers that may not be following the conversations we had.

I agree. Another major user group are developers who are just getting started with Parse Swift SDK,

• who are either entirely new to Parse, maybe coming from a 3rd party like Firebase (update data)
• or who are already using other Parse client SDKs and are expecting consistent behavior / commands among different Parse SDKs.

Maybe it's worth to compare the approach to how similar frameworks solve this in their Swift SDKs and aiming for a consistent simplicity for updating objects among Parse SDKs.

[cbaker6]

Maybe it's worth to compare the approach to how similar frameworks solve this in their Swift SDKs and aiming for a consistent simplicity for updating objects among Parse SDKs.

Can you enlighten us on what/who are these "similar frameworks" who have "Swift SDKs" and explain why you believe they are similar?

[dblythy]

@cbaker6 I regret to inform that I'm not currently available to provide any constructive feedback, however I am excited and grateful for your efforts to ensure consistency/clarity across various SDKs

[mtrezza]

Can you enlighten us on what/who are these "similar frameworks" who have "Swift SDKs" and explain why you believe they are similar?

Firebase or any other framework that can be considered an alternative to Parse. Simplicity and consistency within a framework are surely factors for developers when deciding for a framework.

[cbaker6]

Firebase or any other framework that can be considered an alternative to Parse.

Firebase is actually similar to the Parse Objective-C SDK. If you would like to "follow" Firebase's route, you should create PR's and provide suggestions to the Objective C SDK that add Swift files and extensions to an Objective C based project.

The Parse Swift SDK is not similar to Firebase by any means...

[mtrezza]

I am talking about SDK usage complexity and consistency with other Parse SDKs in the sense of language agnostic, developer friendly design choices. If something is perceived as complex in the Swift SDK by other users, we can look into how we could address these concerns in the Swift SDK, not in the ObjC SDK.

Moreover, I expect the ObjC SDK to be slowly phasing out and eventually be replaced by the Swift SDK. They are different languages, but we can assume that if we just look at Swift adoption rate in the Apple dev community and language popularity. So pointing to the ObjC SDK when someone questions design choices in the Swift SDK is not a sustainable argument.

[cbaker6]

If something is perceived as complex in the Swift SDK by other users, we can look into how we could address these concerns in the Swift SDK, not in the ObjC SDK.

Have you used the Swift SDK before? If so, can you explain what is complex in the usage of this SDK, this PR, and why you feel it's complex?

They are different languages, but we can assume that if we just look at Swift adoption rate in the Apple dev community and language popularity. So pointing to the ObjC SDK when someone questions design choices in the Swift SDK is not a sustainable argument.

"You" assuming, doesn't mean "we" assuming. I don't assume that and you definitely don't speak for me. In addition, comparing irrelevant frameworks to this one and not being able to support your arguments with factual content isn't an argument at all. For example, your comment "I am talking about SDK usage complexity and consistency with other Parse SDKs in the sense of language agnostic, developer friendly design choices". This SDK isn't just different by language, it's designed completely different and built from different principals. Your comments have consistently showed that much is missing from your understanding of how this SDK is actually designed along with differences between Swift and Objective-C. It’s cumbersome to keep answering your questions when it’s apparent you don’t use or understand how this SDK is designed and provide misinformation, but proceed as your info are facts. It feels like you are trolling Issues and PRs without taking the time to understand what you are asking which further wastes my time. I’ve exited previous threads on this repo with you because of this.

[cbaker6]

I regret to inform that I'm not currently available to provide any constructive feedback, however I am excited and grateful for your efforts to ensure consistency/clarity across various SDKs

@dblythy np, thanks for PR 263, the design an discussion on that thread influenced how I went about the originalData property in this PR, so I definitely appreciate your effort

[mtrezza]

Have you used the Swift SDK before? If so, can you explain what is complex in the usage of this SDK, this PR, and why you feel it's complex?

Sure, see #315 (comment) and #242.

[cbaker6]

Sure, see #315 (comment) and #242.

You never answered the questions:

1. Have you used the Swift SDK before?
2. If so, can you explain what is complex in the usage of this SDK, this PR

It seems, you "feel it's complex" because of someone else's comments. "Implementing" a method to do something more specific to a struct's needs is no more "complex" than overriding a class method. A developer familiar with classes and reference types should have no problem with this, particularly when they are looking for very "specific" functionality like only sending updated keys.

[Vortec4800]

Thanks for pinging me on this, sorry I couldn't spend some time on it sooner.

I think this is a really good solution for a really complex problem. While I do worry in general about adding complexity, this does seem like it's mostly optional - and adding an additional key isn't a huge lift. I also think the originalData property is well-named, however I have the insight from following along our conversations so we may want some portion of documentation that talks about the issue, why this is here, why you would want to use it, etc. I haven't had a chance to go through the Playground yet so perhaps it's in there already.

If there is one thing that worries me it's this:

Developers should NOT attempt to enforce properties be defined by defining their property as a non-optional type. Instead, a developer should do this by creating additional initializers (preferably in an extension) and carry out any necessary checks. Enforcing properties to be defined can also be done on the server side via schema setup, Parse Dashboard, or Cloud Code.

I do really like having the Parse object be the source of truth when it comes to vars that are required vs vars that aren't, and having that be enforced by the compiler. I understand the need, these "thin" objects will need to have empty values, but it's an unfortunate side effect. I don't have a solution here, without making a separate thin version with all optional properties, but that would be quite a bit of extra work to maintain. I guess in the end it's fine, and we'll just have to enforce it in other ways as you mentioned.

If the community consensus is still to have an additional protocol, I think we should at least spend some time on the naming. ParseRestorable seems to be a bad option. I think ParseMergeable or ParseReconcilable are a little better, but I don't think they capture the idea of optimizing and update.

I agree on this as well. While I understand why ParseRestorable makes sense, I also don't think it immediately conveys the reason to use it. Perhaps something like ParseOptimizedSendable or something in that world that encapsulates the functionality you would gain by implementing it. If I was a developer only skimming half the documentation (I'm sure that never happens...) and I saw something called ParseRestorable that wasn't required, I would assume it had something to do with state restoration or something and skip it entirely.

I really like that this is optional. The apps my team makes that use Parse are often quite small, and don't use that much traffic. I think for most of the apps, we'd actually skip this protocol and keep things simple given we don't need the speed improvement this would provide on data transmission. That said we do also have a couple very large projects that run on load balancers and clusters, and this would be a big improvement for those and worth the extra work and thought to implement.

[cbaker6]

@Vortec4800 thanks for your feedback!

Developers should NOT attempt to enforce properties be defined by defining their property as a non-optional type. Instead, a developer should do this by creating additional initializers (preferably in an extension) and carry out any necessary checks. Enforcing properties to be defined can also be done on the server side via schema setup, Parse Dashboard, or Cloud Code.

I do really like having the Parse object be the source of truth when it comes to vars that are required vs vars that aren't, and having that be enforced by the compiler. I understand the need, these "thin" objects will need to have empty values, but it's an unfortunate side effect. I don't have a solution here, without making a separate thin version with all optional properties, but that would be quite a bit of extra work to maintain. I guess in the end it's fine, and we'll just have to enforce it in other ways as you mentioned.

There are 3 important reasons for this that many might not realize when they chose to go the non-optional property route:

1. This isn't inline with the standard JSON Swift encoder/decoder and how it uses optionals. You and I discussed this in a previous thread Setting property to nil doesn't save to database #264 (comment)
2. How Parse pointers work and how Parse uses them. Parse Pointers ONLY have the following properties: className, objectId, and __type. Since the Swift SDK is heavily dependent on generics, a developer making a key required (non-optional) that isn't in the aforementioned list makes it impossible to create ParsePointer's from their ParseObject's. This is because they are saying their ParsePointer "requires" this property. See here for an example of this issue. Though someone may argue they don't need Pointer's in the current version of their app, they will most likely need them when their app grows
3. From a POP standpoint, if the ParseObject protocol doesn't require init(), developers can make non-optional values and it's impossible to initialize new versions of their ParseObject's internally in the SDK; preventing .mutable, merge, or anything else that needs a new copy inside the app. For value types with all optional properties, the compiler provides the init() automatically; assuming all other inits are defined in an extension. This is essential since this SDK doesn't use reference types and at times need to return a fresh copy of the value type to the developer.

Originally, It didn't cross my mind that developers would create ParseObject's with non-optional properties since every property given to them from ParseObject is optional. I mentioned on a previous thread that if I would have realized developers wouldn't follow the design of a ParseObject given to them I would have enforced an init() during the first release.

I do really like having the Parse object be the source of truth when it comes to vars that are required vs vars that aren't, and having that be enforced by the compiler.

The source of truth is the Cloud database in which you access through the Parse Server. the ParseObject on the client is a representation of that truth that can be out of date due to entities updating the Cloud database. Your Cloud Code, database indexes, etc. verify requirements for storing data. Even in a distributed database application like the one I mention here, the ParseObject on the client still isn't the source of truth, the local CoreData storage is the truth and ParseObject's are just snapshots.

I really like that this is optional. The apps my team makes that use Parse are often quite small, and don't use that much traffic. I think for most of the apps, we'd actually skip this protocol and keep things simple given we don't need the speed improvement this would provide on data transmission. That said we do also have a couple very large projects that run on load balancers and clusters, and this would be a big improvement for those and worth the extra work and thought to implement.

I feel this supports having it as part of ParseObject as it seamlessly will work if you need it or not. In addition, though not part of this PR, I imagine the setup can enable what you proposed in #268, though I haven't put much thought into how to implement it.

[cbaker6]

@pmmlo @vdkdamian @Vortec4800 For the ParseObject property, I'm thinking this is a better name:

• .mutable -> .mergable

Calling it will look like:

/*: Update your `ParseInstallation` `customKey` value. 
     Performs work on background queue and returns to designated on 
     designated callbackQueue. If no callbackQueue is specified it 
     returns to main queue. 
  */ 
 var installationToUpdate = Installation.current?.mergable // Name change here.  
 installationToUpdate?.customKey = "myCustomInstallationKey2" 
 installationToUpdate?.save { results in 
  
     switch results { 
     case .success(let updatedInstallation): 
         print("Successfully save myCustomInstallationKey to ParseServer: \(updatedInstallation)") 
     case .failure(let error): 
         print("Failed to update installation: \(error)") 
     } 
 } 

Thoughts?

[vdkdamian]

For the ParseObject property, I'm thinking this is a better name:

• .mutable -> .mergable

Yes, I definitely agree

[pmmlo]

Could we do mergeable?

@pmmlo @vdkdamian @Vortec4800 For the ParseObject property, I'm thinking this is a better name:

* `.mutable -> .mergable`

Calling it will look like:

/*: Update your `ParseInstallation` `customKey` value. 
     Performs work on background queue and returns to designated on 
     designated callbackQueue. If no callbackQueue is specified it 
     returns to main queue. 
  */ 
 var installationToUpdate = Installation.current?.mergable // Name change here.  
 installationToUpdate?.customKey = "myCustomInstallationKey2" 
 installationToUpdate?.save { results in 
  
     switch results { 
     case .success(let updatedInstallation): 
         print("Successfully save myCustomInstallationKey to ParseServer: \(updatedInstallation)") 
     case .failure(let error): 
         print("Failed to update installation: \(error)") 
     } 
 } 

Thoughts?

[vdkdamian]

Could we do mergeable?

Yes, this is the correct one, didn't notice it.

[cbaker6]

@pmmlo good catch!