RNET-222: V10 (#2011)

* Some preliminary cleanup * Remove some stuff from cmakelists * Get things building * wip - migrate to column keys from property indices * Fix remaining tests * fix rebase issues * Get all column keys in a single call * Remove dependencies * Reduce the number of objecthandle methods a little * Use PrimitiveValue for some object methods * Remove the manual tests * build on vs 2019 nodes * Add concrete methods for constructing a primitive value * Refactor RealmObject to reduce the number of methods (#2013) * Refactor RealmObject to reduce the number of methods * more fixes * some comments * Use PrimitiveValue for queries too * some after-rebase fixes * Update to latest core and sync (#2031) * Disable sync tests * Use the CPH macs for building v10 (#2034) * Use the CPH macs for building v10 * revert some restrictions * Add decimal support (#2014) * Add decimal support * Add querying support for decimal * Add changelog entry * After rebase fixes * Preserve the decimal128->decimal conversion operator * try removing the objectid remainder * Some *s and &s * More &s * Less &s * CR comments * Use the Bid128 directly in the union * Explicitly define low and high * return the array * Add support for embedded objects (#2017) * Remove ISchemaSource * Split the RealmObject hierarchy * Pass embedded to native, start work on the weaver for embedded objects * Wire more things * Add list methods + recursive tests * More tests * Get more dynamic APIs working * After rebase fixes * Backlinks + more dynamic API * oops * Read embeddedness from disk * add some preservation logic for xamarin.ios * Fix dynamic methods for statically typed platforms * Try to fix ios tests * Only allow x86 and x64 targets for simulator builds (#2037) * Only allow x86 and x64 targets for simulator builds * try with a different OS version * Try #3 * RNET-176: Add support for ObjectId (#2035) * Initial objectid support * Tests + support for ObjectId PKs * Fix some tests * Try to preserve the find method * Changelog * add support for lists of decimals/objectids (#2038) * V10 assorted improvements (#2039) * Replace I1 with U1 for bools * Revert to using property indices (cherry picked from commit f00e79a7ebed995bfeb7b64914a593d34eb7ef91) * Use latest OS * Update Core/Sync and revert to an older OS * checkout correct OS commit * Add MDB Realm support (#2041) * replace sync manager with app * Start working on MDB Realm - Expose App - Expose AppConfiguration - Move stuff from User to App - Expose the new credentials - Move Realm_OpenWithSync to SharedRealm - Replace Uri with partition in SyncConfig * wire up the network transport (without error handling) * Add AppException class Add App.EmailPasswordAuth Add User.Profile * Correct content encoding for the message body * Try to get ci running * Fix a compilation error * Fix compile #2 * Add docs for Credentials * Try using sh instead of readfile * another try * Try with a custom app * .. * . * Test fixes * Reenable session tests * Reenable more tests * Reenable more tests * Try to get some sleep * inline the sleep * Try to cat the app_id instead of executing it 🤦‍♂️ * 🤞 * Add port to baas url * Preserve some serializers * Add user.customdata * Fix tests * Add User.Provider * Use unencrypted metadata realm for tests * Lock the stringbuilder * More attempts to compact the Realm * some docs + warning fixes * Add more docs * Fix some warnings * Rework errors + add api key creation * Add api key tests * fix displayed sdk and platform versions * Add function support * Make the error code internal * more function tests * Add docs for functions * More tests * wip * Add docs for remote mongo client * Add logs link to app exceptions * Add v10 as package publishing branch * Build the package as 10.0.0 * Clean up some things * Clean up some json serialization code * Some cleanup * preserve object serializer * Try to fix iOS tests * Mount /tmp on docker builds for caching purposes * Use dots for alpha builds * Add linkCredentials (#2056) * Clean up tests and ensure we delete everything we create (#2058) * clean up tests and ensure we delete everything we create * Try to fix linux tests * Increase the dummy data size in release * Download realm binaries to a random path (#2061) * Add user identities (#2059) * Add push client API + rename some things (#2063) * Remove Realm Object Server references * Fix up the changelog * Implement MongoCollection APIs (#2068) * Implement MongoCollection APIs * Some test fixes * More tests * Final methods * Update OS * Fix tests with recursive embedded objects * don't link MongoDB.Bson * Update some error codes (#2070) * add jwt and user profile tests (#2071) * Add facebook test * Reorder the arguments in UpdateOne and UpdateMany * Add tests for user.customdata * Update to latest Core/Sync (#2073) * Update to latest Core/Sync * Clean up changelog + update OS * Make GetCustomData a method with generic arg * Expose the http status code of AppException * Fix compilation warnings * Fix ConfirmUser calling the incorrect method
realm · Oct 14, 2020 · 7b77c15 · 7b77c15
1 parent 8d38bc6
commit 7b77c15
Show file tree

Hide file tree

Showing 398 changed files with 15,831 additions and 19,291 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,117 @@
+v10 (TBD)
+------------------
+
+### Breaking Changes
+* We no longer support Realm Cloud (legacy), but instead the new [MongoDB Realm Cloud](https://realm.mongodb.com). MongoDB Realm is a serverless platform that enables developers to quickly build applications without having to set up server infrastructure. MongoDB Realm is built on top of MongoDB Atlas, automatically integrating the connection to your database. ([#2011](https://github.com/realm/realm-dotnet/pull/2011))
+* Remove support for Query-based sync, including the configuration parameters and the `SyncSubscription` types. ([#2011](https://github.com/realm/realm-dotnet/pull/2011))
+* Remove everything related to sync permissions, including both the path-based permission system and the object-level privileges for query-based sync. [Permissions in MongoDB Realm](https://docs.mongodb.com/realm/sync/permissions/) are defined serverside. ([#2011](https://github.com/realm/realm-dotnet/pull/2011))
+* Moved all API for dynamic access on the `Realm` class to `Realm.DynamicApi`:
+  * `Realm.CreateObject(string className, object primaryKey)` is now `Realm.DynamicApi.CreateObject(string className, object primaryKey)`.
+  * `Realm.All(string className)` is now `Realm.DynamicApi.All(string className)`.
+  * `Realm.RemoveAll(string className)` is now `Realm.DynamicApi.RemoveAll(string className)`.
+  * `Realm.Find(string className, long? primaryKey)` is now `Realm.DynamicApi.Find(string className, long? primaryKey)`.
+  * `Realm.Find(string className, string primaryKey)` is now `Realm.DynamicApi.Find(string className, string primaryKey)`.
+* It is now required that all top-level objects in a synchronized Realm have a primary key called `_id`. You can use the `MapTo("_id")` attribute to avoid using unidiomatic names for the model properties.
+* Bumped the minimum target for Xamarin.iOS apps to iOS 9.
+* Bumped the minimum API level for Xamarin.Android apps to 16 (Android 4.1).
+* Renamed `FullSyncConfiguration` to `SyncConfiguration`.
+
+### Enhancements
+* Added support for syncing to MongoDB instead of Realm Object Server. Applications must be created at [realm.mongodb.com](https://realm.mongodb.com).
+* Added an `App` class which is the entrypoint for synchronizing with a MongoDB Realm App.
+* Added `User.CustomData` containing an unstructured document with additional information about the user. Custom data is configured in your MongoDB Realm App.
+* Added `User.Functions`. This is the entry point for calling Remote MongoDB Realm functions. Functions allow you to define and execute server-side logic for your application. Functions are written in modern JavaScript (ES6+) and execute in a serverless manner. When you call a function, you can dynamically access components of the current application as well as information about the request to execute the function and the logged in user that sent the request.
+* Added `User.GetMongoClient` exposing an API for CRUD operations on a Remote MongoDB Service.
+* Added `User.GetPushClient` exposing an API for registering a device for push notifications.
+* Change `SyncConfiguration` to accept partition value instead of a server Uri. Partition values can currently be of types `string`, `long`, or `ObjectId`. Opening a realm by partition value is the equivalent of previously opening a realm by URL. In this case, partitions are meant to be more closely associated with your data. E.g., if you are a large retailer with multiple locations, the partition key can be the store Id and you each Realm will only contain data related to the specified store.
+* Add support for the Decimal128 data type. This is a 128-bit IEEE 754 decimal floating point number. Properties of this type can be declared either as `MongoDB.Bson.Decimal128` type or the built-in `decimal` type. Note that .NET's built-in decimal is 96-bit, so it cannot represent the full range of numbers, representable by `Decimal128`. (PR [#2014](https://github.com/realm/realm-dotnet/pull/2014))
+* Add support for the `ObjectId` data type. This is a 12 byte unique identifier that is common as a document id in MongoDB databases. It can be used a primary key. (PR [#2035](https://github.com/realm/realm-dotnet/pull/2035))
+* Add support for embedded objects. Embedded objects are objects which are owned by a single parent object, and are deleted when that parent object is deleted or their parent no longer references them. Embedded objects are declared by subclassing `EmbeddedObject` instead of `RealmObject`. Reassigning an embedded object is not allowed and neither is linking to it from multiple parents. Querying for embedded objects directly is also disallowed as they should be viewed as complex structures belonging to their parents as opposed to standalone objects. A trivial example is:
+
+  ```csharp
+  public class Address : EmbeddedObject
+  {
+      public string Street { get; set; }
+
+      public string City { get; set; }
+  }
+
+  public class Person : RealmObject
+  {
+      public string Name { get; set; }
+
+      // Address is an embedded object - you reference it as usual
+      public Address Address { get; set; }
+  }
+
+  public class Company : RealmObject
+  {
+      public string PhoneNumber { get; set; }
+
+      // Embedded objects can be contained in lists too
+      public IList<Address> OfficeAddresses { get; }
+  }
+  ```
+
+* Added new dynamic methods for instantiating embedded objects:
+  * `Realm.DynamicApi.CreateEmbeddedObjectForProperty` should be used to create an embedded object and assign it to a parent's property. For example:
+
+    ```csharp
+    // static API
+    var person = new Person();
+    person.Address = new Address
+    {
+        City = "New York"
+    };
+
+    // dynamic API
+    var dynamicPerson = realm.DynamicApi.CreateObject("Person");
+    var address = realm.DynamicApi.CreateEmbeddedObjectForProperty(dynamicPerson, "Address")
+    address.City = "New York";
+    ```
+
+  * `Realm.DynamicApi.AddEmbeddedObjectToList` should be used to create an embedded object and add it to a parent's list property.
+  * `Realm.DynamicApi.InsertEmbeddedObjectInList` should be used to create an embedded object and insert it in a parent's list property at a specified index.
+  * `Realm.DynamicApi.SetEmbeddedObjectInList` should be used to create an embedded object and set it at an index in a parent's list property.
+
+    ```csharp
+    // static API
+    var company = new Company();
+    company.OfficeAddresses.Add(new Address
+    {
+        City = "New York"
+    });
+
+    company.OfficeAddresses.Insert(0, new Address
+    {
+        City = "Palo Alto"
+    });
+
+    company.OfficeAddresses[1] = new Address
+    {
+        City = "New Jersey"
+    };
+
+    // dynamic API
+    var dynamicCompany = realm.DynamicApi.CreateObject("Company");
+    var officeToAdd = realm.DynamicApi.AddEmbeddedObjectToList(dynamicCompany.OfficeAddresses);
+    officeToAdd.City = "New York";
+
+    var officeToInsert = realm.DynamicApi.InsertEmbeddedObjectInList(dynamicCompany.OfficeAddresses, 0);
+    officeToInsert.City = "Palo Alto";
+
+    var officeToSet = realm.DynamicApi.SetEmbeddedObjectInList(dynamicCompany.OfficeAddresses, 1);
+    officeToSet.City = "New Jersey";
+    ```
+
+* The memory mapping scheme for Realm files has changed to better support opening very large files.
+
+### Compatibility
+* Realm Studio: 10.0.0 or later.
+
+### Internal
+* Using Sync 10.0.0 and Core 10.0.0.
+
 ## 5.1.2 (TBD)
 ------------------
 

diff --git a/Jenkinsfile b/Jenkinsfile
@@ -23,11 +23,12 @@ stage('Checkout') {
       userRemoteConfigs: scm.userRemoteConfigs
     ])
 
-    if (env.BRANCH_NAME == 'master') {
-      versionSuffix = "alpha-${env.BUILD_ID}"
+    // V10TODO: temporary set v10 as publishing branch
+    if (shouldPublishPackage()) {
+      versionSuffix = "alpha.${env.BUILD_ID}"
     }
     else if (env.CHANGE_BRANCH == null || !env.CHANGE_BRANCH.startsWith('release')) {
-      versionSuffix = "PR-${env.CHANGE_ID}-${env.BUILD_ID}"
+      versionSuffix = "PR-${env.CHANGE_ID}.${env.BUILD_ID}"
     }
 
     stash includes: '**', excludes: 'wrappers/**', name: 'dotnet-source', useDefaultExcludes: false
@@ -38,7 +39,7 @@ stage('Checkout') {
 stage('Build wrappers') {
   def jobs = [
     'iOS': {
-      rlmNode('osx') {
+      rlmNode('osx || macos-catalina') {
         unstash 'dotnet-wrappers-source'
         dir('wrappers') {
           sh "./build-ios.sh --configuration=${configuration}"
@@ -47,7 +48,7 @@ stage('Build wrappers') {
       }
     },
     'macOS': {
-      rlmNode('osx') {
+      rlmNode('osx || macos-catalina') {
         unstash 'dotnet-wrappers-source'
         dir('wrappers') {
           sh "REALM_CMAKE_CONFIGURATION=${configuration} ./build.sh -GXcode"
@@ -59,9 +60,7 @@ stage('Build wrappers') {
       rlmNode('docker') {
         unstash 'dotnet-wrappers-source'
         dir('wrappers') {
-          buildDockerEnv("ci/realm-dotnet:wrappers", extra_args: "-f centos.Dockerfile").inside() {
-            sh "REALM_CMAKE_CONFIGURATION=${configuration} ./build.sh"
-          }
+          buildWrappersInDocker('wrappers', 'centos.Dockerfile', "REALM_CMAKE_CONFIGURATION=${configuration} ./build.sh")
         }
         stash includes: 'wrappers/build/**', name: 'linux-wrappers'
       }
@@ -74,9 +73,7 @@ stage('Build wrappers') {
       rlmNode('docker') {
         unstash 'dotnet-wrappers-source'
         dir('wrappers') {
-          buildDockerEnv("ci/realm-dotnet:wrappers_android", extra_args: '-f android.Dockerfile').inside() {
-            sh "./build-android.sh --configuration=${configuration} --ARCH=${localAbi}"
-          }
+          buildWrappersInDocker('wrappers_android', 'android.Dockerfile', "./build-android.sh --configuration=${configuration} --ARCH=${localAbi}")
         }
         stash includes: 'wrappers/build/**', name: "android-wrappers-${localAbi}"
       }
@@ -86,13 +83,13 @@ stage('Build wrappers') {
   for(platform in WindowsPlatforms) {
     def localPlatform = platform
     jobs["Windows ${localPlatform}"] = {
-      rlmNode('windows-vs2017') {
+      rlmNode('windows') {
         unstash 'dotnet-wrappers-source'
         dir('wrappers') {
           powershell ".\\build.ps1 Windows -Configuration ${configuration} -Platforms ${localPlatform}"
         }
         stash includes: 'wrappers/build/**', name: "windows-wrappers-${localPlatform}"
-        if (env.BRANCH_NAME == 'master') {
+        if (shouldPublishPackage()) {
           archiveArtifacts 'wrappers/build/**/*.pdb'
         }
       }
@@ -102,13 +99,13 @@ stage('Build wrappers') {
   for(platform in WindowsUniversalPlatforms) {
     def localPlatform = platform
     jobs["WindowsUniversal ${localPlatform}"] = {
-      rlmNode('windows-vs2017') {
+      rlmNode('windows') {
         unstash 'dotnet-wrappers-source'
         dir('wrappers') {
           powershell ".\\build.ps1 WindowsStore -Configuration ${configuration} -Platforms ${localPlatform}"
         }
         stash includes: 'wrappers/build/**', name: "windowsuniversal-wrappers-${localPlatform}"
-        if (env.BRANCH_NAME == 'master') {
+        if (shouldPublishPackage()) {
           archiveArtifacts 'wrappers/build/**/*.pdb'
         }
       }
@@ -164,7 +161,7 @@ stage('Package') {
         packageVersion = getVersion(packages[0].name);
         echo "Inferred version is ${packageVersion}"
 
-        if (env.BRANCH_NAME == 'master') {
+        if (shouldPublishPackage()) {
           withCredentials([usernamePassword(credentialsId: 'github-packages-token', usernameVariable: 'GITHUB_USERNAME', passwordVariable: 'GITHUB_PASSWORD')]) {
             echo "Publishing Realm.Fody.${packageVersion} to github packages"
             bat "dotnet nuget add source https://nuget.pkg.github.com/realm/index.json -n github -u ${env.GITHUB_USERNAME} -p ${env.GITHUB_PASSWORD} & exit 0"
@@ -329,13 +326,18 @@ def NetCoreTest(String nodeName) {
         dotnet build -c ${configuration} -f netcoreapp20 -p:RestoreConfigFile=${env.WORKSPACE}/Tests/Test.NuGet.Config -p:UseRealmNupkgsWithVersion=${packageVersion}
         dotnet run -c ${configuration} -f netcoreapp20 --no-build -- --labels=After --result=${env.WORKSPACE}/TestResults.NetCore.xml
       """.trim()
+
+      String appLocation = "${env.WORKSPACE}/Tests/TestApps/dotnet-integration-tests"
+
       if (isUnix()) {
         if (nodeName == 'docker') {
           def test_runner_image = docker.image('mcr.microsoft.com/dotnet/core/sdk:2.1')
           test_runner_image.pull()
-          withRos('3.23.1') { ros ->
-            test_runner_image.inside("--link ${ros.id}:ros") {
-              script += ' --ros $ROS_PORT_9080_TCP_ADDR --rosport $ROS_PORT_9080_TCP_PORT'
+          withRealmCloud(version: '2020-10-12', appsToImport: ["dotnet-integration-tests": appLocation]) { networkName ->
+            test_runner_image.inside("--network=${networkName}") {
+              def appId = sh script: "cat ${appLocation}/app_id", returnStdout: true
+
+              script += " --baasurl http://mongodb-realm:9090 --baasappid ${appId.trim()}"
               // see https://stackoverflow.com/a/53782505
               sh """
                 export HOME=/tmp
@@ -401,6 +403,19 @@ def reportTests(spec) {
   )
 }
 
+def buildWrappersInDocker(String label, String image, String invocation) {
+  String uid = sh(script: 'id -u', returnStdout: true).trim()
+  String gid = sh(script: 'id -g', returnStdout: true).trim()
+
+  buildDockerEnv("ci/realm-dotnet:${label}", extra_args: "-f ${image}").inside("--mount 'type=bind,src=/tmp,dst=/tmp' -u ${uid}:${gid}") {
+    sh invocation
+  }
+}
+
+boolean shouldPublishPackage() {
+  return env.BRANCH_NAME == 'master' || (env.CHANGE_BRANCH != null && env.CHANGE_BRANCH == 'v10')
+}
+
 // Required due to JENKINS-27421
 @NonCPS
 List<List<?>> mapToList(Map map) {