Sorting stage 2 #6669

ironage · 2023-05-24T20:41:43Z

Strings and Binaries are no longer considered equal even if they have the same payload.
This is breaking for user queries and also for the file format of Set<Mixed> if that set contains strings and binaries. The tests covering the format migration will come later since the next stage in this series will contain another breaking change for sets.

Based on #6668.
Part 2 of breaking up #6406
Part of fixing #6118

☑️ ToDos

📝 Changelog update
🚦 Tests (or not relevant)
C-API, if public C++ API changed.

Partially based on 5f2dda1 Delete some obsolete cruft set_string_compare_method() and everyhing related to it has never actually been used by any SDK, and is not really the correct solution to the problem anyway.

tgoyne · 2023-05-24T21:08:17Z

This seems complicated to actually release. The sort order change is probably fine, but the query change is something that'll need a major version bump on every SDK. If it could be made opt-in somehow then we could avoid needing to do cross-team coordination of major version releases that'd probably result in things getting dragged out for a few extra months.

ironage · 2023-05-24T22:41:48Z

Although it is breaking, I'm hoping we can call it a fix. I don't think we have documented this behaviour anywhere and I'd expect users to be somewhat surprised that searching for x == 'foo' may match their binary blobs (assuming x is a mixed property).

If users are coordinating a multi-app SDK and one SDK upgrades before the other, I hope they can use the type queries as a placeholder to opt-in to the new behaviour until all SDKs pulls in this change Eg. x.@type == 'string' && x == 'foo'. Though I'm not sure cocoa supports that yet.

For FLX queries, the server already does strict type checking for strings (in their parser implementation they had assumed that a quoted constant always meant a string). Fortunately this means that we will not be breaking anyone's FLX queries. The related work to support the new bin(..) syntax is covered by https://jira.mongodb.org/browse/BAAS-20265. Docs changes are tracked by https://jira.mongodb.org/browse/DOCSP-30108

bmunkholm · 2023-05-25T09:24:02Z

Although it is breaking, I'm hoping we can call it a fix.
Shouldn't you then put this under "Fixes" instead in the changelog and rephrase that a bit ;-)

ironage · 2023-05-25T22:11:26Z

Shouldn't you then put this under "Fixes" instead in the changelog and rephrase that a bit ;-)

I forgot, but this actually is a breaking change for queries in RQL that match a constant to a binary property. So I'll keep it in breaking changes and let each SDK decide how to label it for their users.

bmunkholm · 2023-05-26T06:45:02Z

What other alternatives are there to introducing a breaking change that a) requires all SDKs to need to make a major breaking change; b) that change will go undetected by recompiling your app.

ironage · 2023-05-29T17:23:41Z

Here's a summary of the extent of these changes.

Sorting on a Mixed property that has certain combinations of binary and string data can produce wrong or inconsistent ordering. This PR fixes that problem by creating a total ordering.
Mixed property comparison (==, ==[c], contains, beginswith, endswith) for string literals (ex: "foo") will no longer match binaries of the same value (affects: Mixed, List<Mixed>, Set<Mixed>, Dictionary<Mixed>)
- The server side of these FLX queries already had this strongly typed behaviour, so this is not breaking for FLX only for local queries.
RQL queries have a syntax change for queries on binary constants: binary == "foo" or binary == $0 with a String argument will now throw an exception. They should now be written as binary == bin("foo")
- Currently, the server side FLX parser assumes that double or single quote constants are strings and B64"...==" means a binary which is incorrect behaviour (tracked here). Local RQL prints a base 64 value for any constant string that contains non-printable characters.
- We could potentially support the previous syntax in RQL and if the comparison column is a binary, convert the constant from a string to a binary. But I'd rather make a break at this time; binaries are not a common property to query, and there is no documentation on how to query them (and nobody has ever asked that I can recall).
The upcoming sorting change should likely be documented as a breaking change. Eg. "aAbBzZ" to "ABZabz".

jedelbo · 2023-05-30T13:16:32Z

src/realm/mixed.hpp

@@ -694,11 +694,8 @@ inline BinaryData Mixed::get<BinaryData>() const noexcept
 {
    if (is_null())


Perhaps this is wrong. Generally null means that the type is undefined.

Good eye. For some reason we do this for strings and binaries only. We could change it but then it is the responsibility of the caller to always check for nulls. I guess we do this because both StringData and BinaryData store null internally as opposed to our other types that are used with std::optional and this design conflicts with the Mixed type which is supposed to store Null or a StringData which itself can be null. Perhaps we can solve this in a different PR.

jedelbo · 2023-05-30T13:22:13Z

src/realm/parser/driver.cpp

@@ -1024,6 +1024,21 @@ std::unique_ptr<Subexpr> AggrNode::aggregate(Subexpr* subexpr)
    return agg;
 }

+void ConstantNode::decode_b64(util::UniqueFunction<void(StringData)> callback)


Should be a util::FunctionRef

jedelbo · 2023-05-30T13:30:06Z

src/realm/parser/query_flex.ll

@@ -59,6 +59,7 @@ blank   [ \t\r]
 (?i:sort)                   return yy::parser::make_SORT(yytext);
 (?i:distinct)               return yy::parser::make_DISTINCT(yytext);
 (?i:limit)                  return yy::parser::make_LIMIT(yytext);
+(((?i:bin)|(?i:binary))"(") return yy::parser::make_BINARY(yytext);


Parenthesis should not be part of the keyword. Should be part of the syntax

I originally had it without the parenthesis here, but then I found that we would not support columns with the name "binary" which we happened to have tests for already. If I change it to the expected way then I'd get this exception Invalid predicate: 'binary == $6': syntax error, unexpected ==, expecting '('

I took the liberty to add a commit that fixes this.

jedelbo · 2023-05-30T13:48:13Z

src/realm/parser/driver.hpp

@@ -157,6 +157,8 @@ class ConstantNode : public ValueNode {
        FLOAT,
        STRING,
        BASE64,


Perhaps this should be renamed to STRING_BASE64?

It is also valid to use BASE64 inside a binary constant, eg: binary_col == binary(B64\"AA==\"). Would calling it STRING_BASE64 still be appropriate?

But that would be a BINARY_BASE64 if I understand the code correctly

Ah yes I see now, you are correct.

jedelbo · 2023-05-30T13:57:51Z

src/realm/query_conditions.hpp

@@ -73,8 +73,8 @@ struct Contains : public HackClass {
        if (m1.is_null())
            return !m2.is_null();
        if (m1.is_type(type_String, type_Binary) && Mixed::types_are_comparable(m1, m2)) {
-            BinaryData b1 = m1.get_binary();
-            BinaryData b2 = m2.get_binary();
+            BinaryData b1 = m1.export_to_type<BinaryData>();


But Strings and Binary are no longer comparable...
This has consequences in many places.

I've updated these query conditions to do more explicit type checking instead of relying on Mixed::types_are_compatible(). That should speed things up a bit.

tgoyne · 2023-05-30T16:33:40Z

I think there are relatively straightforward scenarios where people could be incidentally querying on binary data as if it were strings. Foundation has a lot of APIs which return a Data which has to be decoded into a string with a specific encoding (e.g. reading from a file or the network, or encoding something to json), and right now you can just dump those into a Mixed property and decode to string on read and things will work fine as long as the strings happen to be utf-8. It's not a good idea, but I don't think we can be at all confident that no one is relying on this.

ironage · 2023-05-30T22:17:51Z

Here are a few options:

Document the breaking change well and bump the major version of the SDK. Users who were relying on this would have to rewrite their queries from mixed == "foo" to mixed == "foo" || mixed == bin("foo") (or the swift equivalent).
The Swift SDK can intercept queries of this form and rewrite the user's query before it hits core so that core can make this breaking change but it doesn't affect swift users. Is this a viable option for the Swift SDK to adopt the break on their own timeline?
Core can create a new query operator (perhaps following in javascript tradition ===) with these strict type checking changes. It would follow that numerics would also get type checking with this operator as well. However, I don't think MongoDB will support this for FLX queries so we would serialize queries of this type to the long form mentioned in point 1.

I can see that we may have users affected by this change, but I feel like it would be better to make a clean break now rather than extending support for the old behaviour especially if it is possible for users to rewrite their queries as a workaround. So I'd vote for option 1 and hope that users can adapt. Especially because Mixed is a relatively newer type (not sure what adoption looks like though). But I may be out of touch with our community and how adverse they are to breaking changes like this one.

src/realm/query_conditions.hpp

src/realm/query_engine.cpp

src/realm/mixed.cpp

nicola-cab

LGTM

jedelbo

LGTM

ironage · 2023-06-12T23:09:28Z

@tgoyne is the severity of the breaking change proposed here untenable to the point that we should not merge this change? Would any of the upgrade options I outlined work, or are there other ways we can make this easier to digest?

tgoyne · 2023-06-14T18:57:22Z

Mixed is over two years old, so just hoping that no one is using it seems like a stretch.

The server needs to continue to support existing queries regardless of what we do here, as otherwise existing clients will be broken. Unless there's some sort of versioning scheme involved, that seems like it means that the new behavior needs a different syntax? I guess we can also just check if there are any FLX queries in existence which would actually be effected by this. Even if there currently aren't any there might be some in the future, but it'd be an annoyance that developers hit while writing the app rather than introducing a bug to existing published apps.

The obvious transformation to implement the old behavior in the query builder would double the runtime of effected queries. I think maybe we'd instead copy and paste the relevant query nodes, modify them to do the desired thing, and only do the transform when serializing?

ironage · 2023-06-15T18:58:28Z

The server needs to continue to support existing queries

The current server behaviour is already strongly typed for mixed queries, so I view this change as a fix which brings the client inline to the server's behaviour. Eg. currently a FLX query of the form mixed == "foo" will only match on strings according to the server. So for this type of string query, we are guaranteed that this change won't break any existing FLX apps.

Where this gets a bit more complicated is that the server also currently has a bug in their implementation where they have been interpreting base64 constants as meaning to match only with binaries. (a query serializes a constant string to base64 if any of the characters are unprintable) Prior to this PR, queries of the form mixed == B64'...' would match on binaries or strings on the client, but would match only on binaries on the server. So the introduction of the binary("...") syntax I also view as a bug fix for FLX.

The breaking change would only be noticeable for local queries, and only for that weird case where people are trying to do string matching on binary data in a mixed property.

ironage · 2023-07-25T16:28:07Z

In a discussion with @tgoyne and @jbreams we concluded that core will go ahead and merge this as a "breaking" change.
When it comes time for the Swift SDK to adopt this, that team can choose to override queries of the type mixed == 'foo' and transform them into the core equivalent of mixed == 'foo' || mixed == binary('foo') in order to support the "json into mixed" use-case. Or they may decide not to do that and call it a fix. In an ideal world, this change will land in the timeline of the Swift SDK's next major version and be covered by a breaking change label just in case someone is affected.

@keys

* Prepare next-major * Remove support for upgrading from pre core-6 (v10) (#6090) * Optimize size of ArrayDecimal128 (#6111) Optimize storage of Decimal128 properties so that the individual values will take up 0 bits (if all nulls or all zero), 32 bits, 64 bits or 128 bits depending on what is needed. * update next major to core 13.4.1 (#6310) * temporary disable failing c api decimal test * Revert "update next major to core 13.4.1" (#6312) * Revert "update next major to core 13.4.1 (#6310)" This reverts commit 59764a2. * appease format checks * Align dictionaries to Lists and Sets when they are cleared. (#6254) * Fix storage of Decimal128 NaNs * Allow Collections to be owned by Collections (#6447) Introduce a new class - ColletionParent - which a collection will refer to as its owner. This class can be specialized as an Obj if the nesting level is 0 or a CollectionList if the collection is nested. * Add interface for defining columns of nested collections * Add CollectionList class * Change CollectionBase::set_owner() interface Make it clear that when an Obj is the owner, then the index must be a ColKey * Implementation `CollectionList::remove()` (#6381) (#6458) Co-authored-by: Jørgen Edelbo <jorgen.edelbo@mongodb.com> * Schema support for nesting collection (#6451) Co-authored-by: Jørgen Edelbo <jorgen.edelbo@mongodb.com> * Handle links in nested collections (#6470) * Handle nullifying links in nested collections * Clear backlinks related to nested collections * Return collection type in Mixed (#6520) * Print nested collections to Json (#6534) * dump to json support info about nested collections for schema * reuse logic for printing nested collections * main logic for expanding nested collections to json, requires to be polished * more testing for nested containers * complete algo for printing nested collections in json format * add testing json files to project * generate json files option set to false * run whole test suite * test nested collections with links * format checks * Move out_mixed_json... functionality to Mixed class * Remove not needed template parameter from CollectionBaseImpl * Delegate to_json to collections * remove commented code * fix audit conflicts --------- Co-authored-by: Jørgen Edelbo <jorgen.edelbo@mongodb.com> * Simplify Obj::get_path() * Store ref in ArrayMixed (#6565) * Cleanup naming and consolidate update strategy * Allow a Mixed containing a ref to be stores in ArrayMixed * Actually store collection type in Mixed (#6583) * Allow Dictionary to contain a collection (#6584) * Make a template specializetion for Lst<Mixed> * Allow Lst<Mixed> to contain a collections * Streamlining interface (#6615) Main change is that insert... will not return the created collection. This has of course big influence on the test cases written for the old API. Virtual interface for setting/getting nested collections created * Api nested collections in OS (#6618) Add interface on both object_store::Collection and the C API to handle collections in Mixed. --------- Co-authored-by: Jørgen Edelbo <jorgen.edelbo@mongodb.com> * Set interface nested collections (#6648) * testing for set<mixed> * c-api for nested sets * fix Set constructor * Get path from collection objects (#6636) * Move NoOpTransactionLogParser to transact_log.hpp * Add nested collection path in transaction log * Optimize get_path() Avoid having the first element in Path being a std::string * Small fixes * Make m_path private in sync::instr::Path * Remove `set_string_compare_method` (#6668) Partially based on 5f2dda1 Delete some obsolete cruft set_string_compare_method() and everyhing related to it has never actually been used by any SDK, and is not really the correct solution to the problem anyway. Co-authored-by: Thomas Goyne <tg@realm.io> * Replication of operations on nested collections * Remove support for query over typed links in Dictionary This feature is not exposed, and should not be done in the way it was implemented. * Use BPlusTree to hold backlinks (#6673) Removes the limit on how many backlinks we can handle * Add StablePath concept * Collection in mixed notification support. (#6660) * Add support in the notification machinery for nested collections. * Avoid passing string parameters by value in KeyPathMapping interface * Use uniform Path representation in query parser We need to be able to handle a path that is just a sequence of strings and integers. The strings can then either be a property name or a key in a dictionary. Before we have known that the last entry in a path would be a property name. We can't assume that anymore, so we just have to follow links as long as that is possible. The rest must then be a path to the wanted value. We must also allow the syntax "dict.key" and dict["key"] to be used interchangeably. A nested dictionary can be used in the same way as an embedded object is used and so the syntax for querying on a specific property should be the same. * Support query on nested collections This includes supporting using index in query on list of primitives * Copy replication nested collections (#6714) * copy replication for nested collections * Remove support for TypedLinks in LinkTranslator Removes some complexity/code. Is easy to re-introduce. Can be safely removed if we disallow creating columns of this type. This can also safely be done, as this feature is not yet used. * Support typed links in nested collections This is about the usual stuff: - When a link is inserted, make sure a backlink is created - When a link is cleared, make sure the backlink is removed. - When object containing a collection containing links is deleted make sure the backlinks are removed. - When the linked-to object is deleted the link should be nullified/removed. - When the linked-to object is made into a tombstone, the link should be updated. - When the linked-to object is recreated, the link should be restored. * Handle exceptions thrown from Obj::get_collection_ref * Support assigning a json string to a mixed property Collect all to_json related functions in one compilation unit. Then it will only be included in the final binary if used. * Support having [*] as part of a path (#6741) Allows you to consider all elements at some level. * added check for set in mixed in the C API (#6764) * Fix collection mismatch for `Set` in `Mixed`. (#6802) * Allow TypedLinks to be part of path to property in queries * Sorting stage 2 (#6669) * Remove `set_string_compare_method` Partially based on 5f2dda1 Delete some obsolete cruft set_string_compare_method() and everyhing related to it has never actually been used by any SDK, and is not really the correct solution to the problem anyway. * strings are no longer equal to binaries * parser supports bin(...) to differentiate binaries * fix sectioned results * fix formatting * review updates * Fix syntax * review feedback changes * fix reported UB * lint --------- Co-authored-by: Thomas Goyne <tg@realm.io> Co-authored-by: Jørgen Edelbo <jorgen.edelbo@mongodb.com> * Fix list type * Syntactical sugar * Throw if syncing a collection nested in Mixed * Support indexing into link collections in Query (#6854) * Support syncing nested Set * Add missing support for getting Sets * Return correct attachement state from nested collections (#6880) Ensures that proper notifications on no longer existing collections are sent out * small changes to the c api for collections in mixed (#6881) * explicit insertion for collections in mixed and return the collection just inserted * Make exceptions thrown by nested collections more consistent (#6875) * Make information on the deletion of a collection available in C API (#6896) * update set_collection for list and have an explicit function for each collection (#6900) * Small changes * Publish Obj::set_json in C API * Check for stale accessors to a collction embedded directly in a Mixed property Change the index held by the collection object from ColKey to a structure (ColIndex) )containing both the index of the column and a key generated for that particular collection. The key value is stored alongside the ref and compared with the key value found in index when trying to obtain the ref for the collection. This commit includes review updates * Fix freezing a nested collection * Remove support for static nested collections * Use more bits in ColIndex key * Improve StringIndex::dump_node_structure * Check for stale accessors to a collction embedded in a dictionary Change the index held by the collection object from std::string to a structure (KeyIndex) ) containing both the beginning of the dictionary key (mostly for debugging purposes) and an index key generated for that particular collection. The key value is stored alongside the ref and compared with the key value found in index when trying to obtain the ref for the collection. * Optimize StableIndex * Refactor StringIndex interface (#6787) * refactor StringIndex interface optimize * StringIndex has a virtual parent SearchIndex * review feedback and fix a warning * More consistent exception handling for nested collections This commit fixes the problem that trying to access position 0 in a newly created nested list would give an exception saying that the collection was gone instead of an out-of-bounds exception. This was because we had a test for attached before validating the index. The solution selected is to remove "ensure_attached" and let the exceptions thrown in "get_collection_ref" flow all the way to the client. This is kind of fundamental change in that we must remove the noexcept specification from "update_if_needed_with_status" and make the "init_from_parent" functions rethrow the exceptions caught. The noexcept functions calling "update_if_..." must add a try..catch block. * Add ability to get collections from Results (#6948) Co-authored-by: Nicola Cabiddu <nicola.cabiddu@mongodb.com> * Fix compilation of RealmTrawler * Logging mutations on tables (#6953) To avoid having the same operation logged twice, the logging in instruction_applier in removed. * Simplify Logger class a bit Logger::m_base_logger_ptr seems not to be used in the class itself. The member is added to the sub-classes that need it. get/set level_threshold need not be virtual is we remove support for NullLogger. * Limiting the output when logging large string and binary values (#6986) * Introduce logging categories * Sorting stage 3 (#6670) * Add tests on BPlusTree upgrade * change the sort order of strings * Add test on upgrade of StringIndex and Set * remove utf8_compare * Add upgrade functionality * Avoid string index upgrade * Update test * Move Set::do_resort() to .cpp file * Generate test_upgrade_database_x_23.realm as on ARM * Revert "Avoid string index upgrade" This reverts commit 333982a. * Fix upgrade logic for string index - Only upgrade if char is signed - Upgrade Mixed columns too * memcmp is faster than std::lexi_cmp and doesn't require casting * optimize: only compare strings once * Upgrade of fulltext index not needed * migrate sets of binaries and better migration test * generate migration Realms on a signed platform * fix lint * avoid a string index migration by using linear search --------- Co-authored-by: Jørgen Edelbo <jorgen.edelbo@mongodb.com> * Upodate Package.swift * Client Reset for collections in mixed / nested collections (#6766) * Fix error after merge * Fix issue using REALM_ENABLE_MEMDEBUG=On * Logging of schema migrations * Use logging categories (#7052) * Logging notification activity * Logging details when opening DB * Fix warning * Update bindgen to support logging categories * Add cases handling Json::value_t::binary * Log free space and history sizes when opening file * Remove unused stuff * Rearrange some code in Set<T> The is to prepare for merge with master. It is more or less a cherry-pick of commit bf5ffd3. * Fix missing NullLogger * Remove support for nested sets * Fix warnings * Remove type_LinkList and col_type_LinkList (#7114) Should have been removed long time ago * Index on list of strings (#7152) * Prepare beta release * Add path.hpp to installed headers * Update release notes * Allow keypath to be provided as argument (#7210) * Remove set from realm_value_type for collections in mixed (#7245) * Add support for collections in indexed mixed fields * [C-API] Fix the return type of realm_set_collection (#7247) * Fix the return type of realm_set_collection * fix some leaking tests * Simplify JSON functionality * Don't leak implementation of BsonDocument and BsonArray to the users. This is done by defining the interface explicitly and in a way that makes it possible to easily change the underlying implementation. * Throw when inserting an embedded object into a list of Mixed * Fix queries on dictionaries in Mixed with @keys * Optimize BsonDocument::find() * Only output '_key': xxx when output mode is plain JSON Fix 9169fa1 * Send notifitations about mutations on nested collections * Restore correct expected json files * Support querying for @SiZe on Mixed This will make sense both for strings, binaries and nested collections. * Support querying with @type on nested collections (#7288) * Refactor ConstantNode::visit() (#7295) This will allow us to use argument substitution in more places. In particuler if TypeOfValue is expected. The visit function in ConstantNode is split up i 2 steps. First the value is extracted into a Mixed - that being directly from the query string or from the arguments. Then the vaule is adapted to what is needed based on the 'hint' parameter. * Fix merge error in dependency list * Fix using stringops query on nested collections * Fix using ANY, NONE, ALL in query on Mixed property * Remove LinkList (#7308) * fix == NONE {x} queries (#7333) * fix == NONE {x} queries * more tests * Fix app URI tests for baasaas (#7342) * Mitigate races in accessing `m_initated` and `m_finalized` in various REALM_ASSERTs (#7338) * Fix a TOCTOU race when copying Realm files Checking if the destination exists before copying is a race condition as the file can be created in between the check and the copy. Instead we should attempt to copy without overwriting the target if it exists. * Use clonefile() when possible in File::copy() * Delete unused sync file action metadata fields * Schema migration tests to use admin API rather than querying backing cluster (#7345) * Add bson library (#7324) Can be used as a reference implementation in tests. * Fix Results nofitifation for changes to nested collections * Use TestDirGuard where applicable * Simplify session tests by consitently using TestSyncManager::fake_user() * Separate TestSyncManager and OfflineAppSession Co-authored-by: James Stone <james.stone@mongodb.com> * Fix sync replication (#7343) * Adjust CMake files to used by vcpkg (#7334) * Fix SPM compilation errors (#7360) Include paths for the tests are set up slightly different for the SPM build from the CMake build. * Prepare release * Update release notes * Prepare release * Update release note * Rewrite the "app: app destroyed during token refresh" test (#7363) * Update to CHANGELOG * Don't allow Core targets to be installed if submodule (#7379) Co-authored-by: Kenneth Geisshirt <kenneth.geisshirt@mongodb.com> Co-authored-by: Jørgen Edelbo <jorgen.edelbo@mongodb.com> * Prepeare release * Update release notes * Do not populate list KVO information for non-list collections (#7378) * Prevent opening files with file format 23 in read-only mode * Don't update backlinks in Mixed self-assignment (#7384) Setting a Mixed field to ObjLink equal to the current value removed the existing backlink and then exited before adding the new one, leaving things in an invalid state. * Eliminate copies when accessing values from Bson types (#7377) Returning things by value performs a deep copy, which is very expensive when those things are also bson containers. Re-align the naming with the convention names for the functions rather than being weird and different. * Use the correct allocator for queries on dictionaries over links (#7382) The base table's allocator was being used to read from the target table. * Bson object should hold binary data in decoded form If you construct a Bson object from a std::vector<char>, the extjson streaming format should encode the binary data. * Fix passing a double as argument to query on Decimal128 (#7387) * Treat missing keys in dictionaries as null in queries (#7391) * Treat missing keys in dictionaries as null in queries * Fix test --------- Co-authored-by: Jørgen Edelbo <jorgen.edelbo@mongodb.com> * Allow using aggregate operations on Mixed properties in queries (#7398) This is something which Cocoa and the query engine supports but the core query parser did not. * [bindgen] Enable support for collections in the `Mixed` data type (#7392) * Adapt to breaking change. * Expose APIs for flat collections in Mixed and add preparation for nested. * Add and expose helper for getting the data type. * Expose a data type enum that includes non-primitives. The JS SDK needs this for checking types of Mixed. * Update casting of enum constants. * Replace access of 'm_type' with use of the 'int()' operator overload. * Expose APIs for setting nested lists in Mixed. * Expose APIs for setting nested dictionaries in Mixed. * Expose APIs for getting nested lists in Mixed. * Expose APIs for getting nested dictionaries in Mixed. * Expose get_obj() on List and Set. * Expose method for getting element type in List. * Remove the need for element type helpers. The JS SDK has managed to use sentinel values in the generated bindings instead. * Remove unused header. * Avoid doing unneeded logger work in Replication Most of the replication log statements do some work including memory allocations which are then thrown away if the log level it too high, so always check the log level first. A few places don't actually benefit from this, but it's easier to consistently check the log level every time. * Prepare release * RCORE-1990 Add X86 Windows Release builder to evergreen (#7383) * Use the bfd linker in the armv7 toolchain (#7406) * Fix several crashes in the object store benchmarks (#7403) * add new index related benchmarks (#7401) * Use updated curl on evergreen windows hosts (#7409) * comment test not working + still missing handling for nested collections * handle collection array for mixed types * lint * please windows builder warnings + x86 * proposed fix for 32-bit * moved call outside assert macro * Fix for 32 bit archs for encoded Arrays (#7427) * tentative fix for 32 bit archs * removed wrong cast to size_t from bf_iterator::set_value() * fix inverted condition in 'unsigned_to_num_bits' * fix inverted condition in 'unsigned_to_num_bits' --------- Co-authored-by: Nicola Cabiddu <nicola.cabiddu@mongodb.com> * lint * Revert "lint" This reverts commit 7ac0073. * lint --------- Co-authored-by: Jørgen Edelbo <jorgen.edelbo@mongodb.com> Co-authored-by: James Stone <james.stone@mongodb.com> Co-authored-by: Thomas Goyne <tg@realm.io> Co-authored-by: Nikola Irinchev <irinchev@me.com> Co-authored-by: Claus Rørbech <claus.rorbech@mongodb.com> Co-authored-by: Kenneth Geisshirt <kenneth.geisshirt@mongodb.com> Co-authored-by: Jonathan Reams <jbreams@mongodb.com> Co-authored-by: Thomas Goyne <thomas.goyne@mongodb.com> Co-authored-by: Lee Maguire <lee.maguire@mongodb.com> Co-authored-by: LJ <81748770+elle-j@users.noreply.github.com> Co-authored-by: Yavor Georgiev <fealebenpae@users.noreply.github.com> Co-authored-by: Finn Schiermer Andersen <finn.schiermer.andersen@gmail.com>

tgoyne and others added 2 commits May 24, 2023 12:13

Remove set_string_compare_method

a8d9e0a

Partially based on 5f2dda1 Delete some obsolete cruft set_string_compare_method() and everyhing related to it has never actually been used by any SDK, and is not really the correct solution to the problem anyway.

strings are no longer equal to binaries

62f7e5e

ironage requested review from jedelbo and nicola-cab May 24, 2023 20:41

ironage self-assigned this May 24, 2023

cla-bot bot added the cla: yes label May 24, 2023

parser supports bin(...) to differentiate binaries

88264ef

ironage force-pushed the js/sorting-stage-2 branch from d816ccd to 88264ef Compare May 24, 2023 20:51

ironage added 2 commits May 24, 2023 16:03

fix sectioned results

d1e17df

fix formatting

5ca6892

ironage mentioned this pull request May 25, 2023

Sorting stage 3 #6670

Merged

3 tasks

Base automatically changed from js/sorting-stage-1 to next-major May 25, 2023 17:22

jedelbo requested changes May 30, 2023

View reviewed changes

ironage and others added 3 commits May 30, 2023 16:30

Merge branch 'next-major' into js/sorting-stage-2

bccd276

review updates

186fed2

Fix syntax

c42dc72

nicola-cab reviewed Jun 1, 2023

View reviewed changes

src/realm/query_conditions.hpp Outdated Show resolved Hide resolved

src/realm/query_engine.cpp Outdated Show resolved Hide resolved

src/realm/mixed.cpp Show resolved Hide resolved

ironage added 2 commits June 2, 2023 16:43

review feedback changes

307d919

Merge branch 'next-major' into js/sorting-stage-2

aa4f35f

ironage requested review from jedelbo and nicola-cab June 2, 2023 23:57

fix reported UB

2c9d1e5

lint

2edb84b

nicola-cab approved these changes Jun 6, 2023

View reviewed changes

jedelbo approved these changes Jun 12, 2023

View reviewed changes

Merge branch 'next-major' into js/sorting-stage-2

450fa04

bmunkholm requested a review from tgoyne June 13, 2023 08:16

Merge branch 'next-major' into js/sorting-stage-2

9044a69

ironage merged commit e76ae9e into next-major Jul 25, 2023
26 checks passed

ironage deleted the js/sorting-stage-2 branch July 25, 2023 18:44

ironage mentioned this pull request Oct 19, 2023

Change string sort order #6406

Closed

3 tasks

github-actions bot locked as resolved and limited conversation to collaborators Mar 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sorting stage 2 #6669

Sorting stage 2 #6669

ironage commented May 24, 2023

tgoyne commented May 24, 2023

ironage commented May 24, 2023

bmunkholm commented May 25, 2023

ironage commented May 25, 2023

bmunkholm commented May 26, 2023

ironage commented May 29, 2023 •

edited

Loading

jedelbo May 30, 2023

ironage May 31, 2023

jedelbo May 30, 2023

jedelbo May 30, 2023

ironage May 30, 2023

jedelbo Jun 1, 2023

ironage Jun 2, 2023

jedelbo May 30, 2023

ironage May 30, 2023

jedelbo Jun 1, 2023

ironage Jun 2, 2023

jedelbo May 30, 2023

ironage May 31, 2023

tgoyne commented May 30, 2023

ironage commented May 30, 2023

nicola-cab left a comment

jedelbo left a comment

ironage commented Jun 12, 2023

tgoyne commented Jun 14, 2023

ironage commented Jun 15, 2023

ironage commented Jul 25, 2023

		@@ -694,11 +694,8 @@ inline BinaryData Mixed::get<BinaryData>() const noexcept
		{
		if (is_null())

@@ @@ -157,6 +157,8 @@ class ConstantNode : public ValueNode { @@
                       FLOAT,
                       STRING,
                       BASE64,

Sorting stage 2 #6669

Sorting stage 2 #6669

Conversation

ironage commented May 24, 2023

☑️ ToDos

tgoyne commented May 24, 2023

ironage commented May 24, 2023

bmunkholm commented May 25, 2023

ironage commented May 25, 2023

bmunkholm commented May 26, 2023

ironage commented May 29, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tgoyne commented May 30, 2023

ironage commented May 30, 2023

nicola-cab left a comment

Choose a reason for hiding this comment

jedelbo left a comment

Choose a reason for hiding this comment

ironage commented Jun 12, 2023

tgoyne commented Jun 14, 2023

ironage commented Jun 15, 2023

ironage commented Jul 25, 2023

ironage commented May 29, 2023 •

edited

Loading