Invoke Async Refactor

[magiccodingman]

Replaced Blazor Interop & Moved To Custom Communication

I've stripped away from Blazor their interop control over the library. No more double serializing and double deserializing. Instead "magicDbMethods.js" was introduced to dynamically call methods and replicate effectively what the modern Blazor Invoke Async already does. This removes the double unnecessary serializations.

Additionally I've moved everything to a streamed async communication to remove all C# to JS limits (aka no 16 MB limit on WASM nor 32 KB limit on server rendered / aka signalR). This introduces slight latency, but uncapped data transfer. The improvements in performance though showed in my results that the increase in latency is vastly outweighed by the removal of double deserialization, making this a net positive all around.

The only down side to this new version memory allocations that occurs at fractional seconds of memory processing and translation. This would only be an issue when data being transferred is somewhere around 70 MB to 100 MB (not fully sure yet). There's a few ideas I'm considering to remedy this. But the issue is inherent to Blazor itself and the fact it does NOT support JSON parse streaming, which is kind of exactly what the doctor ordered here. Without proper Json Parse streaming, options are limited. I will mull over this concept for a bit, but for now we're not just drastically faster, but bandwidth increased by ~4.4X minimum on Blazor WASM and 2,250X higher on SignalR server rendered blazor.

JS Where Massive Improvement

The "where" method in JS has gotten a large boost in performance. instead of utilizing getTable and grabbing large amounts of items to be processed in memory. Now properly utilizing cursors, we properly translate the where statement to iterative row lookups. Making performance both faster and compressing memory use drastically.

Feature	Old Version	New Version	Loss?
Dynamically handles conditions (Equal, GreaterThan, LessThan, etc.)	✅ Yes	✅ Yes	❌ No
Supports OR conditions (jsonQueries with multiple conditions)	✅ Yes	✅ Yes (parallel execution)	❌ No
Supports sorting (orderBy, orderByDescending)	✅ Yes	✅ Yes	❌ No
Supports pagination (skip, take, takeLast)	✅ Yes	✅ Yes	❌ No
Supports indexed queries when possible	❌ No (forced in-memory filtering via .and())	✅ Yes (checks for indexed properties first)	✅ Improved
Supports cursor-based iteration for large datasets	❌ No (always loads everything into memory)	✅ Yes (avoids memory bloat with .each())	✅ Improved
Executes OR queries in parallel	❌ No (sequential execution)	✅ Yes (Promise.all() for speed)	✅ Improved

Review Notes

I'm going to manually just push this to Main later. Was just using this PR as documentation of changes and leaving it hanging for a day or 2 while I think over the changes. I'm also in the midst of altering the predicate builder system as well to be vastly more performant as well. Unsure if that'll be in this PR or another. This will also open the doors for us to create capabilities of more complex query capabilities with Joins and more. Well more like helpful illusions because IndexDB can't do that by default, but these are all very helpful new additions.

I'm also unsure if this PR will have this update or not, but I'm also trying to remove the "Execute" command entirely. It's unnecessary and should work like LINQ to SQL in which you can "ToList()" to get a list or you can work with the IEnumerable with deferred execution, or use that to make individual calls in a loop or anything else.

Though @yueyinqiu I don't see any where statement unit tests. We may want to make a to do list together at some point of unit tests we want to create. Because I just want to make sure I'm not making changes that breaks current functionality.

Feature Requests

• Can use the new "MagicName" Attribute to rename columns without making it an index. cannot customize column name when it's not an index/id #40

Additional fixes

• Fixed complex query statements not translating
• Repaired a variety of query logic
• Fixed camel case non attributed properties. Camel case should NEVER apply to any property within an attributed Magic Table.

People In IndexedDB!

ID	Name	Age	Not Mapped	Access
17	Zack	45		CanRead
18	Luna	35		CanRead, CanWrite
19	Jerry	35		CanRead, CanWrite, CanCreate
20	Jon	37		CanRead
21	Jack	37		CanRead, CanWrite
22	Cathy	22		CanRead, CanWrite
23	Bob	69		CanRead
24	Alex	80		None

---

Complex Query Capabilities!

Within the table, "Id" is the primary key, "Name" is indexed, and "Age" is not an index at all, but utilizes cursors to query.

The following query is executed against IndexedDB using a LINQ-style syntax:

Here is your Markdown-formatted content for GitHub:

People In IndexedDB!

ID	Name	Age	Not Mapped	Access
17	Zack	45		CanRead
18	Luna	35		CanRead, CanWrite
19	Jerry	35		CanRead, CanWrite, CanCreate
20	Jon	37		CanRead
21	Jack	37		CanRead, CanWrite
22	Cathy	22		CanRead, CanWrite
23	Bob	69		CanRead
24	Alex	80		None

Complex Query Capabilities!

The following query is executed against IndexedDB using a LINQ-style syntax:

await manager.Where<Person>(
        x => x.Name.StartsWith("c", StringComparison.OrdinalIgnoreCase)
        || x.Name.StartsWith("l", StringComparison.OrdinalIgnoreCase)
        || x.Name.StartsWith("j", StringComparison.OrdinalIgnoreCase) && x._Age > 35
    ).OrderBy(x => x._Id).Skip(1).Execute();

Results

The query returned the following matching people:

Name: Cathy - Age: 22
Name: Luna - Age: 35
Name: Jon - Age: 37
Name: Jack - Age: 37

---

How the Logic Works

This query correctly filters and processes the dataset due to proper query translation and ordering:

1. Filtering Logic:

   • Name.StartsWith("c") → Matches Cathy.
   • Name.StartsWith("l") → Matches Luna.
   • Name.StartsWith("j") && Age > 35 → Matches Jon and Jack (but not Jerry, because he is 35, not greater than 35).

2. Ordering & Skipping:

   • Ordered by ID in ascending order.
   • Skipping the first result, ensuring correct offset-based pagination.

3. Final Matching Set:

   • Cathy (StartsWith("c")).
   • Luna (StartsWith("l")).
   • Jon (StartsWith("j") and Age > 35).
   • Jack (StartsWith("j") and Age > 35).

This properly shows that we are now querying accordingly in a complex query situation.

Big Feature Change

Due to this rework and repair. I implemented Cursors when items are not indexed. This means that you can iteratively look through rows and properly query even if it's not an index!

[magiccodingman]

This has issues do not merge

[magiccodingman]

Oh damn, I thought my code was breaking the LINQ to IndexDB expression translation. I actually learned the original logic was wrong and off from the beginning. Wow, kind of surprised that wasn't noticed. Maybe it was, not sure. But complex queries aren't working right. I'm going to include in this patch a refactor to fix this

[magiccodingman]

@yueyinqiu This started as a refactor and became additionally major repair and feature request fulfilment. I'll likely make this another release tomorrow or the day after, but I think we'll want to add queries to our unit tests for sure. But lots of updates here today! I had fun since it was on my mind and it was all around good refactoring, lots of bug fixes, and it makes the project much more reliable.