Operations applying query options after custom code

[andreav]

Assemblies affected

RESTier-1.0.0

Reproduce steps

• Define a bound function/action.
• Invoke function/action with some query option (i.e. $filter)

Expected result

• If first function/action parameter is an IQueryable:
  • no query is issued to db
  • user can further manipulate queryable befire hitting the database
• If first function/action parameter is an IEnumerable:
  • query is issued to db including query options
  • IEnumerable received into custom code is the result of that query
  • user can further manipulate returned data

Actual result

• Query is immedeately submitted to db without applying query options
• Then custom controller code is invoked
• First function/action parameter has all records form the "EntitySet" table.

Additional details

Example function:

        [Operation(Namespace = "My.DataModel", IsBound = true, EntitySet = "Account")]
        public IQueryable<Account> FunctionExample(IQueryable<Account> accounts)
        {
            // Here accounts is the whole table, not respecting $filter query options
        }

Example invoking url

  `http://localhost:1406/ApiV1/Account/My.DataModel.FunctionExample()?$filter=Id eq 2`

[andreav]

We found that modifying code inside RTestierController::Get function like ythis makes it work.

index 5487b71..3ce4865 100644
--- a/BackEnd/RESTier/src/Microsoft.Restier.AspNet/RestierController.cs
+++ b/BackEnd/RESTier/src/Microsoft.Restier.AspNet/RestierController.cs
@@ -102,15 +102,17 @@ private ApiBase Api

                 if (lastSegment is OperationSegment)
                 {
-                    result = await ExecuteQuery(queryable, cancellationToken).ConfigureAwait(false);
+                    var applied = await ApplyQueryOptionsAsync(queryable, path, false).ConfigureAwait(false);
+                    result = await ExecuteQuery(applied.Queryable, cancellationToken).ConfigureAwait(false);
+                    //result = await ExecuteQuery(queryable, cancellationToken).ConfigureAwait(false);

                     var boundSeg = (OperationSegment)lastSegment;
                     var operation = boundSeg.Operations.FirstOrDefault();
                     Func<string, object> getParaValueFunc = p => boundSeg.Parameters.FirstOrDefault(c => c.Name == p).Value;
                     result = await ExecuteOperationAsync(getParaValueFunc, operation.Name, true, result, cancellationToken).ConfigureAwait(false);

-                    var applied = await ApplyQueryOptionsAsync(result, path, true).ConfigureAwait(false);
-                    result = applied.Queryable;
+                    //var applied = await ApplyQueryOptionsAsync(result, path, true).ConfigureAwait(false);
+                    //result = applied.Queryable;
                     etag = applied.Etag;

                 }

In your opinion Is this correct?
Should this also be done for Post/Put?

Thank you.

[robertmclaws]

Mike, let's discuss this next week during the standup. @andreav join us at 12pm EST at https://bit.ly/RESTierYouTube if you'd like to participate.

[mikepizzo]

As I understand the issue, this is the correct behavior; according to protocol, any query options (such as $filter) are applied to the result of an action or function, not to the input set.

OData 4.01 introduced a "filter segment" which can be used to apply a filter to the set that a function or action is bound to. The syntax would be something like:

http://localhost:1406/ApiV1/Account/$filter(Id eq 2)/My.DataModel.FunctionExample()

There is limitted support for the filter segment in the OData Library and WebAPI OData, but this is not currently supported in RESTier.

[andreav]

Thank You for the clarification.

However I do not understand that behavior. I though bound operations were meant to operate on a subset of the whole rows.

My use case can help explain me:
we have tables with millions rows. User usually selects hundreds rows with some kind of filter and launches some operation over them. This translates into an invocation of an operation, which will execute some work for every row selected by the user. I thought restier filtered rows for me, instead I receive millions rows in my code and I don't know how to get the originally selected subset.

Which is the point of retrieving everything and then throwing away 90%?

As I said, this is just my use case, sure there is some good reason.

Thank you.

[robertmclaws]

Alright, so we talked about this a bit in the Community Standup yesterday. I think it's best to explain what is going on in the context of C# functions and how compiled languages work. The behavior you were expecting is exactly the same as what I personally would have expected, but once I looked at it this way, the behavior made sense.

I want to start out by pointing out that the URL you used as an example is slightly flawed. If "Account" is an EntitySet, then it should be plural ("Accounts"). For the rest of this discussion, I will be referring to it in the correct plural form.

Original Query

http://localhost:1406/ApiV1/Accounts/My.DataModel.FunctionExample()?$filter=Id eq 2

The order of operations here is this:

• Get the data from the EntitySet: http://localhost:1406/ApiV1/Accounts
• Execute a function against that data: My.DataModel.FunctionExample()
• Filter the results of the function down: ?$filter=Id eq 2

The parenthesis in the original query show that you've already called the function, then the filter is applied to the result of that function.

What you're actually looking for is something closer to this:
http://localhost:1406/ApiV1/Accounts?$filter=Id eq 2/My.DataModel.FunctionExample(), where the order of operations is:

• Get the data from the EntitySet: http://localhost:1406/ApiV1/Accounts
• Filter the down: ?$filter=Id eq 2
• Execute a function against the filtered data: My.DataModel.FunctionExample()

This wasn't actually possible in OData v3 or v4. However, the v4.01 spec was just ratified, and you will be able to do accomplish what you're looking for in the next major release of both WebApi OData and Restier, but with a slightly different syntax.

4.01 Semantics

http://localhost:1406/ApiV1/Accounts/$filter(Id eq 2)/My.DataModel.FunctionExample()
Notice here that the filter is a path segment, and is called like a function, not a QueryString parameter.

Hopefully this makes sense. We will be adding a Unit Test to cover this type of a query, and I will make sure we update the Restier documentation to cover this type of scenario.

Let me know what you think. Special thanks to @mikepizzo for explaining this to me, so I could explain it to you 🙂.

[robertmclaws]

I'm going to go ahead and close this, as it is not a bug, but I've referred to it in my documentation task, and we can still keep discussing it if you so desire. Thanks for your input!

[andreav]

Thank you Robert & Mike for the detailed explanation. I understand the point, and I will try the new version as soon as it will be released. Il mer 10 lug 2019, 17:49 Robert McLaws <notifications@github.com> ha scritto:

…

I'm going to go ahead and close this, as it is not a bug, but I've referred to it in my documentation task, and we can still keep discussing it if you so desire. Thanks for your input! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#642?email_source=notifications&email_token=AAKLTYAROXMBUWIDHDI552TP6YAHNA5CNFSM4HYW6IXKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZT4YMA#issuecomment-510118960>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKLTYB7Y447HO23TJZSLMTP6YAHNANCNFSM4HYW6IXA> .