Support $search for .Net Core

[suadev]

Is there any plan for $search support for asp.net core?

[suadev]

@xuzhg

[bdebaere]

@suadev Can you clarify what you mean, and what you need search for?

You can query for multiple options with the in operator in .NET Core e.g. /odata/MyEntity?$filter=property in (1, 2, 3)

[suadev]

@bdebaere I am talking about this.

$search	The $search system query option allows clients to request entities matching a free-text search expression.For example:http://host/service/Products?$search=blue OR green

https://help.nintex.com/en-us/insight/OData/HE_CON_ODATAQueryCheatSheet.htm#Query

[itanex]

The reason to support this would be to comply with the specification set forth and referred to throughout the documentation for this library.

http://docs.oasis-open.org/odata/odata/v4.0/errata03/os/complete/part1-protocol/odata-v4.0-errata03-os-part1-protocol-complete.html#_The_$format_System

[bdebaere]

@suadev @itanex I don't understand how $search works. $filter uses a property to filter against, but what does $search target with its predicate?
@itanex The specs you linked show how the query option is supposed to be presented, but not how it is supposed to filter. If I knew how, and it's not too much work, I could try my hand at implementing it.

[NetTecture]

There is no standard approach to $search. " The definition of what it means to match is dependent upon the implementation." - that said, you can always parse $search yourself and add it to the IQueryable you return for processing in Odata as prefilter-condition.

[itanex]

The specification of OData 4.0 says:

11.2.5.6
The $search system query option restricts the result to include only those entities matching the specified search expression. The definition of what it means to match is dependent upon the implementation.

So the details are not explicit. Let's look at how another implementation works.

Salesforce
Requests entities that match the search query string as a free-text search expression. Enable this option by selecting Use Free-Text Search Expressions on the external data source.

Salesforce implements this to search all fields returned by the entity or $select and reduce the content returned. Such that the expression from the specification http://host/service/Products?$search=bike would return all products that contains the string bike in any fields returned.

This is just one example of the actual implementations I found, most are very common, and some apply a scoring mechanism in addition.

[bdebaere]

Seeing as how you are also allowed to specify a LambdaExpression e.g. ?$search=x and y, it's not as simple as extending the ODataController with a method one must override like below.

public abstract class ODataController
{
    public virtual IQuerayble Search(string parameter);
}

You'd need something like below. In which case you would handle the conversion of the LambdaExpression to an implementation of your choice. Routing would have to modified so that the method is hit whenever $search is present in the query parameters.

public abstract class ODataController
{
    public virtual IQuerayble Search(LambdaExpression parameter);
}

Alternatively an implementation is possible which allows one to register methods as what to bind $search to. An example call would be builder.EntityType<User>().BindSearch(Search). I quite like that approach as it is not tied to routing.

public class EntityTypeConfiguration<TEntityType>
{
    public void BindSearch<TEntityType>(Func<IQueryable<TEntityType>, LambdaExpression, IQueryable> method)
    {
        // Register search method.
    }
}

public IQueryable Search(IQueryable<User> queryable, LambdaExpression lambdaExpression)
{
    // Add filtering based on lambdaExpression.
    return queryable;
}

public class User
{
    public string Name { get; set; }
}

I'm not sure if LambdaExpression is the best type though. From my limited knowledge I see that "and" and "or" should be supported, but beyond that LambdaExpression might be too broad.

[itanex]

A $search expression as defined by the specification follows the following grammar.

search     = '$search' EQ BWS searchExpr
searchExpr = ( OPEN BWS searchExpr BWS CLOSE
             / searchTerm 
             ) [ searchOrExpr
               / searchAndExpr
               ]

searchOrExpr  = RWS 'OR'  RWS searchExpr
searchAndExpr = RWS [ 'AND' RWS ] searchExpr

searchTerm   = [ 'NOT' RWS ] ( searchPhrase / searchWord )
searchPhrase = quotation-mark 1*qchar-no-AMP-DQUOTE quotation-mark
searchWord   = 1*ALPHA ; Actually: any character from the Unicode categories L or Nl, 
                       ; but not the words AND, OR, and NOT

Extracted from odata-abnf-construction-rules.txt as referenced by [OData-ABNF] in the spec.

The spec refers to this as a search expression and not a lambda (functional expression). The syntax provides for grouping and isolating binary evaluations, nothing more.

product$search=bike
Example: [bike_record_1, bike_record_2, bike_record_3]

product$search=bike not moutain
Example: [bike_record_1, bike_record_3]
bike_record_2 is excluded because it is tagged ['bike', 'mountain']

product$search=bike not (moutain-bike and bmx)
Example: [bike_record_1]
bike_record_2 is excluded because it is tagged ['bike' 'mountain'] bike_record_3 is excluded because it is tagged ['bike'] and its title contains the string bmx

[raviteja333]

Any plans to support this in the near future?

[jiggyswift]

I posted this in Hassan's blog, but it really belongs here on this issue. I don't think the Microsoft team really understands how useful this feature is. For example, salesforce supports OData v4 $search feature already for free form text searches, and right now the Middleware throws an exception if $search is included in a querystring. So this needs to be fixed, and the information in the querystring should be exposed in ODataQueryOptions.

My noted to Hassan were:
Hassan, can you talk to your team about supporting the OData v4 $search feature (in aspnetcore oData). I have been following your posts, and was inspired enough to build an OData API exposing some of my services, which I then tried to hook into salesforce (so Salesforce could natively query my business object).

In general, it worked a treat (thanks to your team), but with one annoying problem: Salesforce allows you to enter a free form text search string at the top of the page. Salesforce converts that search into either:
a) a whole bunch of Field1.Contains OR Field2.Contains (repeat for 40 fields).....which doesn't work for me performance wise.
or
b) a querystring param $search=MySearchString included in the OData query.

Option "B" would be perfect, and the expectation is the developer does whatever they want with the search string supplied (your team should not be trying to do anything with the search string other than read it correctly and expose it to me as a developer). Read up on the OData v4 spec if you want to know more about $search, but it allows the client to send a free form search to the backend and the interpretation of how to do the search is left up to the backend.

The problem with your current implementation is that an exception is thrown by the middleware if $search is found in the querystring. My best thought to get around this is to actually rewrite that querystring on the aspnetcore serverside form "$search=Blah" to "search=Blah" (so the middleware doesn't throw an exception).

Note: This seems like an opportunity to provide access to $search in the ODataQueryOptions object. That simple change would allow my code to check if $search has been supplied, and provide some pre-filtering (however I want).

I should add (in case it is not clear), that you might think the solution could be solved with a Route exposing an OData endpoint, like /MySearchString/OData or /Odata?search=MySearchString. However this doesn't solve the problem because clients like Salesforce use the OData v4 standard ($search) and do not let you configure another way to send the fee form text search string to the backend.

Hopefully I have explained this sufficiently, but this change would add to your compliance with OData v4, plus make dotnetcore a much better backend for Salesforce (as the OData client accessing the dotnetcore hosted OData service).
Thanks.

[fuentco]

Hello, Any updates on implementing $search for OData? I am using Grid component of Syncfusion and the implementation uses this parameter.

[EugeneKud]

Hello, Any updates on implementing $search for OData? I am using Grid component of Syncfusion and the implementation uses this parameter.

Me too!

I use the SfGrid and it would be nice to have a cross-column search as demonstrated here:

https://blazor.syncfusion.com/demos/datagrid/search?theme=bootstrap4

Having this feature, even as a separate NuGet package would be very useful!

[NetTecture]

Let me say that there is NOTHING stopping you from doing it now as FUNCTION:

• One parameter that is the search string

There is nothing that stops you from actually returning a collection AND in the function you get access to query parameters. You CAN configure in the backend your IQueryable to do the search term filtering, then process the filter query parameters.

Done.

Doing that for quite some time now. As long as you can live with running this through a function, it works like a charm.

[EugeneKud]

Well, there is nothing stopping me from implementing the whole OData protocol (or a custom version of it) manually. This was the way I did it for several years before stumbling on Syncfusion components.

However, it does seem strange to implement an OData standard feature (described in the BASIC tutorial: https://www.odata.org/getting-started/basic-tutorial/#system_query_option_search_18) with custom code.

[Sreejithpin]

@mikepizzo

[mikepizzo]

I used to have a sample that showed how it was possible to use $search in WebAPI, but it was pretty convoluted, and relied on custom code to actually apply the logic to the query.

The problem with $search is that the semantics of what it means to match is left intentionally undefined. So one implementation may do a "contains" search on all (textual) properties (and perhaps even textual properties of related entities), while another might do a fulltext search on one or more content stream properties.

Perhaps a middle ground would be if we allowed the service implementation to plug in code to handle $search. We could provide the parsed $search expression, and the custom logic could either apply it directly to the results or return a LINQ expression that we could merge into the query.

Would such a hook be helpful? Would it be helpful to define a default implementation (i.e., contains on each string property?)

[bdebaere]

@mikepizzo I think a hook would be better, something like what I suggested (but I'm assuming you have a better idea), or at the very least allow for a hook. I think OData can use a bit more extensibility in most areas.

[EugeneKud]

@mikepizzo @bdebaere Yes, a hook would be very helpful!

Moreover, a default implementation with contains (only on textual properties of searched entity, without going into properties of related entities) would be just perfect. In my opinion, it feels like an expected behavior and would cover the requirements of many developers.

[maulik-modi]

@mikepizzo , @habbes , @xuzhg , Most of us agree here that Extensibility option based on Hook is what we would require.

[MMeshkiny]

@mikepizzo , @habbes , @xuzhg , Most of us agree here that Extensibility option based on Hook is what we would require.

me too

[julealgon]

Shouldn't this issue/feature be migrated to the https://github.com/OData/AspNetCoreOData repo?

[xuzhg]

@julealgon I enabled the $search binder in the https://github.com/OData/AspNetCoreOData

See comments: OData/AspNetCoreOData#410 (comment)

[xuzhg]

@ALL, here's a post for $search: https://devblogs.microsoft.com/odata/compute-and-search-in-asp-net-core-odata-8/

[gathogojr]

$search and $compute enable in Microsoft.AspNetCore.OData 8.0.5 release