-
Notifications
You must be signed in to change notification settings - Fork 32
Chainable query class for FeatureService #33
Comments
This would be sweet. Makes total sense |
Agreed, would be brilliant. Is there a standardised/conventional way to express where clauses like this? I'm thinking wildcards, AND vs OR, etc.? |
@nixta i dont think so you could do something like this Query().within(extent).where({
"city": "Portland",
"score": "> 50",
"type": "Food Cart"
}) I think each key/value in the "where" could be joined by an and "AND". Might need to think of a way to do "OR"s though. Query().within(extent).where("WHERE city IS portland AND score > 50 AND type IS food cart"). ^^ Probably not valid SQL |
I have a suggestion. Extents are challenging to work with and figure out, especially for non-GIS and newbie GIS developers. What about a radius based search? |
@andygup that sounds great. I think
|
Nice. Having that as an option would be awesome. |
Matt Priour
.where ( { city: "Portland" }).or({ state:"TX"})
|
@mpriour that looks great to me. For clarity since the email stripped it .where ( { city: "Portland" }).or({ state:"TX"}) |
I also think a convention of using an array for a value within a 'where' or Matt Priour |
Like .where ({
state:["TX", "OR"]
}) |
Yep Matt Priour |
Ok looks like I get to tackle this and try to get it working in time for UC. |
I found a couple of things that might be useful. nodejsdb opts for chaining at the operator level. So you'd get Also some interesting ideas in Fluent Query Builder but some strange ones too. @patrickarlt and @mpriour Agree re: array (see jsondb's last e.g. |
Don't want to hijack the conversation but… :) @andygup The radius search would need more server-side help - still need something like |
Chaining AND and OR operators as functions might get hairy very quickly -- each function in the chain would have to be aware of the preceding function, we'd have to think about operator precedence as well, for example:
|
@nixta it's easy to generate the radius-based polygon geometry on the client. No need for a server round trip. |
@ngoldman You wouldn't have Also, I think you would have to insist that the above be |
https://github.com/brianc/node-sql might be worth a look, they're building a "sql string builder for node" that supports chainable operators. They even have support for patterns like this: .where(
user.name.equals('boom').and(user.id.equals(1))
).or(
user.name.equals('bang').and(user.id.equals(2))
) |
@ngoldman you can't have more then on geometry in a query. look at the Looking at the query docs we would have @nixta it wasn't me but that would be cool to do in a future release. I think the nodedbjs is harder to read then raw SQL the fluent query building stuff looks a little like what we are doing here but in PHP. I think we need to go for something that feels really good in javascript. |
@nixta @patrickarlt yep, simple radius search is the use case. Just drawing a basic circle polygon in either wgs84 or web mercator and using that as the input geometry for the query. |
my fear is that someone will expect a circle search from a point and radius where we would actually be building a bounding box and searching on that. |
@JerrySievert you can search on my geometry so we could search on the resulting polygon itself not its bounding box. See the geometry param on http://resources.arcgis.com/en/help/arcgis-rest-api/#/Query_Feature_Service/02r3000000w5000000/ |
@patrickarlt @andygup OK. If that's the use case, then yes, absolutely. I was thinking more about the reason one would do a query like that. You have a focal point and want stuff nearby. It kind of pokes a hole in FeatureService queries. Combined with the feature limit on single queries on a FeatureServices, that can get inaccurate as soon as the outline gets biggish compared with your data density. I.e. if there are 2000 features within the radius you've requested but the FeatureService only returns 1000 per query, you won't get the 1000 closest, you'll get whatever comes back first in the R-Tree (probably). Again, true for any query, but I was thinking of a different use-case. Like I said, I feel I'm hijacking this conversation as this is a side-issue to do with Server. @JerrySievert I think we can search within the radius fairly well (which is what @andygup meant). No need to degrade to bounding box. |
@nixta is it impossible to page through a feature service query? I dont see any options to do it on the docs page. It sucks that we can never get all the features for a query and that this is a limitation of feature services. |
@nixta I'm fairly certain when you create the feature service you can bump up the number of results that can be returned. I don't have any links handy that I can post on it. @patrickarlt would you really want to return all features? Could cause performance issues for browser-based apps, especially mobile ones. |
@andygup web workers help deal with some of those performance issues. and don't forget, this is ostensibly a library for node that "just happens" to have a browser build - at least until we rename it something like |
@patrickarlt Correct (AFAIK, though I'll be over the moon to be wrong). I suppose you could use the @andygup You would at least want a way to get them all even if it was up to you to page through them to build the complete set. As it stands, you don't even know that you've hit the limit and you haven't got them all. And yes, I think you can up the return count when you create the feature service (although IIRC current advice is that you shouldn't do it without good reason). EDIT: See my response below - you can page (from the client) and you can tell you've hit the limit in 10.1 services upwards. |
Ah! I was wrong(ish). You can do paging, but it has to be client-side paging. So, you make a request using However, @patrickarlt, I don't see that a FeatureService declares what its max feature count is, so one would have to know that on a per-service basis. |
@andygup I want all the features because clients are powerful enough now to index 10000k+ points. clustering and indexing on the client is very fast, DOM manipulation is the limiting factor but can be worked around. Also since this is for node you might want all the features to do your own processing on them. |
OK. I was extra-wrong. As of 10.1, you can tell if your query would have returned more records than the FeatureService allows. You will get Also as of 10.1, you can read |
Great so we know we, exceeded the limit, what the limit is and how much we exceeded it by. But we still cant figure out how to make another request to get ALL the features. |
Seems simple enough. I see two possible options:
Option 2 seems better unless I've missed something. All dependent on the service being 10.1 or greater of course. Update: Apparently, on a pre-10.1 service we should use option 1 and request batches of 500 |
Oh, to clarify, you would pass the Hope this is making some sense. |
Here is a link to chainable complex query generator that has some nice syntax examples: |
@nixta - those 2 paging options are the same ones that I found when I last researched it as well. |
What is the reason to limit queries to 1000? Technically you could set it to whatever you want when you create a service so why limit it? |
@patrickarlt It's a server setting. I helps prevent people from swamping the server with excessively large responses, etc... Basically in pre 10.1, if you truly want ALL the features, you HAVE to do any |
@mpriour Yep. Thanks. I was a little surprised I was given the number 500 too. Have asked for clarification. |
I'm curious where this ended up. Seems to have derailed on pagination. I'm interested in the original topic of a chainable, relational algebra, type of api. Very similar to Arel. This type of API would be easier to work with and also enable currying. Anyone want to help move it forward? |
I walk talking with @mpriour about this it would be great to have a class that would abstract the complexities of querying a feature service.
Here was the example I gave @mpriour.
Query().within(extent).where({"city": "Portland"})
We could also hang this off
FeatureService
.The text was updated successfully, but these errors were encountered: