Implement query language

[proycon]

Implement a higher-level query language. Effectively parsing and translating high-level queries into calls to lower-level search methods. One major challenge is to find in what parts of a query to execute before others, in such a way that the search space is as small as possible (= quickest search results and most performant).

Depends on:

• Formulate a STAM Query Language stam#12
• Implement search methods (extended model) #7

This will take significant time to implement (wild guess: 150 hours).

[proycon]

I've done a lot of brainstorming on this this week and this is a highly complex topic.

• A query language needs to be formulated (Formulate a STAM Query Language stam#12)
• A parser needs to be written to parse the language
• The parsed query needs to be transformed in to an executable Query Plan, at this point a dependency graph/tree is usually formed, and a query optimiser finds an execution order that is performant.
• Last the query engine (interpreter) executes the query plan and returns the results

(I think my earlier time estimate can be at least doubled or tripled)

I'm taking a step back a step back at this stage and postponing such an implementation until a later point in time where it becomes more relevant. I think at this stage (with no real users yet) it's more prudent to focus on searching with the lower-level API that is already implemented, and optimize and document that API.

[proycon]

Coming back to this topic: I did some further investigation and came to the conclusion that the current lower-level API is insufficient for proper search and querying. Expressing a higher-level search question with this API is simply too complex. So a higher-level search interface (API) is something that can not be avoided anymore.

[proycon]

A higher-level API has now been implemented, it's still short of a fully-fledged query language, but it makes this less of a priority.

[proycon]

The query language has been implemented, they may still be some loose ends but most of it works nicely.