typed data transfer objects for woql queries

[matko]

Currently we have the ability from WOQL to return structured json output through a generic templating mechanism. While this is great, there's a disadvantage with the current design: our json output is not schema-validated.

While this is probably fine for many sorts of use cases, it would be nice if we could guarantee that the output of a woql query follows a particular structure. This would allow clients to ingest this data using auto-generated class definitions. It could also allow us to insert the result of a woql query straight into the database, or otherwise integrate it into pipelines that expect some sort of data.

We could possibly use the ordinary schema we already have for this, but data transfer objects are not necessarily the same as storage objects. for one, they don't necessarily need IDs everywhere. So we may want to expand the schema language a bit with this.

[matko]

The same problem is actually there for woql queries that may take documents as input. These too would be nice if they were typed, but they don't have to be typed in the way we type classes. So a more generic way of looking at this is that we may want typed data transfer objects for use in wqol queries.

Those types can either be part of the schema, or we could submit them along with the woql query.

[matko]

For stored queries, being able to express inputs and outputs in terms of typed data transfer objects would be nice too.

[GavinMendelGleason]

This would indeed be great, but where does the WOQL query get stored and how do we refer to the type designation of the element referred to? This is something I haven't thought a satisfactory answer for. Should we be storing queries in the schema directly?

The current templating mechanism breaks all guarantees about form and structure but decouples results from the schema which is convenient if you don't store the woql using the same schema.

[matko]

IMO the logical place for storing reusable WOQL queries is the schema yes. Alternatively we may want an extra graph type for it in order to decouple the two, but it works as schema information as well ("here is a list of standard things you can do with this schema"). That said, typing can be nice even if you're not storing the query at all. And in various cases, not typing things can be nice too, so I still like the current templating mechanism.

We could have an extra woql word to create typed variables (or annotate existing variables with a type). Variables with an annotated type will error when they are ever unified with something not matching the type. Then we make sure that our document interface accessors can work with these variables. This may also be nice for the other woql words, as it'd allow us to check at compile time that certain operations are not going to work (like trying to do arithmetic with nodes).

So, in pseudo flow woql:

WOQL.typed(Person, X).get_document('http://somewhere/data/Person/gavin', X)