A "special function" for dynamic/custom/variables/on-the-fly queries

[frederikhors]

What would you like to be added:

A "special feature" for fully custom and dynamic queries using GraphQL variables.

Why is this needed:

A really incredible peculiarity of SuperGraph is that of creating very fast pre-compiled queries.

This clashes with the needs that emerged from issues #114 and #115: having the ability to create dynamic and custom queries.

It would be great to be able to use GraphQL's intrinsic dynamism (expressed by the variables in the paypload) to create queries on-the-fly (obviously not pre-compiled), (maybe) slower but certainly more useful in some applications.

Example:

{
  "query": "query players($order_by: playerOrderBy!) {
    players(order_by: $order_by) {
      id
      created_at
    }
  }",
  "operationName": "players",
  "variables": {
    "order_by": {
      "id": "desc"
    }
  }
}

or

{
  "query": "query players($where: playerWhere!) {
    players(where: $where) {
      id
      created_at
    }
  }",
  "operationName": "players",
  "variables": {
    "where": {
      "and": {
        "price": {
          "gt": "10"
        },
        "not": {
          "id": {
            "eq": "5"
          }
        }
      }
    }
  }
}

What you can today

I think one of the ways to get around this today with SuperGraph is to create many queries in clients (javascript or whatever it is) each for every need.

Example:

1. Sorting by different columns; by ID DESC:

   export const all_players_by_id_desc = `
     query players {
       players(order_by: {id: desc}) {
         id
         created_at
       }
     }
   `

2. Sorting by different columns; by created_ad DESC:

   export const all_players_by_created_at_desc = `
     query players {
       players(order_by: {created_at: desc}) {
         id
         created_at
       }
     }
   `

And so on...

As you can understand, this is really expensive.

With SuperGraph I increase the backend productivity but I decrease the frontend one.

Can you suggest something different, @dosco? Where do you think I'm wrong?

[dosco]

Queries are never entirely dynamic it's always a finite set. I could provide a flag for you to disable the production mode entirely so all queries are always compiled. FYI compiling is very fast so it won't be an issue unless you are dealing with a very high QPS. The issue then is that anyone on the internet can then execute any query against your DB partially limited by your backend filter and column configs.

However I'll never recommend allowing the public internet to control your order by column name or set an arbitrary limit flag both of which can be used to DOS your database. The order by set could be defined in the backend and it could be limit to one of those and limit could have a cap defined in the config.

If you use fragments your GraphQL queries can pretty easy to write and really light weight and they will also be fast and secure.

fragment Player on players {
  id
  created_at
}

query getPlayersIdDesc {
  players(order_by: { id: desc }) { ...Player  }
}

query getPlayersCreatedAtDesc {
  players(order_by: { created_at: asc }) { ...Player  }
}

[frederikhors]

Queries are never entirely dynamic it's always a finite set.

Why?

I could provide a flag for you to disable the production mode entirely

There is no need.

Quite simply: I just need to filter on some columns of some tables.

unless you are dealing with a very high QPS

I am.

The issue then is that anyone on the internet can then execute any query against your DB partially limited by your backend filter and column configs.

Can you explain better these risks?

I wish I could configure which columns to filter.

Create the appropriate indexes only on those columns and go live in production with these settings.

I would like to configure a maximum LIMIT too.

Et voilà!

query getPlayersIdDesc {...}

query getPlayersCreatedAtDesc {...}

Taking only these two columns (id and created_at) I would already be at 4 queries (considering the directions: asc or desc).

But then I would also like to filter by id AND created_at and therefore I need more queries.

For this reason I am proposing you to have a "dynamic" part/function/feature of SuperGraph, maybe even not optimized on the DB (I don't care to wait, as a user I know that this could be a long query on a non-indexed DB, I wait more, I really don't care as long as I can get the data I need).

---

From here: #114 (comment):

The problem I have is to ORDER BY and filter with WHERE in a completely dynamic way.

Example

Take Github's issues page > Sort menu, what I need is very similar:

I can sort results by many columns (even by reactions!).

Another example

In this tutorial here: https://www.howtographql.com/react-urql/7-pagination-and-cache-updates/ they use this query:

export const FEED_QUERY = gql`
  query FeedQuery($first: Int, $skip: Int, $orderBy: LinkOrderByInput) {
    feed(first: $first, skip: $skip, orderBy: $orderBy) {
      count
      ...
    }
  }
`

And they use variables object:

const variables = React.useMemo(() => ({
  skip: isNewPage ? (page - 1) * 10 : 0,
  first: isNewPage ? 10 : 100,
  orderBy: isNewPage ? 'createdAt_DESC' : null
}), [isNewPage, page])

Here the code

Another example

Here a user is trying to solve the same problem using sqlc:

sqlc-dev/sqlc#364 (comment)

I need to select on t1, which the filters are determined from UI, so I need any combination of filters, it means I should support following 9 queries on my back-end.
Users can also sort on any column they wish (ASC/DESC).
Which is 9*4 = 36 different possible queries for a table with 2 fields.

---

As you can see I need to sort the results not only by one column but many.

And creating so many javascript queries is wasteful and counterproductive. Which is the opposite of what SuperGraph would like to achieve: 100% productivity. 😄

[frederikhors]

I had never seen the Github's API: https://developer.github.com/v4/explorer/.

This is exactly what I would like to do with SuperGraph:

THIS IS AMAZING!

[dosco]

FYI you don't have to specify the variable types in the query the types are derived from the database column information internally. Super Graph ignores this type information.

query FeedQuery($first: Int, $skip: Int, $orderBy: LinkOrderByInput) {
  feed(first: $first, skip: $skip, orderBy: $orderBy) {
    count
  }
}

can be

query FeedQuery {
  feed(first: $first, skip: $skip, orderBy: $orderBy) {
    count
  }
}

I'm thinking on how to implement this correctly I agree if it helps the higher purpose "100% productivity" then we should figure out how to implement it.

[frederikhors]

I'm thinking on how to implement this correctly I agree if it helps the higher purpose "100% productivity" then we should figure out how to implement it.

This feature aligns SuperGraph with all the products already established on the market (such as Postgraphile and Hasura).

And it makes it better than the other products both because of its flexible use (both as a library and as a service) and because of Go, an extraordinary language.

Thanks for your constant effort.

I can't wait to try the changes and make this project shine more and more!

😄😄😄

[frederikhors]

@dosco, not to rush you or annoy you: is there any news on this?

Or do I have to write over a hundred client-side queries for my next project?

😄 😄 😄

🥰

[dosco]

Sorry about that will look into some of these issues next week

[dosco]

Closing fixed in f4354f6