Dynamic schema

[sunli829]

Async-graphql lacks an important feature that most graphql implementations do not. Support both static and dynamic schema, and only with dynamic schema can write something like hasaru.

I have some preliminary ideas about the implementation of this feature. Do you think this feature is important?

@nicolaiunrein @phated

[nicolaiunrein]

I’m relatively new to GraphQL and haven’t encountered dynamic schemas. Could you provide a link? Or an example use-case?

[sunli829]

https://github.com/graphql-go/graphql

Like this, allows change schema at run time.
Not code generation, no type safety, and poor performance.

[nicolaiunrein]

Alright so here is a somewhat incomplete and unordered list of thoughts:

Cons

• The effort of implementing it.
• People might choose a not ideal implementation for their problem because they are unaware of the cost.

Pros

• The task of handling dynamic schemas is expensive in every language and I would always prefer to implement an inefficient task in a fast language than in a slow one.

• dynamic schemas in rust would still outperform implementations in most other languages + give you still a decent amount of typesafety which is important because you might have no control over the schemas you are serving.

• Can be handy for rapid prototyping.

• Not every project needs top performance.

• Sometimes usability is key. Eg. a service implementing a graphql endpoint for maintenance or admins etc.

• If implementing dynamic schemas means decoupling the logic from the codegen then you get even more benefits. It becomes easier to

  • add new features like flatten fields for example.

  • add new crates like an async-graphql-client which depends on the same core types.

• I would:

  • prepare the codebase by extracting the core functionality.

  • implement it on top of the core.

  • make it a feature which is disabled by default.

  • clearly state in the docs that static schemas. are to be preferred over dynamic schemas where ever possible.

  • Show benchmarks of dynamic vs static.

[sunli829]

I tried it out yesterday, but it didn't work😂, and I'm still thinking about a better way to do it. The main reason is that rust does not support generic specialization, solving the problem with newtype makes the library particularly verbose to use.

[nicolaiunrein]

I will definitely 🎉 when specialization lands on stable rust.

[nicolaiunrein]

Have you looked at the Syllogism crate for specialization?

[sunli829]

Thank you for telling me about Syllogism crate. I'll see if it works for me. 😁

[ruseinov]

Yep, dynamic schema would definitely be something of interest.
I'm currently building a "hacky" way to do this as it's not properly supported anywhere, for example in my project I need to have

RecordType {
...
schema: JSONMap
}

and Record {
recordTypeId: ..
data: JSONMap (this data is being validated using a json schema from RecordType)
}

And I'd like to support some sort of querying by these dynamic fields. At the moment I'm looking at custom implementation for this as this is tricky and not really supported by the graphql spec, considering that in the example above each RecordType might have a different schema.

But, getting back to reality - it would be nice to support dynamic schema!

[sunli829]

Validation the JSONMap you could try async_graphql::InputValueValidator.

[ruseinov]

@sunli829 I'll give it a shot, I can probably integrate that with http://rustless.org/valico/doc/valico/

[sunli829]

This should solve the problem perfectly.😁

[ruseinov]

@sunli829 yeah, except I need to pass said schema to the validator.
That would be possible if the context got passed to the validator as well, so that I could go to the database and check, otherwise I can't see how to implement this. But it's alright if I just have to check within the mutation body.

[sunli829]

If you need a context, the Validator can't do that right now, because it's in the query validation phase, and it hasn't created a context yet.😁

[sunli829]

I think we should probably add an execution phase validator called Guard.

[ruseinov]

I think we should probably add an execution phase validator called Guard.

Yeah, that would be awesome.

If you need a context, the Validator can't do that right now, because it's in the query validation phase, and it hasn't created a context yet.😁

Yeah, I have figured as much.

[ruseinov]

@sunli829 should I add a ticket for Guard?
I'd contribute it myself, but so far a bit swamped with other coding duties :/

[sunli829]

This should be finished by tomorrow. I have been trying other improvements for the last two days.

[ruseinov]

Curious to see what you came up with for other issues we had!

[sunli829]

In fact, I'm still thinking about how to use Guard, because even without this feature, a single line of code can solve the problem.

#[field(guard="require_auth")
async fn value(&self) -> i32 {...}

async fn value(&self, ctx: &Context<'_>) -> FieldResult<i32> {
   require_auth(ctx)?;
}

The guard attribute doesn't offer any convenience, and I'm still wondering if this feature is necessary.

[sunli829]

Curious to see what you came up with for other issues we had!

I tried to optimize the way the interface is defined, but I was not satisfied with either approach.

[ruseinov]

In fact, I'm still thinking about how to use Guard, because even without this feature, a single line of code can solve the problem.

#[field(guard="require_auth")
async fn value(&self) -> i32 {...}

async fn value(&self, ctx: &Context<'_>) -> FieldResult<i32> {
   require_auth(ctx)?;
}

The guard attribute doesn't offer any convenience, and I'm still wondering if this feature is necessary.

That actually seems very nice to me, because then we could apply several guards as well, and that's several method calls.
If we could also specify params for the guards - even better. For example, field(guard="require_auth(role=ADMIN)") or similar.

Take a look at some of the features of https://docs.nestjs.com/graphql/tooling for inspiration, they have some powerful annotations in place that make working with graphql a blast!

[sunli829]

Thank you, that gave me some inspiration, and I'm working on that right now.

[sunli829]

One important feature I overlooked is that guard can be applied to the SimpleObject.

[ruseinov]

One important feature I overlooked is that guard can be applied to the SimpleObject.

Yeah, I mean there are plenty of nice looking features there, but not all of them can be implemented in rust the same way, as it's not the same.

For example, there is an ability to annotate methods and types so that they end up being parts of the query/mutation/subscription without actually being one object AFAIK. That allows for splitting the schema by functionality, we could kind of mimic that by having a composite struct I suppose.

[sunli829]

I created a new issue #43, adding this feature was more difficult than I expected, I needed to modify some code generation, and it probably took a little longer to complete.

[chipsenkbeil]

@sunli829, what became of this issue? I wasn't able to follow along properly to understand if this was implemented. There was mention of passing new schemas to validators and not having a context available. Then there was discussion about a guard and it looks like that was implemented, although I don't recall seeing guards mentioned in the book.

A use case I have in mind is enabling users of vimwiki-server to define custom logic that can leverage the data models I've built for the vimwiki language and respond to some custom query using an embedded language like Lua that connects to the Rust server.

My only thought right now without some way to define a custom schema alongside my static schema is through some general-purpose query:

{
    lua_query(key: "some identifier", input: "some input") {
        key
        output
    }
}

[D1plo1d]

Am I right in thinking that dynamic schemas would also be needed to implement an Apollo Gateway replacement in rust?

[sunli829]

Am I right in thinking that dynamic schemas would also be needed to implement an Apollo Gateway replacement in rust?

Yes, you are absolutely correct. 😁

[Visic]

Dynamic schemas would be a big win for my production work. I have/am developing a system for the company I work for which allows us to add/remove/change our systems behavior at runtime using a series of configuration files, part of which is being able to manipulate our webserver/graphql interface. Right now I have to use a super generic interface that works for all situations but isn't ideal due to how vague it is.

[Bjohnson131]

Hey @sunli829 you closed this as completed. was there a PR for this feature that I can look at?

[aschaeffer]

@Bjohnson131 Please take a look at https://github.com/apollographql/apollo-rs

Currently, if you want to implement a dynamic schema, you have to implement it by yourself. Use apollo-encoder to generate a schema SDL and apollo-parser to parse GraphQL requests.

[sunli829]

I'm working on this, it's going well, and a preview will be released this week. 🙂

[sunli829]

I underestimated the amount of work, but it should be done this week.

[Bjohnson131]

This is so big! If you get a preview, let me know! I'll be following!

[sunli829]

I opened a PR to track progress.

[xoac]

This issue can be probably closed