Draft DX v2: gql-centric project configuration

[shykes]

This draft explores separating client-side queries and server-side graphql types

[shykes]

ptal @sipsma @aluzzardi

[sipsma]

Like the idea here a lot, if I'm understanding right this is both replacing the current operations.graphql file but also enhancing it further by integrating the notions of action/artifact/service (and the CLI correspondence there).

Among many other things this allows you to skip having to define really simplistic actions in code. E.g. today in todoapp the build and test actions are just passthrough to yarn.script, so defining that via typescript/go/etc. can be pretty verbose relative to just writing a quick query here.

One question: are you imagining that this section would be optional, or is it required to always put something here in order to be invokable?

My main motivation for asking is that I still like the goal of there being a path to using cloak where all the user has to do is write some code in their favorite language, configure an sdk, and now they can invoke their action. The idea being A) completely minimal boilerplate and B) no requirements to know anything about graphql. This wouldn't be the only way to use cloak of course, just one path made available by certain sdks to cater to a particular persona of user.

Obviously that is quite far from the current state of cloak and would entail much more (e.g. code-first schema support), but it would require that this section be optional (defaults would be filled in by cloak using heuristics), so want to check if that's feasible within the context of what you are currently imagining.

[aluzzardi]

GraphQL has native support for this in the form of a file with a list of named queries, e.g.

query deploy {
}

It's not just a convention, it's part of the spec and supported server-side (e.g. you send the entire thing verbatim and pass an operationName to the server to select which one you want to use).

The advantage of using the standard format is we can piggyback on the ecosystem. For instance, graphql-eslint can do live syntax checking, auto-completion and adds "Execute Query" straight from the IDE when you edit a GraphQL operations file.

Another advantage of operations is arguments:

query deploy($name: String = "dagger-io") {
  netlify {
    deploy(name: $name)
  }
}

This would technically allow us to do things like cloak do deploy --name baz (with dagger-io being the default name)

[shykes]

The problem is that, for these pipelines to be useful, they need to be a sequence of queries. For example to take advantage of the FS { save } and Secret { save } tricks. A single query is simply not powerful enough for this IMO.

I agree named queries would be a perfect fit. I just don't know how to overcome the challenge of a single graphql query not being expressive enough to express even the most common actions (eg. todoapp deploy requires yarn build + netlify deploy so it's a non-starter).

[aluzzardi]

I see what this is solving, but have a few reservations.

Syntactically speaking, we're already in muddy waters because our "queries" are somewhat mutations. One could argue that e.g. image(ref: "alpine") { file(path: "/etc/alpine-release") } is still a query since we're not modifying the API state (e.g. this query will not have any side effect on other queries). save however is an API state modification.

But the real kicker is execution. host, yarn and netlify would get executed in parallel, we have no control over that. If we manage to implement our own custom execution model, the user won't have the expectation of specific-order execution (diversion from standard).

A potential solution would be to use mutations rather than queries. Contrary to queries, mutations are executed sequentially. However, this would prevent us from chaining -- we'd have to flatten actions as mutations (e.g. yarnBuild, netlifyDeploy, readFile, etc). The downside is we'd be always sequence operations, no parallelism (e.g. yarnBuild(source: "website") \n yarnBuild(source: "docs") would execute one after the other). Also gets a little verbose (e.g. image(ref: "alpine") { save(id: "alpine") } \n readFile(fs: "alpine", path: "/etc/alpine-release") Worth exploring the idea though.

===

I think in many cases, chaining and a few extra bits would get us 80% there. However it's not without some weirdness either, e.g.:

query deploy {
  yarn(script: "build") {
    netlify {
      deploy(token: "$NETLIFY_TOKEN")
    }
  }
}

In this example, netlify is chained off yarn's FS. And a default secret provider is able to understand $KEY. However this would only work for single FS.

[sipsma]

I think in many cases, chaining and a few extra bits would get us 80% there. However it's not without some weirdness either

Agree that we can take chaining quite far with approaches like that (being able to extend Filesystem from user extensions, secret provider idea, etc.), which is great, but yeah maybe the only way to cover 100% of cases is to either:

1. Support something like the save mutation, which is sort of turning graphql operations into a mini-scripting language (and raises the other questions you mentioned)
2. Say that for cases where chaining doesn't work, you have to use a higher-level language (anything there is an sdk for)

[shykes]

2. Say that for cases where chaining doesn't work, you have to use a higher-level language (anything there is an sdk for)

In the terminology of this draft: you would have to write a custom API type, so that your client-side query becomes a simple pass-through to that type.

[shykes]

Closing in favor or #74 . Thanks for the feedback everyone!