General graphql notebook

[sschneider-ihre-pvs]

Is it possible to have a general graphql notebook not bound to github?

[joyceerhl]

That's possible. The pieces that are GitHub specific right now are:

• Authentication
• The Octokit-based GitHub GraphQL notebook controller/kernel which is used to run GitHub GraphQL queries and mutations

The pieces which can be shared are:

• The notebook serializer
• The syntax highlighting

Do you have a specific GraphQL API in mind that I can look into adding generic support for?

[sschneider-ihre-pvs]

I was more thinking of providing a configuration for an endpoint so that you can run it against any graphql api.

[sschneider-ihre-pvs]

For example when you host your own graphql server.

[acao]

@joyceerhl would love to join forces for this! we are currently rewriting vscode-graphql, and we're about to split up query execution as a seperate extension that builds on vscode-graphql. We are considering using notebook API for this, though we have a lot more to learn about notebook API. As far as I can tell, you authored the vscode notebook API yourself for jupyter notebooks originally ? Amazing stuff! we could even make vscode-graphql optional, if someone wants just highlighting and execution.

thus, a simple endpoint configuration could be sufficient as @sschneider-ihre-pvs suggested, or optionally with vscode-graphql, the graphql-config endpoint could be extracted from configuration as it is now, where users can provide custom headers, etc

we have a few useful utilities that may be useful:

• autocompletion, validation & hover for graphql via our language server
• autocompletion, validation & hover for variables JSON via a utility for generating json schema for variables (it's currently in use for our demo github API explorer)
• a reference client for graphiql via @graphiql/toolkit where the emerging graphql transport specs have been implemented - websocket subscriptions, @defer and @stream, SSE coming soon. We also plan to support automatically serialising alternative message formats.

[joyceerhl]

I'd love to add notebooks support in vscode-graphql :) I didn't write the notebooks API (that would be @rebornix, @roblourens, @connor4312, @jrieken et al). I have experience using it at work and in personal projects.

OTOH, I'm relatively new to GraphQL and did not know that graphql config files were a thing--thanks for all the pointers! I think it would be pretty straightforward to introduce a generic .graphql-nb notebookType and contribute a GraphQL notebook controller which reads the endpoint configuration. I'll take a stab at it hopefully this weekend.

[sschneider-ihre-pvs]

\o/

[acao]

@joyceerhl that sounds awesome! I got to work with @rebornix when creating monaco-graphql 🤓

Keeping it that simple seems like it will be great for you and your users. Apologies for hijacking this feature request 😬. graphql-config is probably a lot more than you need to worry about at this point, let alone our LSP server.

Really excited about this project! Let me know if there’s any way we can help :)

[rebornix]

@acao it was great time pairing with you on monaco-graphql. I have been working on the notebook support for the last two years. It sounds like great idea to have a generic graphql notebook, which has basic execution support (through graphql-config) and intellisense (through graphql-languageserver).

Once we have a generic graphql notebook, the github graphql support can be done through a notebook controller, which offers execution (and authentication under the hood). From users's perspective there is no difference, so this is more or less an implementation detail.

[joyceerhl]

I have a bare-bones prototype of a generic GraphQL notebook which lets you run and save queries based on endpoints in your GraphQL config files:

gqlnb.mp4

In the recording above, I'm creating a single 'GraphQL' notebook controller, and at runtime I'm reusing the
vscode-graphql extension's GraphQLContentProvider to look for endpoints based on the config file in the project directory. In my recording, the config file points to GitHub's GraphQL endpoint like so, allowing me to run a query about the currently authenticated user (myself):

endpoints: {
  default: {
    url: "https://api.github.com/graphql",
    headers: {
      Authorization: `Bearer ${process.env.VSCODE_GITHUB_GRAPHQL_API_TOKEN}`,
    },
  },
},

I prototyped this in vscode-graphql itself so that I could reuse the following logic that already exists for executing queries in the vscode-graphql extension:

• Parsing config files for endpoints
• Extracting literals from a text document
• Network helper logic

And by putting the GitHub schema file in the project root and referencing it in my config file, I also get syntax highlighting and some schema help.

The branch is here (fair warning, this code is not production ready and I'm sharing it just so you have an idea of what it takes to hook the existing execution logic up to a notebooks UI): joyceerhl/vscode-graphql-notebook@1c5e631

Some notes from this prototype:

• The current GraphQL execution experience prompts the user to pick an endpoint if there is more than one configured. I think it could be a nice experience with the notebooks UI to instead register a controller for every endpoint that the user has in their config file, so you aren't prompted to pick the endpoint every time you want to run a query. However, note that the selected endpoint would then be global for the notebook document, so you'd need to either have one notebook per endpoint, or switch controllers a lot. Any thoughts on this @sschneider-ihre-pvs?
• I recall that LSP might be getting first class support for notebooks, and then we could update the GraphQL language server to be notebooks-aware. Maybe @rebornix has more details?
• @acao you mentioned wanting to split execution into its own extension that builds on top of vscode-graphql. Would you prefer that I duplicate all the code (GraphQLContentProvider, NetworkHelper, SourceHelper) that I used for my prototype into a new extension vscode-graphql-notebook?
• The current GraphQL execution experience prompts the user to fill in variables when a query is executed--it could be a nice experience to allow setting variables in some table/tree view UI so you can tweak and rerun them easily
• Because we are using process.env, which isn't available in the browser, this approach will not work in VS Code for the Web :(
• @acao I'm not familiar with "subscription" operations in GraphQL, how does that operation signal that it's "done"? This is important for making sure that we end the notebook cell execution task when all the data that we need to report in the notebook cell outputs has returned (otherwise the notebook cell would continue to execute indefinitely)

[rebornix]

However, note that the selected endpoint would then be global for the notebook document, so you'd need to either have one notebook per endpoint, or switch controllers a lot

We can probably leverage the controller affinity to improve this. We can have a preferred endpoint for a workspace, which can be used as the default controller. When users switch to another controller, it will be memorized for current notebook document.

---

Re Variable.

I'm wondering if it makes sense to explore variable form in outputs to make it more Notebook-y. When users run a query, generate a template variable json, with placeholders, in a pending output. Users can fill them in to continue the execution.

This would also align with the new UX @acao and team proposed in graphiql roadmap, hiding the variable view by default and revealing it only when necessary.

This might require hookups between execution and output renderer. I'd love to help prototype this if you want @joyceerhl .

[acao]

@joyceerhl this branch of yours is amazing! I'm so excited for where we can take things for our users. I forgot to mention that we're migrating vscode-graphql to graphql/graphiql, however this is a really great starting point, and I love where you're taking things!

As far as joining with vscode-graphql or publishing your own for execution that extends it, I'm not sure I'm decided, what do you think makes sense? Having one that's just for the language server gives us a leaner surface area for a reference implementation, but our users have come to expect and enjoy using the operation execution. What you are demonstrating could be so very powerful!

As far as the network helper, I was thinking of replacing that with @graphiql/toolkit createGraphiQLFetcher(), though this client is kind of confusing to implement because of the iterative nature of it. It could use more documentation for standalone usage. It supports subscriptions and defer/stream. It would be better to use than urql or apollo client as before as ideally we don't want to endorse any one client or pattern with graphql org projects.

With subscriptions, you're correct, the intention is usually for long-running data subscriptions, such as with websockets or server sent events. We will need to have the user interrupt the exection

With AsyncIterables such as with @defer and @stream, we may need to iteratively render the JSON output as graphiql does, but there is always a point at which the request completes. The last payload will have hasNext: false in the metadata

[acao]

Hey so, I’m going to be working on finishing the migration of vscode-graphql this weekend!

I think a separate extension is a great idea after some reflection - it could be used on top of vscode-graphql extension or without it, or with an alternative LSP server extension or without it

Would folks here like to schedule a meeting that we can record and we can decide together on how it should work? Or use a working document? Y‘all are pacific coast I assume, and I’m in Berlin so meeting times are tricky

[joyceerhl]

Sorry for the late followup on this--I am more than happy to submit this functionality to vscode-graphql (either shipping it as part of the source or as an extension pack) since the alternative is to duplicate code or expose extension api for discovering configurations and running queries. Whatever you think will be best for users :) I'm available to meet this weekend--I think a decent overlap in timezones would be 8am PST - 3pm PST, which is 5pm - midnight Berlin time.

[sschneider-ihre-pvs]

If you don't mind, I would like to join in, just to see how you guys work :D

[acao]

@joyceerhl apologies myself for the delayed reply, I was in the midst of moving!

That sounds great! I'm terribly sorry as I didn't fully explain the goal for the 1.0.0 of our extension vscode-graphql - it will be only an LSP service plugin, and we will remove the operation execution behaviour we currently have in vscode-graphql, and replace it with a built-in recommendation for your alternative extension!

You're all of course free to add the extension to our monorepo, or maintain it on your own however you please, and we would gladly contribute either way!

I am available to meet today or tomorrow if that is still possible. How does somewhere between 10-2pm PST sound? whatever works best!

[joyceerhl]

Thanks for explaining, that makes sense to me 😊 let’s chat 12.30pm PST / 9.30pm Berlin time on Sunday?

[acao]

sounds great!

[joyceerhl]

@acao let me know what the best way to set up a call with you is 😅 I'm reachable @joyceerhl on twitter if you want to send a dm there

[hinogi]

If you don't mind, I would like to join in, just to see how you guys work :D

Don't forget about me :) I wanna lurk a bit :D

[joyceerhl]

Published a pre-release of a generic GraphQL Notebooks extension which lets you select configured endpoints and execute queries against it: https://marketplace.visualstudio.com/items?itemName=joyceerhl.vscode-graphql-notebook

I created a work backlog tracking what needs to go into a v1 release: https://github.com/users/joyceerhl/projects/1/

I also published a pre-release of this GitHub GraphQL Notebooks extension which drops its notebook serializer in favor of depending on the serializer contributed by vscode-graphql-notebook (which also handles github-graphql-nb files). GitHub GraphQL Notebooks continues to contribute a controller, now for the generic gqlnb notebookType, so any existing users should see no breakage.

[acao]

@joyceerhl hey there! I would still love to set up a call. I am sick with covid now, and on top of that have so many things to attend to before I can come back to OSS work, but hopefully we can set up something in the coming weeks

[joyceerhl]

@acao I’d love to meet—how does next weekend (May 7-8) look for you?

[acao]

@joyceerhl that works great for me!

[joyceerhl]

Great, shall we aim to meet at 10.30am PST / 7.30pm Berlin time on May 7? I think your Twitter DMs are locked so I don’t have a way to send you a Zoom/Google Meet link, but if you don’t mind setting it up, my email is joyceerhl@gmail.com 😊

[sschneider-ihre-pvs]

What about having another channel in discord for this?

[acao]

@joyceerhl invite sent! Also apologies about my twitter account, I disabled it because I was getting too distracted 😂

@sschneider-ihre-pvs I think that would be a great idea!

[sschneider-ihre-pvs]

@joyceerhl invite sent! Also apologies about my twitter account, I disabled it because I was getting too distracted 😂

@sschneider-ihre-pvs I think that would be a great idea!

Communication problem, solved 👍

[acao]

@sschneider-ihre-pvs for now, vscode-graphql channel in the discord should be fine. I will ask fellow mods about this, potentially a channel for general vscode + graphql discussion, for anyone working on vscode extensions related to graphql, with the eventual goal of delivering a language bundle of extensions for the community?

update: the extension pack concept in vscode is what I'm referring to, a family of related, perhaps even some interdependent extensions that can all contribute towards a general experience of a language

[acao]

I've created a dedicated discord channel for this discussion on the graphql discord server!
https://discord.gg/xX9xDBzu4Y