Plugin Meta Extractor: Initial version

[mckn]

What is this?

The Plugin Meta Extractor (@grafana/plugin-meta-extractor) is a new package that can be used to extract meta-information from the plugin's source code. The goal is to be able to automatically generate static information about the plugin without needing the plugin-dev to care about these (e.g. extension point information).
View README
PR for adding the plugin to our webpack config

What kind of meta-data?

Currently we only extract basic information about the extensions that the plugin registers.

Supported plugin format

The tool currently only recognises extensions that are registered in the following format:

// src/module.ts
export const plugin = new AppPlugin<{}>()
  .setRootPage(App)
  .configureExtensionLink<PluginExtensionPanelContext>({ 
    // ...
  })
  .configureExtensionLink<PluginExtensionPanelContext>({ 
    // ...
  });

How does it work?

It uses the typescript AST to find usages of the functions that register extensions. The package also exposes a Webpack plugin that can be used to generate a meta-info file during the build of the plugin.

How to use it?

CLI

1. Check out the branch
2. Install dependencies - npm install
3. Go to the package - cd packages/plugin-meta-extractor
4. Build the package - npm run build
5. Run the binary - ./dist/bin/run.js ~/my-grafana-plugin/src/module.ts _(prints to the console)
6. Use npm link to depend on the local version of the package while developing

Code

import { extractPluginMeta } from '@grafana/plugin-meta-extractor';

const entryPoint = `${PLUGIN_ROOT}/src/module.ts`;
const pluginMeta = extractPluginMeta(entryPoint); // see the format of the data below

Webpack

import { GrafanaPluginMetaExtractor } from '@grafana/plugin-meta-extractor';

export default {
 // ...
 plugins: [
  // updates the `generated` property of the plugin.json in the dist directory
  new GrafanaPluginMetaExtractor(),
  // ...
  ]
};

What data does it give back?

It always returns with a valid JSON in the following format:

{
  "extensions": [
    {
      "type": "link",
      "extensionPointId": "grafana/dashboard/panel/menu",
      "title": "Open from time series or pie charts (path)",
      "description": "This link will only be visible on time series and pie charts"
    },
    {
      "type": "link",
      "extensionPointId": "grafana/dashboard/panel/menu",
      "title": "Open from time series or pie charts (onClick)",
      "description": "This link will only be visible on time series and pie charts"
    }
  ]
}

Todo

• Use the webpack config entry parameter and handle the different scenarios
  • A single string: entry: "./module.ts"
  • An array of strings: entry: ["./module.ts", "./datasource/module.ts"]
  • An object of entries: entry: { module: "./module.ts" }
  • A promise resolving any of the above
• Test it with all plugin versions (app, panel, datasource)
• Only update the plugin.json if necessary

📦 Published PR as canary version: Canary Versions

✨ Test out this PR locally via:

npm install @grafana/plugin-meta-extractor@0.0.1-canary.805.f998cc3.0
# or 
yarn add @grafana/plugin-meta-extractor@0.0.1-canary.805.f998cc3.0

[github-actions]

Hello! 👋 This repository uses Auto for releasing packages using PR labels.

✨ This PR can be merged and will trigger a new patch release.
NOTE: When merging a PR with the release label please avoid merging another PR. For further information see here.

[jackw]

Great work on this @mckn and @leventebalogh ! 👏

I've left a few comments for consideration.

[sunker]

Think this looks really cool. Awesome work on this!
Some high level questions from me.

In real world scenarios today, can it be the case that an app plugin conditionally configures an extension? For example only registering an extension link in case a certain feature toggle is enabled. In that case, will it be problematic that extensions are extracted at build time rather than runtime?

If I understand correctly, we can traverse the AST also in the case the entry file is a javascript file. So in theory we could support extraction also for javascript plugins? It seems we use the typechecker in a few places in the code. Are we targeting typescript plugins only?

I understand this is still in experimental phase, but I'm curious to know how this fits into the story around registering extensions. Do you think using this tool to extract extensions built time could become a requirement, or would we support extracting them runtime too? How does this relate to the reactive extensions registry feature (that is not merged yet)?

[marefr]

Are you imagine the generated content to go into the plugin.json or how are you imagine this to be consumed within grafana?

[leventebalogh]

@sunker Thanks for the questions!

In real world scenarios today, can it be the case that an app plugin conditionally configures an extension? For example only registering an extension link in case a certain feature toggle is enabled. In that case, will it be problematic that extensions are extracted at build time rather than runtime?

Unfortunately we don't yet support every dynamic pattern with this tool for registering extensions from a plugin (we only support basic chained method calls on the AppPlugin instance at this point) - however we are planning to do so in the future.
The plugin.generated.json file that we generate will only give information to Grafana core about when a plugin should be preloaded. In case a plugin uses some runtime conditions to decide if it registers an extension, the tool should record the static meta-data for the extension regardless, which means that the plugin will always be preloaded for that extension-point, and it can decide during runtime if it registers the extension.

(Something like this can also be done in the chained-method-call pattern: returning undefined from the configure() function of the extension will hide the extension.)

If I understand correctly, we can traverse the AST also in the case the entry file is a javascript file. So in theory we could support extraction also for javascript plugins? It seems we use the typechecker in a few places in the code. Are we targeting typescript plugins only?

I am afraid this will only work properly for plugins written in typescript at this point (module.ts / module.tsx).

I understand this is still in experimental phase, but I'm curious to know how this fits into the story around registering extensions. Do you think using this tool to extract extensions built time could become a requirement, or would we support extracting them runtime too? How does this relate to the reactive extensions registry feature (that is not merged yet)?

This tool is planned to be used seamlessly in our plugin build configuration that is provided by create-plugin, however the meta-info it generates will only be used to decide when a plugin should be preloaded. (Plugins will still be able to dynamically register an extension point, for these scenarios - at least for now - we will always need to preload the plugin.)

[leventebalogh]

@marefr

Are you imagine the generated content to go into the plugin.json or how are you imagine this to be consumed within grafana?

Hmm, after discussing this with @jackw and @mckn we decided to not put it in plugin.json, but to create a separate file - for now called plugin.generated.json -, just to make plugin devs more aware that this is something that shouldn't be edited manually.

[mckn]

I think this looks great! A question regarding the webpack plugin. Will it update the plugin.json in the /dist folder or the one in /src?

[leventebalogh]

I think this looks great! A question regarding the webpack plugin. Will it update the plugin.json in the /dist folder or the one in /src?

That's a good question. It only updates the one in the /dist folder and leaves the one in /src intact. (When running npm run dev it updates the one in the dist folder on every code change.)

[jackw]

Awesome work on this @mckn and @leventebalogh . 🚀

[grafana-plugins-platform-bot]

🚀 PR was released in @grafana/create-plugin@4.9.1, @grafana/plugin-meta-extractor@0.0.1 🚀