Roadmap: Grafana plugins platform

[mckn]

We would like to make the experience of extending Grafana when building a plugin as smooth as butter. Hopefully you have a bunch of ideas that we can improve.

Note: this issue is targeting the APIs being used to build plugins/apps to extend Grafana.

What to do:

• Add a comment explaining the feature you want to be added to the plugins APIs.
• Add a comment including a link to an existing issue, open or closed.
• Thumbs up comments 👍 and we will try to use this as an indicator of demand (prefer adding a 👍 to an existing comment rather than adding a new comment).
• It can be difficult to prioritize work for a project like Grafana which can be used in many different ways, it's hard to tell how common some might be or how popular a feature request is, your feedback would be greatly appreciated!

One request, we can keep the thread easier to read if we don't have too many discussions, prefer creating a separate issue or use the existing issue to have specific conversations.

Things we already have planned:

Note: this is a living document and is prone to change.

Grafana 8.3:

• Automatically suggest compatible plugin version that will work with your current Grafana version when managing plugins.
• Display a warning in plugins catalog if you are trying to uninstall a plugin that is being used and configured as a datasource.
• Make is possible to easily publish/subscribe to appEvents in a plugin.

Grafana 8.4:

• Support running plugins remotely so you can run one instance of a plugin and use it for multiple Grafana instances.

Later version of Grafana:

• Better manage changes between Grafana version so that we always strive for backward compatibility.
• Enhance plugin developer tooling so it is easier to use and debug across all major OS.
• Improve the CI pipeline support for helping internal and external plugin developers building awesome plugins.
• Add APIs to make it easy to support internalization in plugins.
• Introduce a developer portal with documentation, migration guides etc. to make it easier for developers to understand how to extend Grafana.

[joshhunt]

One thing that could be interesting to look is whether eslint could statically identify anti-patterns and provide warnings in your editor. Grafana could develop a custom eslint plugin (see https://github.com/vercel/next.js/blob/canary/packages/eslint-plugin-next/lib/index.js) and have it a part of the default template.

[srclosson]

Setting expectations with developers that "Your plugin must at least support the last 1 year of grafana releases" (or something to that effect) and then providing tools to ensure developers can easily tests their plugins for compatibility would be stellar.

[joshhunt]

I would also love to see a clean up of the DataSourceApi / DataSourceWithBackend typescript interfaces with some stricter types for the methods, and simplifying the obligations of developers in terms of methods to implement, or the shape of the data to return

e.g. applyTemplateVariables is basically typed to return any, but it apparently looks like it needs to return properties like refId: target.refId or , I'm unsure why. IMHO Grafana should just handle that for me, and I shouldn't have the opportunity to screw that up by omitting the field Having better types for these functions. See https://github.com/grafana/grafana/blob/83b6df7/public/app/plugins/datasource/cloud-monitoring/datasource.ts#L106-L116

[joshhunt]

It could be interesting to look into adding "Auth Providers" as a core (plugin?) concept to Grafana. This would be a way to centralise authentication implementations in the code, and help users manage data source credentials in Grafana.

Hypothetically the user can go to a "Credentials" settings page in Grafana and configure some Azure (for example) credentials. Then when I'm setting up Azure Monitor, ADX, and any other data sources rather than entering in credentials specific to that data source, I could select from the available credentials I have set up. UX wise, you should have the ability to create new credentials inline, if you don't have any yet.

Data sources could express (through plugin ids??) which auth types they're compatible with (or require), and the grafana http stack could transparently apply those auth middleware to requests made with it. Once a data source has opted into receiving/using Auth Providers, it should not have to do anything else to use this, especially create any UI - Grafana can render the chose/create credentials UI itself on the data source config page.

I imagine this would be an extension of the current core auth options available through DataSourceHTTPSettings, and could solve some UX problems that's starting to show as it supports more auth types optionally for some data sources.

A major consideration that could prevent this from being a good idea is that (by design!!!) currently you cannot reuse secrets between multiple data sources. Once you've entered a secret into a data source, you cannot reuse that secret to use elsewhere. I'm unsure whether its safe or not to "trust" admins to be able to reuse secrets like this.

[richardqlam]

For Grafana Cloud, we often have overlapping concepts that exist both within Grafana Core and within the Cloud Portal. For example, API keys exist both at the Grafana level and at the Portal level. As we bring more of these concepts out of the Portal directly into Grafana, we want to be able to consolidate these concepts -- users shouldn't have to distinguish the difference between two menu options that say the same thing.

It would be great, then, if there was:

• The ability to hide specific Core sidenav buttons or sub-menu options, altogether
• The ability to add new sub-menu options to existing Core sidenav buttons
• The ability to modify existing Core pages so that new tabs and pages cab be added AND existing ones can be removed. For example, add a new page called "cloud integrations" to the existing Core Configuration page, or prevent the existing "Teams" pages from appearing/being accessed.

[wz2b]

I would like to request the ability to have 3rd parties sign plug-ins. I raised this issue in #26287 but it didn't stick. My reasons for wanting this are:

• Verify that the author's identity via the standard SSL chain of trust system that everybody already understands
• Verify that a plugin hasn't been tampered with either before you got it or after it was installed

The way I want to do this is to have the ability to use the chain of trust by either:

• signing plug-ins with a real code signing certificate issued by Comodo, Sectigo, etc. i.e. root is in /etc/ssl/certs
• allow administrators to install their own root certificates, mainly for self-signing
• Require the author to have a code-signing certificate as established by the x.509 usage extension

I don't love is the hard coded method of validating the plug-in signature being used today.

The certificate keystore doesn't have to be fancy. I would make a search path or list that you can specify in grafana.ini, for example:

[plugins]
plugin_signing_certificates=/etc/ssl/certs/*.pem;some_self_signing_ca.pem;something_else.pem

then, in addition to checking the hard-coded internal grafana certificate the user could specify others. I would default to checking the system keystore (usually /etc/ssl/certs if you're on something linux-y). Note I also added the ability to include specific additional certificates.

How this would work is:

• If someone has a real code-signing certificate, then it would just work with one caveat: I would like grafana having a warning at the time of installation that says something like "Warning, you are installing a plugin not developed by Grafana Labs"
• If someone doesn't have one, then they generate their own "root" CA and install that as above

I think what this does is allows you to have 3rd party plugins where at a minimum you can identify the source of the plugin - you know where it came from and you know nobody else modified it, because it was signed by me, and Sectigo (or whomever) verifies that I'm really me.

The instructions to create a self-signed root and certificate are easy openssl commands, or it could easily be added to the mage system - if we can get consensus on this, I will volunteer to do that.

References that are relevant to this:

• https://grafana.com/docs/grafana/latest/developers/plugins/sign-a-plugin/ describes a signing process. My only beef with it is that the manifest file feels a little non-standard to me ... I'd rather the manifest be just the content and have the signature in an external file. I'd also prefer to be able to sign with mage but that's not a strong requirement.
• https://grafana.com/docs/grafana/latest/plugins/plugin-signatures/ describes a process where, if I interpret it correctly, every plugin has to be signed by grafana? What I feel is a gap is the ability to have a signed plugin that's outside of the grafana catalog, without having to throw away the signature mechanism entirely. This way you still get 1) traceability via the chain of trust 2) the plugin is tamper-proof

If we can get some consensus on how to proceed I will contribute code to this, but I don't want to do it if it's not likely to be merged, so let's discuss.

-- Update --
I did a little reading on this and I get now that it's PGP based rather than x.509. Maybe Grafana intends to keep it that way. I personally think I would opt for the X.509 and use the standard CA chain of trust. But I understand that might be a problem for some people because X.509 code signing certificates cost money and, at this point in time, there don't seem to be any free alternatives. Within the PGP world things don't change from what I said here, except that instead of relying on chain-of-trust you really need to trust my PGP certificate, which I think is riskier. I'm wondering if Grafana has the appetite to support both approaches.

[unknowndevQwQ]

Setting expectations with developers that "Your plugin must at least support the last 1 year of grafana releases" (or something to that effect) and then providing tools to ensure developers can easily tests their plugins for compatibility would be stellar.

I think it's more intuitive to follow the semantic version to ensure reliability than to follow the time

[torkelo]

@richardqlam why hide Teams? it's a pretty core feature for organizing dashboards & folder

[torkelo]

The ability to hide specific Core sidenav buttons or sub-menu options, altogether
The ability to add new sub-menu options to existing Core sidenav buttons
The ability to modify existing Core pages so that new tabs and pages can be added AND existing ones can be removed. For example, add a new page called "cloud integrations" to the existing Core Configuration page, or prevent the existing "Teams" pages from appearing/being accessed.

The easiest way to accomplish this might be through a custom cloud build of enterprise, enterprise can modify the nav tree and add core pages. Having a plugin be able to do that could be tricky. Another option would be a very complex configuration scheme where the nav tree is defined in a config file but that has a lot of challenges as well.

[richardqlam]

@richardqlam why hide Teams? it's a pretty core feature for organizing dashboards & folder

This was just a random example, and a bad one. A more appropriate one may be "Users" since it is managed separately in the Cloud Portal. But either way, was just listing an example not an actual request.

[Willena]

Since the introduction of transformations, I've seen that the team keeps adding more transforms to Grafana (it is great !).
I would love to see a new plugin API dedicated to transformations. So that transformations that are not considered important for Grafana Core could still be added using a plugin.

Any thoughts on that ?

Edit by @mckn: Started a discussion around this here

[yosiasz]

Make the following a bit more smart so that it lists out all teams and buckets (like variables work for devs 😉 )
Sometimes you forget team names and bucket names and have to leave this config to hunt down exact spelling of names.
Thanks!

[bobfrankly]

I'd really like to see a way to add our own valueFormatters via plugins so we don't have to bug the core devs over stuff like #37838 . We'd just be able to write a new currency formatter that doesn't compress the values by omitting the trailing zeros, or whatever it happens to be that we need, and sharing if applicable.

More on this:#37978 .

[mckn]

Make the following a bit more smart so that it lists out all teams and buckets (like variables work for devs 😉 )
Sometimes you forget team names and bucket names and have to leave this config to hunt down exact spelling of names.
Thanks!

This I great and I have forwarded it to the people working with the influx datasource plugin.

[tolzhabayev]

We are closing this issue as we are switching to a new format of transparency around plugin platform development.

You can always submit a feature request as well as upvote existing ones here: https://github.com/grafana/grafana/discussions/categories/plugins
And see our public roadmap here: https://github.com/orgs/grafana/projects/76/views/9