Create plugin: CI workflow improvements #890

sunker · 2024-04-24T12:08:47Z

What this PR does / why we need it:
This PR is making a few changes to the e2e related workflow bits:

Use the right snippet in the article that walks through how to run e2e tests in ci
Not doing a sparse checkout of the repo anymore in the e2e test job. Doing a sparse checkout have little benefits, and may cause confusion when contributors wants to go beyond the basic steps in the workflow. See this thread for example.
Adds step that uploads Playwright report to GCS in case org is Grafana. Please note that even though someone were to falsely specify grafana as the org, this step would still not work for them as it requires access to id-tokens that are only available for repos within the grafana org.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

📦 Published PR as canary version: Canary Versions

✨ Test out this PR locally via:

npm install @grafana/create-plugin@4.7.1-canary.890.babb00f.0
npm install @grafana/plugin-e2e@1.1.2-canary.890.babb00f.0
# or 
yarn add @grafana/create-plugin@4.7.1-canary.890.babb00f.0
yarn add @grafana/plugin-e2e@1.1.2-canary.890.babb00f.0

github-actions · 2024-04-24T12:16:18Z

Hello! 👋 This repository uses Auto for releasing packages using PR labels.

✨ This PR can be merged and will trigger a new patch release.
NOTE: When merging a PR with the release label please avoid merging another PR. For further information see here.

mckn

LGTM!

jackw · 2024-05-02T15:56:01Z

packages/create-plugin/templates/github/ci/.github/workflows/ci.yml

+{{#if_eq orgName "grafana"}}
+  id-token: write
+{{/if_eq}}


Are we sure we want to introduce Grafana specific code to create-plugin?

I am kind of leaning towards what Jack says, and try to keep this more generic (not introducing internal plugin development related knowledge in the templates).

I guess I can see your worries. It is easy it becomes a slippery slope with lots of Grafana internal specific logic added to the template. On the other hand, we are creating lots of pluggins internally so it might be worth having some of this stuff added via the template. I'm a bit torn 🙈

so what is the alternative? Another separate workflow for internal devs?

Yep my biggest concern here isn't so much this single addition but that we open the door for a community tool to become peppered with our own internal teams "needs". I would vote that we create separate workflows for our internal teams. From what I've gathered they each have their own ideas and would be good to pool those folk together to create something that they can share internally. I'm pretty sure most of them edit/delete this workflow anyhow.

Additionally this template variable is based on user input. If I want my plugin to be called grafana-awesome-panel I might (through trial and error) come to the conclusion that to do that I need to use grafana as the org name when I scaffoled. Eventually I submit my plugin for review and I'm asked to change the org name unaware that my plugin was scaffolded with specifics used by Grafanas own plugin teams.

Yeah I agree adding Grafana specific code here is not ideal. I'm happy to move this to Braintank or wiki.

jackw · 2024-05-02T15:56:44Z

packages/create-plugin/templates/github/ci/.github/workflows/ci.yml

+
+{{#if_eq orgName "grafana"}}
+      # Repos in the grafana Github org can publish the report to GCS so that they can be browsed without having to download them.
+      # This will make the report public on the Internet, so beware to skip this step if the report contains sensitive information.


What sort of sensitive information could this expose? Could having it as a default be rather risky?

hmm, I'm starting to think that a better approach would be to upload artifacts to github instead (if we want to enable this functionality). Then it would work for all users regarding if it is within the grafana org or not.

uploading to github would not give you a possibility to link to it - you would have to download the report artifact in a zip file and run npx playwright show-report

uploading to GCS gives you a way to share a link with someone who doesnt want to download unzip and run report locally - so kind of a convenience feature.

Example: grafana/grafana#86835 (comment)

What sort of sensitive information could this expose? Could having it as a default be rather risky?

Same as any e2e test report more or less - it contains your browser state, screenshots of the execution, etc. and if you have something sensitive in there it will be in the report. Since it is quite impossible to automatically know about what could theoretically be sensitive on the screen there is no automated solution to this - microsoft/playwright#19992

Hence the guidance in the comments above the action - if you are for some reason using real credentials or have sensitive information in the flow - you have to handle it yourself or not publish it to a public bucket. For most of the plugin devs in grafana org that are just testing with mocks and dummy data it is absolutely fine and even more convenient to share reports via GCS Bucket / URL (same as we do in grafana/grafana).

Yep, and the same security concern applies when uploading artifacts to GH as the report becomes public on the Internet. Playwright docs have example workflows where reports are being uploaded to GH artifacts, bit they don't mention security concerns.

I'm rather safe than sorry - let's comment out the upload artifacts step.

leventebalogh · 2024-05-03T06:41:27Z

packages/create-plugin/templates/github/ci/.github/workflows/ci.yml

+{{#if_eq orgName "grafana"}}
+  id-token: write
+{{/if_eq}}


I am kind of leaning towards what Jack says, and try to keep this more generic (not introducing internal plugin development related knowledge in the templates).

packages/create-plugin/templates/github/ci/.github/workflows/ci.yml

sunker · 2024-05-22T06:10:35Z

Thanks for the review folks. Have removed the grafana specific bits and commented out the report uploading step (see this commit). Let me know what you think.

grafana-plugins-platform-bot · 2024-05-27T05:45:55Z

🚀 PR was released in @grafana/create-plugin@4.10.6 🚀

sunker added 4 commits April 24, 2024 14:03

use the right snippet

1f374b2

remove sparse checkout

1145c2a

use package manager name variable when installing browsers

bd1f989

add publish report step for grafana repos

c394453

sunker added patch Increment the patch version when merged release Create a release when this pr is merged labels Apr 24, 2024

sunker added 3 commits April 24, 2024 14:22

uncomment step

405e4ae

add correct permissions

9a1fb86

exec install

5d6b0d7

sunker marked this pull request as ready for review April 24, 2024 13:05

sunker requested a review from a team as a code owner April 24, 2024 13:05

sunker requested review from mckn and removed request for a team April 24, 2024 13:05

mckn approved these changes Apr 29, 2024

View reviewed changes

jackw reviewed May 2, 2024

View reviewed changes

leventebalogh reviewed May 3, 2024

View reviewed changes

sunker added 2 commits May 22, 2024 08:08

remove grafana specific code

ca48da8

comment out report step

babb00f

sunker mentioned this pull request May 23, 2024

plugin-e2e: fix docs for playwright #916

Closed

sunker merged commit dc2ba55 into main May 27, 2024
13 checks passed

sunker deleted the create-plugin/fix-ci-workflow branch May 27, 2024 05:40

grafana-plugins-platform-bot bot added the released This issue/pull request has been released. label May 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create plugin: CI workflow improvements #890

Create plugin: CI workflow improvements #890

sunker commented Apr 24, 2024 •

edited by grafana-plugins-platform-bot bot

github-actions bot commented Apr 24, 2024 •

edited

mckn left a comment

jackw May 2, 2024

leventebalogh May 3, 2024

mckn May 3, 2024

tolzhabayev May 3, 2024

jackw May 3, 2024

sunker May 21, 2024

jackw May 2, 2024

mckn May 3, 2024

tolzhabayev May 3, 2024 •

edited

tolzhabayev May 3, 2024 •

edited

sunker May 21, 2024

sunker May 21, 2024

leventebalogh May 3, 2024

sunker commented May 22, 2024

grafana-plugins-platform-bot bot commented May 27, 2024

Create plugin: CI workflow improvements #890

Create plugin: CI workflow improvements #890

Conversation

sunker commented Apr 24, 2024 • edited by grafana-plugins-platform-bot bot

github-actions bot commented Apr 24, 2024 • edited

mckn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tolzhabayev May 3, 2024 • edited

Choose a reason for hiding this comment

tolzhabayev May 3, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sunker commented May 22, 2024

grafana-plugins-platform-bot bot commented May 27, 2024

sunker commented Apr 24, 2024 •

edited by grafana-plugins-platform-bot bot

github-actions bot commented Apr 24, 2024 •

edited

tolzhabayev May 3, 2024 •

edited

tolzhabayev May 3, 2024 •

edited