Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add database security monitoring dashboards #175

Merged
merged 6 commits into from
Sep 5, 2022

Conversation

yota-p
Copy link
Contributor

@yota-p yota-p commented Aug 23, 2022

In this PR, I would like to add 2 dashboards for monitoring database security in Amazon Redshift.

Background

Currently Redshift users need to query system tables to monitor security features (e.g. user, schema, table, privileges) configured in Redshift. This requires querying multiple system tables and writing SQL.
By using this dashboard, user will be able to navigate these information in Redshift quickly without writing SQL.

Amazon Redshift Identities and Objects dashboard

This dashboard will visualize identity (user/role/group) and database objects (schema/table etc).

p1
p2

Amazon Redshift Privileges dashboard

This dashboard will visualize privileges (who have what access).

p3

@yota-p yota-p requested a review from a team as a code owner August 23, 2022 12:06
@yota-p yota-p requested review from sunker and kevinwcyu and removed request for a team August 23, 2022 12:07
@CLAassistant
Copy link

CLAassistant commented Aug 23, 2022

CLA assistant check
All committers have signed the CLA.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Yota Hamaoka seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@yota-p
Copy link
Contributor Author

yota-p commented Aug 24, 2022

Hi team,
There are workflows failing with Error: Input required and not supplied: token.
Would someone from code owner help me resolve this, or can we just ignore?

Workflows are failing at:

token: ${{secrets.GH_BOT_ACCESS_TOKEN}}

token: ${{secrets.GH_BOT_ACCESS_TOKEN}}

As far as I searched, this error is happening because PR from forked repository cannot reference secrets.GH_BOT_ACCESS_TOKEN due to GitHub's security feature (same as #125 ).
There's a workaround like using pull_request_target described in this article but I would like to hear your thoughts.
https://securitylab.github.com/research/github-actions-preventing-pwn-requests/

@fridgepoet
Copy link
Member

Thanks for looking into that; I don't think you need to worry about those checks for now.

I think you'll need to add these dashboards to the plugin.json so they will be suggested on the Dashboards page. Like:

"includes": [
    {
      "type": "dashboard",
      "name": "Redshift Monitoring",
      "path": "dashboards/redshift-monitoring.json"
    },
    {
      "type": "dashboard",
      "name": "Redshift Privileges",
      "path": "dashboards/redshift-privileges.json"
    },
    {
      "type": "dashboard",
      "name": "Redshift Identities and Objects",
      "path": "dashboards/redshift-identities-and-objects.json"
    }
  ],

@sunker
Copy link
Collaborator

sunker commented Aug 25, 2022

Thanks for contributing @yota-p!

When I test the dashboards, the query editors aren't loading properly.
Screenshot from 2022-08-25 11-41-54

To work around this, you may need to tips the option Export for sharing externally when exporting your dashboard.
Screenshot from 2022-08-25 11-38-54

@yota-p
Copy link
Contributor Author

yota-p commented Aug 25, 2022

@fridgepoet @sunker Thank you for the response. I'll check plugin.json and dashboard definition file.
I need some time to build the plugin on my side and test it. WIll let you know once ready.

@yota-p
Copy link
Contributor Author

yota-p commented Aug 26, 2022

@fridgepoet
Updated plugin.json. I built the plugin locally and confirmed the dashboards can be imported & are working.
ScreenShot 2022-08-26 at 21 37 29

@yota-p
Copy link
Contributor Author

yota-p commented Aug 26, 2022

@sunker
Interesting, I didn't see the same error. The dashboards are loading properly on when I tested.
Would you try again with the latest commit?

ScreenShot 2022-08-26 at 21 43 40

I used the option Export for sharing externally. FYI, I also made changes to the JSON manually since exported file still contains environment specific info (e.g. datasource uid). I can still load the dashboard properly so I think it's not the cause here. Changes I made are:

  • Removed __inputs, __elements, __requires, iteration
  • Replaced “datasource”: {“type”: xxx, “uid”: xxx} to "datasource": "${ds}"
  • Replaced values of templating.list[0].current.text, templating.list[0].current.value to default

@kevinwcyu
Copy link
Contributor

Hi @yota-p, could you change the following in the dashboards

"datasource": "${ds}"

to the following

"datasource": {
  "type": "grafana-redshift-datasource",
  "uid": "${ds}"
},

I suspect that's what's causing @sunker to not be able to see the correct query editor. Just a bit of background. We changed the shape of the datasource some time in v8.3.x of Grafana I believe. It used to be a string, but now it's an object with the type and uid property. I think what's happening is when we read the datasource property after importing the dashboard, it's setting the datasource as the whatever is set as the default datasource for the Grafana instance.

I was seeing the same issue as @sunker when I changed my default datasource to a different plugin, but using the new format for the datasource property fixed it.

@yota-p
Copy link
Contributor Author

yota-p commented Aug 30, 2022

@kevinwcyu Thank you for the suggestion. I changed the format for datasource property and confirmed it's loading properly on my end (I'm using latest v9.1.1). Would you review?

Copy link
Contributor

@kevinwcyu kevinwcyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution @yota-p. It looks good from my end.

@sunker, could you check to see if the query editor is loading for you when you get a moment?

Copy link
Collaborator

@sunker sunker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thanks for contributing @yota-p.

@sunker sunker merged commit a1c650d into grafana:main Sep 5, 2022
@yota-p
Copy link
Contributor Author

yota-p commented Sep 6, 2022

Great, thank you team!

@yota-p
Copy link
Contributor Author

yota-p commented Sep 20, 2022

Hi @sunker @kevinwcyu ,
Do you have a plan when to release the next version?

This feature is merged but not released yet. If possible, I want this feature to be released by the end of September (not a hard limit tho).

@sunker
Copy link
Collaborator

sunker commented Sep 22, 2022

Yes it was just released in 1.2.0. Thanks again for contributing!

@yota-p
Copy link
Contributor Author

yota-p commented Sep 22, 2022

Looks good. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

6 participants