feat: Add support for OPA authorizer #474

sbernauer · 2024-02-20T09:33:46Z

Description

Closes #400

Definition of Done Checklist

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
Please make sure all these things are done and tick the boxes

Author

Give feedback

Changes are OpenShift compatible

Changes are OpenShift compatible
Options
Successfully updated the issue's project

There was an error updating the issue's project
CRD changes approved

CRD changes approved
Options
Successfully updated the issue's project

There was an error updating the issue's project
CRD documentation for all fields, following the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs-style-guide).

CRD documentation for all fields, following the style guide.
Options
Successfully updated the issue's project

There was an error updating the issue's project
Integration tests passed (for non trivial changes)

Integration tests passed (for non trivial changes)
Options
Successfully updated the issue's project

There was an error updating the issue's project
Changes need to be "offline" compatible

Changes need to be "offline" compatible
Options
Successfully updated the issue's project

There was an error updating the issue's project
Options

Reviewer

Give feedback

Code contains useful comments

Code contains useful comments
Options
Successfully updated the issue's project

There was an error updating the issue's project
Code contains useful logging statements

Code contains useful logging statements
Options
Successfully updated the issue's project

There was an error updating the issue's project
(Integration-)Test cases added

(Integration-)Test cases added
Options
Successfully updated the issue's project

There was an error updating the issue's project
Documentation added or updated. Follows the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs-style-guide).

Documentation added or updated. Follows the style guide.
Options
Successfully updated the issue's project

There was an error updating the issue's project
Changelog updated

Changelog updated
Options
Successfully updated the issue's project

There was an error updating the issue's project
Cargo.toml only contains references to git tags (not specific commits or branches)

Cargo.toml only contains references to git tags (not specific commits or branches)
Options
Successfully updated the issue's project

There was an error updating the issue's project
Options

Acceptance

Give feedback

Feature Tracker has been updated

Feature Tracker has been updated
Options
Successfully updated the issue's project

There was an error updating the issue's project
Proper release label has been added

Proper release label has been added
Options
Successfully updated the issue's project

There was an error updating the issue's project
Options

rust/crd/src/lib.rs

…as merged

NickLarsenNZ

LGTM

rust/operator-binary/src/security/opa.rs

tests/templates/kuttl/kerberos/12-rego-rules.yaml

sbernauer · 2024-02-23T13:52:51Z

In terms of CRD change we have two options:

1. Enable authorizer and group-mapper simultaneous

  # Enable authorizer and group-mapper at the same time
  clusterConfig:
    authorization: # optional
      opa: # mandatory
        configMapName: opa # mandatory
        package: hdfs # mandatory

Good, because consistent and users can not enable authZ and forget about group mapping
Good, because rego rules can rely on the groups being propagated (although not recommended)

2. Enable authorizer and group-mapper separately

  clusterConfig:
    authorization: # optional
      opaAuthorization: # mandatory
        configMapName: opa # mandatory
        package: hdfs # mandatory
      opaGroupMapping: # optional
        configMapName: opa # mandatory
        package: hdfs # mandatory

Good, because more flexible, e.g. you can enable AuthZ without group mapping (which you basically would get for free)
Bad, because more complex and error-prone

Originally I was thinking of 2., but now I am in favor of 1., as it's simpler and more consistent and 2. only enables stuff we should probably not support :)
User can always use configOverrides to easily partially enabled stuff when they really really want to.

sbernauer · 2024-02-27T13:34:55Z

Started https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/hdfs-operator-it-custom/128/

adwk67

Just reviewed the docs so far with a few comments. Nit: we use regorule, rego rule and rego-rule here: I don't mind which it is but we should be consistent. The opa docs seem to use two separate words i.e. rego rules.

docs/modules/hdfs/pages/usage-guide/security.adoc

rust/operator-binary/src/security/kerberos.rs

tests/templates/kuttl/kerberos/30-access-hdfs.yaml.j2

tests/templates/kuttl/kerberos/12-rego-rules.yaml.j2

Co-authored-by: Andrew Kenworthy <andrew.kenworthy@stackable.de>

…bled

sbernauer · 2024-02-28T12:34:24Z

@adwk67 feedback should be addressed

sbernauer · 2024-02-28T12:44:07Z

Started https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/hdfs-operator-it-custom/129/

adwk67

LGTM! Can merge when the CI tests are all done.

adwk67

LGTM! Can merge when the CI tests are all done.

sbernauer · 2024-02-28T13:07:52Z

And also a full testsuite run: https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/hdfs-operator-it-custom/130/

sbernauer · 2024-02-28T15:05:01Z

Another full testsuite run, after I increased the resources in stackabletech/ci@40937a9:
https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/hdfs-operator-it-custom/131/

sbernauer · 2024-02-28T16:44:36Z

Full testsuite passed 🚀

sbernauer added 6 commits February 19, 2024 15:04

feat: Add OPA authorizer

fe70f06

feat: First draft of CRD structure

8fe3c18

changelog

a1908a7

let tests pass again

21b0645

re-enable chaos monkey

34206ad

re-add pull policy

19faf6a

sbernauer commented Feb 20, 2024

View reviewed changes

rust/crd/src/lib.rs Outdated Show resolved Hide resolved

sbernauer added 2 commits February 20, 2024 15:21

add some docs

ca55767

charts

cccd8fb

sbernauer marked this pull request as ready for review February 20, 2024 14:27

remove comment

387d56b

sbernauer force-pushed the feat/opa-authorizer branch from 7cb366f to a685927 Compare February 20, 2024 14:40

Move example into file

c2b60aa

sbernauer force-pushed the feat/opa-authorizer branch from a685927 to c2b60aa Compare February 20, 2024 14:43

sbernauer self-assigned this Feb 20, 2024

Remove uneeded HADOOP_CLASSPATH, as stackabletech/docker-images#569 w…

3a80b31

…as merged

NickLarsenNZ self-requested a review February 21, 2024 07:55

NickLarsenNZ previously approved these changes Feb 21, 2024

View reviewed changes

rust/operator-binary/src/security/opa.rs Outdated Show resolved Hide resolved

sbernauer commented Feb 23, 2024

View reviewed changes

tests/templates/kuttl/kerberos/12-rego-rules.yaml Outdated Show resolved Hide resolved

Merge branch 'main' into feat/opa-authorizer

43e1eaa

sbernauer dismissed NickLarsenNZ’s stale review via 43e1eaa February 23, 2024 13:10

sbernauer added 8 commits February 26, 2024 14:04

Add section on username extraction

90b659e

Adopt to CRD decision

09fcee9

Rip out GroupMapper

f16adea

changelog

f97e043

add needed file

4866a8e

round of docs

2fe626d

improve docs

3961cfb

improve docs

a84930e

english linter

402e8c9

add doc comments

7af2ecc

adwk67 self-requested a review February 27, 2024 13:42

adwk67 requested changes Feb 27, 2024

View reviewed changes

adwk67 reviewed Feb 27, 2024

View reviewed changes

rust/operator-binary/src/security/kerberos.rs Show resolved Hide resolved

tests/templates/kuttl/kerberos/30-access-hdfs.yaml.j2 Outdated Show resolved Hide resolved

tests/templates/kuttl/kerberos/12-rego-rules.yaml.j2 Outdated Show resolved Hide resolved

adwk67 reviewed Feb 27, 2024

View reviewed changes

tests/templates/kuttl/kerberos/12-rego-rules.yaml.j2 Outdated Show resolved Hide resolved

adwk67 and others added 7 commits February 27, 2024 16:58

added group entry for check-hdfs

cb31af6

Apply suggestions from code review

d4b70ba

Co-authored-by: Andrew Kenworthy <andrew.kenworthy@stackable.de>

Update docs/modules/hdfs/pages/usage-guide/security.adoc

2d4c24f

Co-authored-by: Andrew Kenworthy <andrew.kenworthy@stackable.de>

comment out

40884e4

Improve rego

dba095f

Set hadoop.user.group.static.mapping.overrides in case OPA is not ena…

bb264da

…bled

Use envsubst instead of kubectl apply inline

efd0cec

sbernauer requested a review from adwk67 February 28, 2024 12:34

adwk67 previously approved these changes Feb 28, 2024

View reviewed changes

Only envsubst $NAMESPACE

8a52deb

sbernauer dismissed adwk67’s stale review via 8a52deb February 28, 2024 12:49

adwk67 approved these changes Feb 28, 2024

View reviewed changes

sbernauer added this pull request to the merge queue Feb 28, 2024

Merged via the queue into main with commit 4505bf7 Feb 28, 2024
30 checks passed

sbernauer deleted the feat/opa-authorizer branch February 28, 2024 16:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add support for OPA authorizer #474

feat: Add support for OPA authorizer #474

sbernauer commented Feb 20, 2024 •

edited

Author

Reviewer

Acceptance

NickLarsenNZ left a comment

sbernauer commented Feb 23, 2024

sbernauer commented Feb 27, 2024

adwk67 left a comment

sbernauer commented Feb 28, 2024

sbernauer commented Feb 28, 2024

adwk67 left a comment

adwk67 left a comment

sbernauer commented Feb 28, 2024

sbernauer commented Feb 28, 2024

sbernauer commented Feb 28, 2024

feat: Add support for OPA authorizer #474

feat: Add support for OPA authorizer #474

Conversation

sbernauer commented Feb 20, 2024 • edited

Description

Definition of Done Checklist

Author

Reviewer

Acceptance

NickLarsenNZ left a comment

Choose a reason for hiding this comment

sbernauer commented Feb 23, 2024

1. Enable authorizer and group-mapper simultaneous

2. Enable authorizer and group-mapper separately

sbernauer commented Feb 27, 2024

adwk67 left a comment

Choose a reason for hiding this comment

sbernauer commented Feb 28, 2024

sbernauer commented Feb 28, 2024

adwk67 left a comment

Choose a reason for hiding this comment

adwk67 left a comment

Choose a reason for hiding this comment

sbernauer commented Feb 28, 2024

sbernauer commented Feb 28, 2024

sbernauer commented Feb 28, 2024

sbernauer commented Feb 20, 2024 •

edited