User info fetcher

[nightkr]

Description

This injects a new component into the OPAs, which is used to query a directory backend (such as Keycloak) for additional user information, such as groups and roles. See #477. This is a first step towards implementing #237.

Definition of Done Checklist

• Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
• Please make sure all these things are done and tick the boxes

Author

Edit tasklist title

Beta Give feedback Tasklist Author, more options

• Rename
• Copy markdown
• Delete tasklist

Delete tasklist

Delete tasklist block?

Are you sure? All relationships in this tasklist will be removed.

Cancel Delete

1. Changes are OpenShift compatible

   Changes are OpenShift compatible
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove
2. CRD changes approved

   CRD changes approved
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove
3. Helm chart can be installed and deployed operator works

   Helm chart can be installed and deployed operator works
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove
4. Integration tests passed (for non trivial changes)

   Integration tests passed (for non trivial changes)
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove

Add item to Author

Reviewer

Edit tasklist title

Beta Give feedback Tasklist Reviewer, more options

• Rename
• Copy markdown
• Delete tasklist

Delete tasklist

Delete tasklist block?

Are you sure? All relationships in this tasklist will be removed.

Cancel Delete

1. Code contains useful comments

   Code contains useful comments
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove
2. (Integration-)Test cases added

   (Integration-)Test cases added
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove
3. Documentation added or updated

   Documentation added or updated
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove
4. Changelog updated

   Changelog updated
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove
5. Cargo.toml only contains references to git tags (not specific commits or branches)

   Cargo.toml only contains references to git tags (not specific commits or branches)
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove

Add item to Reviewer

Acceptance

Edit tasklist title

Beta Give feedback Tasklist Acceptance, more options

• Rename
• Copy markdown
• Delete tasklist

Delete tasklist

Delete tasklist block?

Are you sure? All relationships in this tasklist will be removed.

Cancel Delete

1. Feature Tracker has been updated

   Feature Tracker has been updated
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove
2. Proper release label has been added

   Proper release label has been added
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove

Add item to Acceptance

Once the review is done, comment bors r+ (or bors merge) to merge. Further information

[nightkr]

This is very much a prototype, before any actual release we'd need to make the backends configurable, and at least have the scaffolding to add support for more directory backends (such as LDAP).

There's also a corresponding Trino integration, over at https://github.com/stackabletech/trino-operator/tree/spike/user-info-fetcher.

[fhennig]

heya! Looks like this is related to this long open ticket we have: #237

I've got a few questions. Will every OPA instance query the group data on its own? How long is group membership for a user cached?

[nightkr]

Yes, good catch.

[nightkr]

The current spike doesn't do any caching. We have a few avenues for turning it on eventually, either adding it to the group-fetcher sidecar ourselves, or by turning on OPA's built-in http.send cache. The former would give us more control, the latter would be trivial to implement.

Each OPA currently runs independently of each other, so each instance would be run its own independent cache unless we introduce a shared cache component of some kind.

[fhennig]

Got it, thanks 👌

[soenkeliebau]

Just so this isn't lost, @Jimvin expressed a keen interest to have ways of invalidating specific caches as well as @lfrancke who I believe mentioned the ability to evict specific users from all caches.
Probably not for version 0, but worth keeping in mind.

All comments resolved

[sbernauer]

Gone through all changes again (including the new labels mechanism) and LGTM.
Many thanks for everyone that has participated, it was a great group effort!

[sbernauer]

We need the fixes from stackabletech/operator-templating#306, but with these the kuttl test passes again

[sbernauer]

Test https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/opa-operator-it-custom/85 passed 🚀

[NickLarsenNZ]

Tests pass, good to go. Thanks everyone for your contributions