Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy userinfofetcher regorules #580

Draft
wants to merge 9 commits into
base: spike/bundle-builder-v2
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ All notable changes to this project will be documented in this file.
### Added

- Support enabling decision logs ([#555]).
- Added regorule library for accessing user-info-fetcher ([#580]).

### Changed

Expand All @@ -20,6 +21,7 @@ All notable changes to this project will be documented in this file.

[#555]: https://github.com/stackabletech/opa-operator/pull/555
[#578]: https://github.com/stackabletech/opa-operator/pull/578
[#580]: https://github.com/stackabletech/opa-operator/pull/580
[#583]: https://github.com/stackabletech/opa-operator/pull/583

## [24.3.0] - 2024-03-20
Expand Down
5 changes: 5 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 28 additions & 0 deletions Cargo.nix

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

77 changes: 16 additions & 61 deletions docs/modules/opa/pages/usage-guide/user-info-fetcher.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -59,81 +59,36 @@

NOTE: The OAuth2 Client in Keycloak must be given the `view-users` _Service Account Role_ for the realm that the users are in.

// TODO: Document how to use it in OPA regorules, e.g. to authorize based on group membership
== Example rego rule

[NOTE]
.About unencrypted HTTP
====
The User info fetcher serves endpoints over clear-text HTTP.

It is intended to only be accessed from the OPA Server via _localhost_ and to not be exposed outside of the Pod.
====

[source,rego]
----
package test # <1>

# Define a function to lookup by username
userInfoByUsername(username) := http.send({
"method": "POST",
"url": "http://127.0.0.1:9476/user",
"body": {"username": username}, <2>
"headers": {"Content-Type": "application/json"},
"raise_error": true
}).body

# Define a function to lookup by a stable identifier
userInfoById(id) := http.send({
"method": "POST",
"url": "http://127.0.0.1:9476/user",
"body": {"id": id}, <3>
"headers": {"Content-Type": "application/json"},
"raise_error": true
}).body

currentUserInfoByUsername := userInfoByUsername(input.username)
currentUserInfoById := userInfoById(input.id)
----

<1> The package name is important in the OPA URL used by the product.
<2> Lookup by username
<3> Lookup by id

For more information on the request and response payloads, see <<_user_info_fetcher_api>>

== User info fetcher API

HTTP Post Requests must be sent to the `/user` endpoint with the following header set: `Content-Type: application/json`.
User information can be retrieved from regorules using the functions `userInfoByUsername(username)` and `userInfoById(id)` in `data.stackable.opa.userinfo.v1`.

You can either lookup the user info by stable identifier:
An example of the returned structure:

[source,json]
----
{
"id": "af07f12c-a2db-40a7-93e0-874537bdf3f5",
"username": "alice",
"groups": [
"/superset-admin"
],
"customAttributes": {}
}
----

or by the username:
For example, the following rule will allow access for users in the `/admin` group:

[source,json]
----
{
"username": "alice",
}
[source,rego]
----
package test

If the user is found, the following response structure will be returned:
import rego.v1

[source,json]
----
{
"id": "af07f12c-a2db-40a7-93e0-874537bdf3f5",
"username": "alice",
"groups": [
"/superset-admin"
],
"customAttributes": {}
default allow := false

allow if {
user := data.stackable.opa.userinfo.v1.userInfoById(input.userId)
"/admin" in user.groups

Check notice on line 92 in docs/modules/opa/pages/usage-guide/user-info-fetcher.adoc

View workflow job for this annotation

GitHub Actions / LanguageTool

[LanguageTool] docs/modules/opa/pages/usage-guide/user-info-fetcher.adoc#L92

If a new sentence starts here, add a space and start with an uppercase letter. (LC_AFTER_PERIOD[1]) Suggestions: ` Groups`, ` groups` Rule: https://community.languagetool.org/rule/show/LC_AFTER_PERIOD?lang=en-US&subId=1 Category: CASING
Raw output
docs/modules/opa/pages/usage-guide/user-info-fetcher.adoc:92:21: If a new sentence starts here, add a space and start with an uppercase letter. (LC_AFTER_PERIOD[1])
 Suggestions: ` Groups`, ` groups`
 Rule: https://community.languagetool.org/rule/show/LC_AFTER_PERIOD?lang=en-US&subId=1
 Category: CASING
}
----
2 changes: 1 addition & 1 deletion docs/modules/opa/partials/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* xref:opa:usage-guide/index.adoc[]
** xref:opa:usage-guide/listenerclass.adoc[]
** xref:opa:usage-guide/policies.adoc[]
// ** xref:opa:usage-guide/user-info-fetcher.adoc[] Not yet ready for public consumption
** xref:opa:usage-guide/user-info-fetcher.adoc[]
** xref:opa:usage-guide/resources.adoc[]
** xref:opa:usage-guide/logging.adoc[]
** xref:opa:usage-guide/monitoring.adoc[]
Expand Down
1 change: 1 addition & 0 deletions rust/bundle-builder/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ flate2.workspace = true
futures.workspace = true
hyper.workspace = true
snafu.workspace = true
stackable-opa-regorule-library = { version = "0.0.0-dev", path = "../regorule-library" }
stackable-operator.workspace = true
tar.workspace = true
tokio.workspace = true
Expand Down
32 changes: 21 additions & 11 deletions rust/bundle-builder/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -181,14 +181,19 @@ enum BundleError {
#[snafu(display("ConfigMap is missing required metadata"))]
ConfigMapMetadataMissing,

#[snafu(display("file {file_name:?} in {config_map} is too large ({file_size} bytes)"))]
#[snafu(display("file {file_path:?} is too large ({file_size} bytes)"))]
FileSizeOverflow {
source: TryFromIntError,
config_map: ObjectRef<ConfigMap>,
file_name: String,
file_path: String,
file_size: usize,
},

#[snafu(display("failed to add static file {file_path:?} to tarball"))]
AddStaticRuleToTarball {
source: std::io::Error,
file_path: String,
},

#[snafu(display("failed to add file {file_name:?} from {config_map} to tarball"))]
AddFileToTarball {
source: std::io::Error,
Expand All @@ -211,20 +216,15 @@ impl BundleError {

async fn build_bundle(store: Store<ConfigMap>) -> Result<Vec<u8>, BundleError> {
use bundle_error::*;
fn file_header(
config_map: &ObjectRef<ConfigMap>,
file_name: &str,
data: &[u8],
) -> Result<tar::Header, BundleError> {
fn file_header(file_path: &str, data: &[u8]) -> Result<tar::Header, BundleError> {
let mut header = tar::Header::new_gnu();
header.set_mode(0o644);
let file_size = data.len();
header.set_size(
file_size
.try_into()
.with_context(|_| FileSizeOverflowSnafu {
config_map: config_map.clone(),
file_name,
file_path,
file_size,
})?,
);
Expand All @@ -237,6 +237,16 @@ async fn build_bundle(store: Store<ConfigMap>) -> Result<Vec<u8>, BundleError> {
let mut tar = tar::Builder::new(GzEncoder::new(Vec::new(), flate2::Compression::default()));
let mut resource_versions = BTreeMap::<String, String>::new();
let mut bundle_file_paths = BTreeSet::<String>::new();

for (file_path, data) in stackable_opa_regorule_library::REGORULES {
let mut header = file_header(file_path, data.as_bytes())?;
tar.append_data(&mut header, file_path, data.as_bytes())
.context(AddStaticRuleToTarballSnafu {
file_path: *file_path,
})?;
bundle_file_paths.insert(file_path.to_string());
}

for cm in store.state() {
let ObjectMeta {
name: Some(cm_ns),
Expand All @@ -249,8 +259,8 @@ async fn build_bundle(store: Store<ConfigMap>) -> Result<Vec<u8>, BundleError> {
};
let cm_ref = ObjectRef::from_obj(&*cm);
for (file_name, data) in cm.data.iter().flatten() {
let mut header = file_header(&cm_ref, file_name, data.as_bytes())?;
let file_path = format!("configmap/{cm_ns}/{cm_name}/{file_name}");
let mut header = file_header(&file_path, data.as_bytes())?;
tar.append_data(&mut header, &file_path, data.as_bytes())
.with_context(|_| AddFileToTarballSnafu {
config_map: cm_ref.clone(),
Expand Down
13 changes: 13 additions & 0 deletions rust/regorule-library/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[package]
name = "stackable-opa-regorule-library"
description = "Contains Stackable's library of common regorules"
version.workspace = true
authors.workspace = true
license.workspace = true
edition.workspace = true
repository.workspace = true
publish = false

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
16 changes: 16 additions & 0 deletions rust/regorule-library/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Stackable library of shared regorules

This contains regorules that are shipped by the Stackable Data Platform (SDP) as libraries to help simplify writing authorization rules.

## What this is not

This library should *not* contain rules that only concern one SDP product. Those are the responsibility of their individual operators.
nightkr marked this conversation as resolved.
Show resolved Hide resolved

## Versioning

All regorules exposed by this library should be versioned, according to Kubernetes conventions.

This version covers _breaking changes to the interface_, not the implementation. If a proposed change breaks existing clients,

Check failure on line 13 in rust/regorule-library/README.md

View workflow job for this annotation

GitHub Actions / markdownlint

[markdownlint] rust/regorule-library/README.md#L13

MD049/emphasis-style Emphasis style [Expected: asterisk; Actual: underscore]
Raw output
rust/regorule-library/README.md:13:21 MD049/emphasis-style Emphasis style [Expected: asterisk; Actual: underscore]

Check failure on line 13 in rust/regorule-library/README.md

View workflow job for this annotation

GitHub Actions / markdownlint

[markdownlint] rust/regorule-library/README.md#L13

MD049/emphasis-style Emphasis style [Expected: asterisk; Actual: underscore]
Raw output
rust/regorule-library/README.md:13:55 MD049/emphasis-style Emphasis style [Expected: asterisk; Actual: underscore]
add a new version. Otherwise, change the latest version inline.

Ideally, old versions should be implemented on top of newer versions, rather than carry independent implementations.
NickLarsenNZ marked this conversation as resolved.
Show resolved Hide resolved
4 changes: 4 additions & 0 deletions rust/regorule-library/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
pub const REGORULES: &[(&str, &str)] = &[(
"stackable/opa/userinfo/v1.rego",
include_str!("userinfo/v1.rego"),
)];
23 changes: 23 additions & 0 deletions rust/regorule-library/src/userinfo/v1.rego
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
package stackable.opa.userinfo.v1

# Lookup by (human-readable) username
userInfoByUsername(username) := http.send({
nightkr marked this conversation as resolved.
Show resolved Hide resolved
"method": "POST",
"url": "http://127.0.0.1:9476/user",
"body": {"username": username},
"headers": {"Content-Type": "application/json"},
"raise_error": true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wanted to mention that we other examples where we assert on the return code and don't use raise_error:
https://github.com/stackabletech/demos/blob/61601f8ed879c4e8ba0766ec0fe0320ee959bdb7/stacks/end-to-end-security/trino-policies.yaml#L176-L184

I don't have any preference, but other ones might have and we should agree on one.

}).body

# Lookup by stable user identifier
userInfoById(id) := http.send({
"method": "POST",
"url": "http://127.0.0.1:9476/user",
"body": {"id": id},
"headers": {"Content-Type": "application/json"},
"raise_error": true
}).body

# Lookup by context
currentUserInfoByUsername := userInfoByUsername(input.username)
currentUserInfoById := userInfoById(input.id)
16 changes: 0 additions & 16 deletions tests/templates/kuttl/aas-user-info/10-install-opa.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,6 @@ commands:
- script: |
kubectl apply -n $NAMESPACE -f - <<EOF
---
apiVersion: v1
kind: ConfigMap
metadata:
name: test
labels:
opa.stackable.tech/bundle: "true"
data:
test.rego: |
package test

userInfoByUsername(username) := http.send({"method": "POST", "url": "http://127.0.0.1:9476/user", "body": {"username": username}, "headers": {"Content-Type": "application/json"}, "raise_error": true}).body
userInfoById(id) := http.send({"method": "POST", "url": "http://127.0.0.1:9476/user", "body": {"id": id}, "headers": {"Content-Type": "application/json"}, "raise_error": true}).body

currentUserInfoByUsername := userInfoByUsername(input.username)
currentUserInfoById := userInfoById(input.id)
---
apiVersion: opa.stackable.tech/v1alpha1
kind: OpaCluster
metadata:
Expand Down
2 changes: 1 addition & 1 deletion tests/templates/kuttl/aas-user-info/30-assert.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ kind: TestAssert
metadata:
name: test-regorule
commands:
- script: kubectl exec -n $NAMESPACE test-regorule-0 -- python /tmp/test-regorule.py -u 'http://test-opa-server-default:8081/v1/data/test'
- script: kubectl exec -n $NAMESPACE test-regorule-0 -- python /tmp/test-regorule.py -u 'http://test-opa-server-default:8081/v1/data/stackable/opa/userinfo/v1'
16 changes: 0 additions & 16 deletions tests/templates/kuttl/keycloak-user-info/10-install-opa.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,6 @@ commands:
- script: |
kubectl apply -n $NAMESPACE -f - <<EOF
---
apiVersion: v1
kind: ConfigMap
metadata:
name: test
labels:
opa.stackable.tech/bundle: "true"
data:
test.rego: |
package test

userInfoByUsername(username) := http.send({"method": "POST", "url": "http://127.0.0.1:9476/user", "body": {"username": username}, "headers": {"Content-Type": "application/json"}, "raise_error": true}).body
userInfoById(id) := http.send({"method": "POST", "url": "http://127.0.0.1:9476/user", "body": {"id": id}, "headers": {"Content-Type": "application/json"}, "raise_error": true}).body

currentUserInfoByUsername := userInfoByUsername(input.username)
currentUserInfoById := userInfoById(input.id)
---
apiVersion: opa.stackable.tech/v1alpha1
kind: OpaCluster
metadata:
Expand Down
2 changes: 1 addition & 1 deletion tests/templates/kuttl/keycloak-user-info/30-assert.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ kind: TestAssert
metadata:
name: test-regorule
commands:
- script: kubectl exec -n $NAMESPACE test-regorule-0 -- python /tmp/test-regorule.py -u 'http://test-opa-server-default:8081/v1/data/test'
- script: kubectl exec -n $NAMESPACE test-regorule-0 -- python /tmp/test-regorule.py -u 'http://test-opa-server-default:8081/v1/data/stackable/opa/userinfo/v1'
Loading