Add RFC: Fine-grained sub-resource permissions - #32
Conversation
Propose making all DB-indexed resource types independently permissionable behind a configuration flag. In simplified mode (default), behavior is identical to today. In fine-grained mode, admins can grant on run, trace, assessment, logged_model, model_version, and other sub-resources independently. Key use cases: - Run access without experiment management (#11496) - Assessment-only write access (evaluation boundary) - Model version lifecycle separation from registered_model
| What they want: `NO_PERMISSIONS` on experiments + `EDIT` on runs. | ||
|
|
||
| Today: impossible. The only workaround is granting experiment EDIT, which also | ||
| lets users create new experiments and get automatic MANAGE on them — the opposite |
There was a problem hiding this comment.
Did you mean "users create new runs"? Experiment EDIT is not the create gate. Anyone that is authenticated when workspaces are off can create experiments. When workspaces are on, MANAGE I think is required when they are on.
| | Resource | DB table | API examples | Current resolution | Proposed resolution (fine-grained) | | ||
| |----------|----------|--------------|-------------------|-----------------------------------| | ||
| | `run` | `runs` | CreateRun, UpdateRun, DeleteRun, LogMetric, LogBatch | `_get_permission_from_run_id()` → experiment | `_get_permission_from_run_id()` → run | | ||
| | `trace` | `trace_info` | StartTrace, EndTrace, GetTrace, SearchTraces, SetTraceTag | `_get_permission_from_trace()` → experiment | `_get_permission_from_trace()` → trace | |
There was a problem hiding this comment.
It's not straightforward to do with the current DB schema but a user experience enhancement could be granting read access at the trace session level. I'm thinking it'd be tedious to grant permissions to all traces in a chat bot session if you need a human to look at that specific session. The UI could have a "share session" button in the future.
There was a problem hiding this comment.
Yeah I think this could be a good candidate for resource conditions level auth to handle, e.g. read to all traces with session tag X. Will track in that RFC PR.
| - **Fail-closed enforcement for unregistered resource types** — today, routes | ||
| without a registered validator are fail-open (any authenticated user can | ||
| access). This proposal preserves that behavior. | ||
| - **Parent-scoped sub-resource grants** — in fine-grained mode, a grant like |
There was a problem hiding this comment.
Could we reconsider this for certain resources such as assessments? The rationale being that if an experiment logically maps to a single agent's traces, you probably want to grant assessment access per agent, not globally at the workspace level.
There was a problem hiding this comment.
I think we can use Condition Based Access Control for this use case as well, and in this PR will just add the resource type.
| | `logged_model` | `logged_models` | CreateLoggedModel, GetLoggedModel, DeleteLoggedModel | `_get_permission_from_model_id()` → experiment | `_get_permission_from_model_id()` → logged_model | | ||
| | `prompt_optimization_job` | `jobs` | CreatePromptOptimizationJob, GetPromptOptimizationJob | `_get_permission_from_prompt_optimization_job_id()` → experiment | `_get_permission_from_prompt_optimization_job_id()` → prompt_optimization_job | | ||
| | `review_queue` | `review_queues` | CreateReviewQueue, UpdateReviewQueue, AddItemsToReviewQueue | `_get_permission_from_review_queue_id()` → experiment | `_get_permission_from_review_queue_id()` → review_queue | | ||
| | `label_schema` | `label_schemas` | CreateLabelSchema, UpdateLabelSchema, DeleteLabelSchema | `_get_permission_from_label_schema_id()` → experiment | `_get_permission_from_label_schema_id()` → label_schema | |
There was a problem hiding this comment.
I think label_schema makes sense to keep at the experiment level since I think this additional granularity adds very little value.
|
|
||
| ## Detailed design | ||
|
|
||
| ### New resource types |
There was a problem hiding this comment.
What about the generic job endpoints?
There was a problem hiding this comment.
Not sure how we want to handle these since the existing APIs aren't part of the Auth Model, but I defined a criteria in the RFC for adding new resource types like job endpoints that should assist with future PRs.
|
|
||
| | Resource | DB table | API examples | Current resolution | Proposed resolution (fine-grained) | | ||
| |----------|----------|--------------|-------------------|-----------------------------------| | ||
| | `model_version` | `model_versions` | CreateModelVersion, UpdateModelVersion, DeleteModelVersion | `_get_permission_from_model_version()` → registered_model | `_get_permission_from_model_version()` → model_version | |
There was a problem hiding this comment.
A registered model without versions is not useful. I think if a user has permission to the registered model, the prompt, the MCP server, they should have access to the versions within it. Otherwise, you'd have to grant permission after every new version.
I then also question the value of having permissions granted directly on a versioned entity in the registry, but I'm not opposed to it.
There was a problem hiding this comment.
Yeah right now it's not useful, but with support for Condition based access control we could, for example, restrict updates to versions with labels like production, champion, QA, etc. to protect them from edit. (Registered model would have USE in this case to handle create case)
|
|
||
| ## Detailed design | ||
|
|
||
| ### New resource types |
There was a problem hiding this comment.
What about the MCP registry endpoints?
| Search filtering follows the same pattern as today's `filter_search_experiments`: | ||
|
|
||
| ```python | ||
| def filter_search_traces(user, traces, workspace): |
There was a problem hiding this comment.
This will be problematic in the UI. If the user just has permission to a single trace, then either most pages are empty or MLflow has to query every single trace in the experiment to see if the user has permission to it.
This is being solved by doing request level filtering rather than post response filtering:
mlflow/mlflow#24964
There was a problem hiding this comment.
Restricted new resources being added to * only for now instead of supporting ID as well. Can revisit once request level filtering implemented.
| 3. Sub-resource grants remain stored but have no effect — they can be cleaned up | ||
| or left in place for a future switch back | ||
|
|
||
| # Open questions |
There was a problem hiding this comment.
Have you considered how the UI would work? If the user has access to one run, will they see the experiment the run is in?
There was a problem hiding this comment.
They would need experiment READ permissions. I've reworked to a inheritance model so this should flow more naturally now.
| before switching modes — see [Adoption strategy](#adoption-strategy) for the | ||
| safe transition path._ | ||
|
|
||
| ### Fine-grained mode (opt-in) |
There was a problem hiding this comment.
I don’t like that the simplified/fine-grained choice is all-or-nothing. Have you considered keeping top-level grants as they are today (they apply to children) and let optional sub-resource grants raise permissions?
This is similar to how workspaces work where you can have a broad workspace permission grant but have a more specific permission within the workspace (e.g. read on workspace A but manage on experiment 2 in the workspace).
There was a problem hiding this comment.
+1, I think child resource should inherits from higher hierarchy by default.
There was a problem hiding this comment.
We can still support evaluator requirement by adding "NONE" as a configurable action.
# Evaluator: can view experiments and write assessments, but no read access to runs and models.
client.add_role_permission(pipeline_role.id, "experiment", "*", "READ")
client.add_role_permission(pipeline_role.id, "assessment", "*", "EDIT")
client.add_role_permission(pipeline_role.id, "run", "*", "NONE")
client.add_role_permission(pipeline_role.id, "logged_model", "*", "NONE")
There was a problem hiding this comment.
Reworked to being the inheritance model y'all suggested given that it supports all the customer use cases I am aware of.
There was a problem hiding this comment.
We can still support evaluator requirement by adding "NONE" as a configurable action.
Current model takes max permissions only, which could work if NONE takes higher precedence than all other permissions. This would effectively introduce DENY functionality which I don't see downsides with but it seems to go against the original RBAC RFC proposal.
| before switching modes — see [Adoption strategy](#adoption-strategy) for the | ||
| safe transition path._ | ||
|
|
||
| ### Fine-grained mode (opt-in) |
There was a problem hiding this comment.
+1, I think child resource should inherits from higher hierarchy by default.
| before switching modes — see [Adoption strategy](#adoption-strategy) for the | ||
| safe transition path._ | ||
|
|
||
| ### Fine-grained mode (opt-in) |
There was a problem hiding this comment.
We can still support evaluator requirement by adding "NONE" as a configurable action.
# Evaluator: can view experiments and write assessments, but no read access to runs and models.
client.add_role_permission(pipeline_role.id, "experiment", "*", "READ")
client.add_role_permission(pipeline_role.id, "assessment", "*", "EDIT")
client.add_role_permission(pipeline_role.id, "run", "*", "NONE")
client.add_role_permission(pipeline_role.id, "logged_model", "*", "NONE")
| - **Fine-grained auth for non-DB-indexed resources** (e.g., individual spans | ||
| within a trace) — these are stored in blob/artifact storage, not queryable as | ||
| independent DB entities. | ||
| - **Fail-closed enforcement for unregistered resource types** — today, routes |
There was a problem hiding this comment.
This has been addressed in mlflow/mlflow#25308 very recently - MLflow auth will be fail-closed. This shouldn't affect RFC shape but fyi.
There was a problem hiding this comment.
Ack, yes the concern is more around the lack of enforcement for unregistered resource types.
|
|
||
| | Resource | DB table | API examples | Current resolution | Proposed resolution (fine-grained) | | ||
| |----------|----------|--------------|-------------------|-----------------------------------| | ||
| | `gateway_endpoint_binding` | `gateway_endpoint_bindings` | CreateGatewayEndpointBinding, DeleteGatewayEndpointBinding | `validate_can_update_gateway_endpoint` → gateway_endpoint | `_get_permission_from_gateway_endpoint_binding()` → gateway_endpoint_binding | |
There was a problem hiding this comment.
Gateway endpoint binding is a link between an endpoint and a consumer of it e.g. scorer. In this case, I think this should be defined by minimum of the endpoint and the consumer permissions, not a dedicated permission entry.
There was a problem hiding this comment.
Currently both endpoint and binding are managed by the endpoint's resource permissions. Min of either would go against the existing max model. So either approach is a breaking change.
| "assessment", | ||
| "logged_model", | ||
| "model_version", | ||
| "prompt_optimization_job", |
There was a problem hiding this comment.
nit: Prompt optimization job is a bit strange state today - it exists in data model but no real SDK/UI is built on top of it (just half abandoned for some reason). We should probably just clean it up.
There was a problem hiding this comment.
Makes sense, removed from scope
Rework to the max/escalation model (top-level grants inherit; optional child grants raise), address reviewer comments on the resource set, resolution mechanics, scoping boundaries, and behavioral compatibility.
|
@rrrkharse could you please address this comment #19 (comment) as well from the auth RFC that was just merged? |
| the child. | ||
|
|
||
| ``` | ||
| effective(child) = max(inherited_parent_permission, direct_child_grant) |
There was a problem hiding this comment.
I think this isn't intuitive. Could we model this after file permissions where the more granular one wins?
There was a problem hiding this comment.
I'm now on the fence about this. What do you think @B-Step62 ?
|
|
||
| The experiment grant is `READ`, not lower: a user must be able to read an | ||
| experiment to discover and target it (`GetExperiment` / `SearchExperiments` require | ||
| experiment `can_read`), and must be a workspace member (workspace `USE`). Run |
There was a problem hiding this comment.
and must be a workspace member (workspace
USE)
I don't think this is true today. Are you proposing to add this as part of this RFC?
| creation also permits managing the model entry. | ||
|
|
||
| - **Example user:** a data scientist who registers new versions of existing models. | ||
| - **Policy:** `(registered_model, *, READ)` + `(model_version, *, EDIT)` |
There was a problem hiding this comment.
Nit:
| - **Policy:** `(registered_model, *, READ)` + `(model_version, *, EDIT)` | |
| - **Policy:** `(registered_model, *, READ)` + `(registered_model_version, *, EDIT)` |
| creation also permits managing the model entry. | ||
|
|
||
| - **Example user:** a data scientist who registers new versions of existing models. | ||
| - **Policy:** `(registered_model, *, READ)` + `(model_version, *, EDIT)` |
There was a problem hiding this comment.
It might be out of scope for this RFC, but it'd be nice to have a finegrained permission for setting aliases (e.g. only user1 can set a registered model or prompt or MCP server alias to production). This is a customer use case I've seen and Langfuse does this.
| | `registered_model` | `model_version` | | ||
| | `prompt` | `prompt_version` | | ||
| | `scorer` | `scorer_version` (excluded) | | ||
| | `mcp_server` | `mcp_server_version` (excluded) | |
There was a problem hiding this comment.
Why not include this? It seems simple to add and it makes it consistent with the other AI asset registries in MLflow.
| Notes: | ||
| - **Grain is wildcard-only in this RFC.** Every grantable child is at `*` grain; | ||
| id-level grain (`(trace, <id>, …)`) will be added once request-level search-filter | ||
| push-down is in place (see [Search filtering](#search-filtering)). |
There was a problem hiding this comment.
It might be best to not promise this unless you are 100% you will be doing that lol
| this RFC lands; sub-resources marked **Yes** are what it adds. All grants are | ||
| per-workspace. | ||
|
|
||
| | Parent | Sub-resource | Grantable? | [Grain](#search-filtering) | Escalation use case | Addable later? | |
There was a problem hiding this comment.
Since all grains are wildcard, let's maybe simplify this RFC to just assume that and to drop the #search-filtering section and just keep this as a note in "Out of Scope"
| The subsections below walk each box: the entry point + validator map, the fold and | ||
| its clauses, and how the child's parent (and workspace) are resolved. | ||
|
|
||
| #### Entry point and validator interface |
There was a problem hiding this comment.
In the future, I think this level of implementation details can be left to the PR.
| return get_permission(best) if best is not None else None | ||
| ``` | ||
|
|
||
| #### Proposed changes to the permission fold |
There was a problem hiding this comment.
I'm debating if this should be opt-in because it's essentially adding double the amount of DB queries for a permission check on these APIs. If the RFC stays with max wins, you could check the parent permission first and then fallback to the more granular which makes it as efficient in most cases.
There was a problem hiding this comment.
I see this is addressed under "Performance". I think with #19 , this will actually be more costly.
| listing/opening experiments gates on `(experiment, …)` READ, a different resource | ||
| type the run grant never matches. To navigate to a run in the UI a user needs | ||
| experiment READ as well, which is why the run-logging use case pairs them: | ||
| `(experiment, *, READ)` + `(run, *, EDIT)`. This matches today's behavior — child |
There was a problem hiding this comment.
Is the best practice then to not mix experiments with traces and ML runs? I could see an admin saying they should have access to runs but not traces.
mprahl
left a comment
There was a problem hiding this comment.
I left some comments but I don't think any are blocking. This will be a good feature!
Summary
Proposes making all DB-indexed resource types independently permissionable behind a configuration flag (
permission_granularity = simplified | fine_grained).Problem: MLflow's RBAC recognizes only top-level resource types (experiment, registered_model, etc.). Sub-resources (runs, traces, assessments, logged models, model versions) inherit permissions from their parent — there is no way to grant different access levels on runs vs traces vs assessments within the same experiment.
Solution: A two-mode configuration:
simplified(default): today's behavior, fully backward compatiblefine_grained(opt-in): all sub-resource types become independently permissionableKey use cases:
Design highlights:
resolve_resource()helper per validator — minimal code changeRelated: mlflow/mlflow#11496