feat(compiler): support registered host-await builtins for natural function call syntax by kusha · Pull Request #667 · microsoft/regorus

kusha · 2026-04-08T20:07:04Z

Summary

Extends the RVM compiler to accept a list of host function names at compile time. Calls to registered names are compiled directly into HostAwait instructions, allowing policy authors to write lookup(input.account_id) instead of __builtin_host_await(input.account_id, "lookup").

Motivation

No compile-time validation of host-supported identifiers: The identifier passed to __builtin_host_await is a runtime string. Typos like "lokup" instead of "lookup" are silently accepted by the compiler and only discovered at runtime when the host rejects the identifier.
Wrapper workaround breaks caller identification: Defining a wrapper lookup(x) := __builtin_host_await(x, "lookup") compiles the HostAwait instruction inside the wrapper module, not the calling module — breaking caller identification for authorization and auditing.
Host can override standard builtins: Registering a name that matches a standard Rego builtin (e.g. time.parse_duration_ns) lets the host intercept that call and provide its own implementation.

Changes

Compiler (`src/languages/rego/compiler/`)

register_host_await_builtin() on Compiler to declare host function names. Validates arg_count == 1 and rejects the reserved name __builtin_host_await.
compile_from_policy_with_host_await() entry point that accepts the builtin list. Existing compile_from_policy() delegates with an empty list.
determine_call_target() extended with resolution order: explicit __builtin_host_await → registered host-await → user-defined → standard builtin.
Both explicit and registered paths emit identical HostAwait { dest, arg, id } bytecode.

Documentation (`docs/rvm/`)

instruction-set.md: Registered builtin syntax, resolution order, argument handling.
architecture.md: Updated to describe both emission paths.

Tests (`tests/rvm/rego/`)

Extended test harness with HostAwaitBuiltinSpec and argument assertion.
9 YAML test cases: suspend/resume, run-to-completion, multiple names, queue, shadowing, object packing, arg_count rejection, reserved name rejection, standard builtin override.

Constraints

arg_count must be 1: The HostAwait instruction carries a single argument register. Use object packing to pass multiple values: lookup({"key1": v1, "key2": v2}). This can be lifted in the future by having the compiler auto-pack multiple arguments into an array or object.
__builtin_host_await is reserved: Attempting to register this name produces a compile-time error. It is handled by a dedicated code path (explicit 2-argument form) and cannot be overridden.

Limitations

No conflict detection: A registered name that collides with a standard builtin silently wins. No warning is emitted.
__builtin_host_await still exists: The raw builtin remains functional. Policy authors who discover it can use arbitrary identifier strings. However, the host controls how identifiers are resolved and can reject or error on unexpected identifiers at runtime.

Allow hosts to register function names at compile time so that calls to those names emit HostAwait instructions directly, enabling natural syntax like fetch(x) instead of __builtin_host_await(x, "fetch"). - Add host_await_builtins map and register_host_await_builtin() to Compiler - Validate arg_count == 1 and reject reserved __builtin_host_await name - Extend determine_call_target() resolution: explicit > registered > user > builtin - Both explicit and registered paths emit identical HostAwait bytecode - Add compile_from_policy_with_host_await() entry point in rules.rs - Extended test harness with HostAwaitBuiltinSpec and args assertion - 9 YAML test cases: suspend/resume, run-to-completion, multiple names, queue, shadowing, object packing, arg_count rejection, reserved name rejection, standard builtin override - Documentation: instruction-set.md, architecture.md

Copilot

Pull request overview

This PR extends the Rego→RVM compiler to support “registered” host-await builtins, allowing natural function-call syntax (e.g. lookup(x)) to compile directly into HostAwait instructions, while retaining the explicit __builtin_host_await(arg, id) path.

Changes:

Add Compiler::compile_from_policy_with_host_await(...) and Compiler::register_host_await_builtin(...) to configure host-await builtin names (with arg_count == 1 enforced).
Extend call target resolution to prioritize explicit __builtin_host_await, then registered host-await names, then user-defined functions, then standard builtins.
Update the YAML-based RVM Rego test harness to pass registered builtins and (in suspendable mode) assert the host-await argument payload; add a new YAML suite covering registered host-await scenarios.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
`src/languages/rego/compiler/mod.rs`	Adds compiler state + API to register host-await builtins and validate registration constraints.
`src/languages/rego/compiler/rules.rs`	Adds a new compile entry point that accepts registered host-await builtins and wires it into compilation.
`src/languages/rego/compiler/function_calls.rs`	Implements resolution order + emits `HostAwait` for registered names by auto-loading the identifier literal.
`tests/rvm/rego/mod.rs`	Extends harness to pass registered builtin specs into compilation and validate suspendable host-await arguments.
`tests/rvm/rego/cases/registered_host_await.yaml`	Adds test coverage for registered host-await behavior (shadowing, queueing, override, validation errors).
`docs/rvm/instruction-set.md`	Documents registered host-await builtin syntax, resolution order, and argument handling constraints.
`docs/rvm/architecture.md`	Updates architecture walkthrough to describe both explicit and registered `HostAwait` emission paths.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/languages/rego/compiler/function_calls.rs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Mark Birger <birgerm@yandex.ru>

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-09T18:23:17Z

tests/rvm/rego/mod.rs

+                                assert_eq!(
+                                    actual, expected,
+                                    "HostAwait argument mismatch for {:?}: expected {:?}, got {:?}",
+                                    identifier, expected, actual
+                                );


assert_eq! will panic on a HostAwait argument mismatch, which bypasses this test harness’s normal error propagation (and can skip listing dumps / want_error handling). Prefer returning an anyhow::Error (or using the existing bail-with-listing pattern by threading case context) so failures are reported consistently.

Suggested change

assert_eq!(

actual, expected,

"HostAwait argument mismatch for {:?}: expected {:?}, got {:?}",

identifier, expected, actual

);

if actual != expected {

return Err(anyhow::anyhow!(

"HostAwait argument mismatch for {:?}: expected {:?}, got {:?}",

identifier,

expected,

actual

));

}

Copilot · 2026-04-09T18:23:17Z

tests/rvm/rego/cases/registered_host_await.yaml

+
+        result := kv_store(input.key, input.value)
+    query: data.demo.result
+    # Registration panics because arg_count must be 1.


The comment says registration "panics" when arg_count is invalid, but the compiler path returns a compile-time error (register_host_await_builtin returns Err). Consider rewording to avoid implying a panic.

Suggested change

# Registration panics because arg_count must be 1.

# Registration fails with an error because arg_count must be 1.

anakrish

Overall looks good.

anakrish · 2026-04-09T18:29:48Z

tests/rvm/rego/cases/registered_host_await.yaml

@@ -0,0 +1,240 @@
+cases:


More test cases to lock down behavior:

Out param syntax: e.g: lookup(in, out)

Test case that uses __builtin_host_await and registerd builtins in same policy

Empty host await builtins

anakrish · 2026-04-09T18:33:38Z

src/languages/rego/compiler/rules.rs

+    pub fn compile_from_policy_with_host_await(
+        policy: &CompiledPolicy,
+        entry_points: &[&str],
+        host_await_builtins: &[(&str, usize)],


Scenarios to define behavior for:

Duplicate entries in host_await_builtins

Empty host_await_builtins

A name is empty or whitespace

anakrish · 2026-04-09T18:37:17Z

src/languages/rego/compiler/function_calls.rs

+                    (arg_regs[0], arg_regs[1])
+                } else {
+                    // Registered host-awaitable builtin — identifier is the function name
+                    if arg_regs.len() != 1 {


Need to define behavior when registered host await builtin shadows a user write policy rule (both regular and function)

anakrish · 2026-04-09T18:39:19Z

src/languages/rego/compiler/function_calls.rs

        }

+        // Check registered host-awaitable builtins
+        if let Some(&arg_count) = self.host_await_builtins.get(original_fcn_path) {


We need to detect shadowing here.

anakrish · 2026-04-09T18:41:35Z

src/languages/rego/compiler/function_calls.rs


+        // Check registered host-awaitable builtins
+        if let Some(&arg_count) = self.host_await_builtins.get(original_fcn_path) {
+            return Ok(CallTarget::HostAwait {


Maybe it might be clear to distinguish between the builtin and the custom registered builtins via a new variant say RegisteredHostAwait. That way the rest of the code doesn't need to depend on the builtin's name (__builtin_host_await) to distinguish between builtin and registered.

anakrish requested a review from Copilot April 8, 2026 20:55

Copilot started reviewing on behalf of anakrish April 8, 2026 20:56 View session

Copilot AI reviewed Apr 8, 2026

View reviewed changes

src/languages/rego/compiler/function_calls.rs Outdated Show resolved Hide resolved

Update src/languages/rego/compiler/function_calls.rs

2864312

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Mark Birger <birgerm@yandex.ru>

kusha mentioned this pull request Apr 9, 2026

feat(compiler): FFI and C# bindings for RVM, incl. host-await built-ins #672

Draft

anakrish requested a review from Copilot April 9, 2026 18:19

Copilot started reviewing on behalf of anakrish April 9, 2026 18:20 View session

Copilot AI reviewed Apr 9, 2026

View reviewed changes

anakrish reviewed Apr 9, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(compiler): support registered host-await builtins for natural function call syntax#667

feat(compiler): support registered host-await builtins for natural function call syntax#667
kusha wants to merge 2 commits intomicrosoft:mainfrom
kusha:markbirger/registered-host-await-builtins

kusha commented Apr 8, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 9, 2026

Uh oh!

Copilot AI Apr 9, 2026

Uh oh!

anakrish left a comment

Uh oh!

anakrish Apr 9, 2026

Uh oh!

anakrish Apr 9, 2026

Uh oh!

anakrish Apr 9, 2026

Uh oh!

anakrish Apr 9, 2026

Uh oh!

anakrish Apr 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	# Registration panics because arg_count must be 1.
	# Registration fails with an error because arg_count must be 1.

Conversation

kusha commented Apr 8, 2026

Summary

Motivation

Changes

Compiler (src/languages/rego/compiler/)

Documentation (docs/rvm/)

Tests (tests/rvm/rego/)

Constraints

Limitations

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

anakrish left a comment

Choose a reason for hiding this comment

Uh oh!

anakrish Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

anakrish Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

anakrish Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

anakrish Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

anakrish Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Compiler (`src/languages/rego/compiler/`)

Documentation (`docs/rvm/`)

Tests (`tests/rvm/rego/`)