Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/dep_rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -146,8 +146,8 @@ jobs:
# with default features
just test ${{ inputs.config }} ${{ inputs.hypervisor == 'mshv' && 'mshv2' || '""'}}

# with only one driver enabled (driver mshv/kvm feature is ignored on windows) + seccomp
just test ${{ inputs.config }} seccomp,${{ inputs.hypervisor == 'mshv' && 'mshv2' || inputs.hypervisor == 'mshv3' && 'mshv3' || 'kvm' }}
# with only one driver enabled (driver mshv/kvm feature is ignored on windows)
just test ${{ inputs.config }} ${{ inputs.hypervisor == 'mshv' && 'mshv2' || inputs.hypervisor == 'mshv3' && 'mshv3' || 'kvm' }}

# make sure certain cargo features compile
just check
Expand Down
10 changes: 0 additions & 10 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 3 additions & 9 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,8 @@ test-like-ci config=default-target hypervisor="kvm":
@# with default features
just test {{config}} {{ if hypervisor == "mshv" {"mshv2"} else {""} }}

@# with only one driver enabled + seccomp + build-metadata + init-paging
just test {{config}} seccomp,build-metadata,init-paging,{{ if hypervisor == "mshv" {"mshv2"} else if hypervisor == "mshv3" {"mshv3"} else {"kvm"} }}
@# with only one driver enabled + build-metadata + init-paging
just test {{config}} build-metadata,init-paging,{{ if hypervisor == "mshv" {"mshv2"} else if hypervisor == "mshv3" {"mshv3"} else {"kvm"} }}

@# make sure certain cargo features compile
just check
Expand Down Expand Up @@ -145,7 +145,7 @@ like-ci config=default-target hypervisor="kvm":
{{ if config == "release" { "just bench-ci main " + if hypervisor == "mshv" { "mshv2" } else if hypervisor == "mshv3" { "mshv3" } else { "kvm" } } else { "" } }}

# runs all tests
test target=default-target features="": (test-unit target features) (test-isolated target features) (test-integration "rust" target features) (test-integration "c" target features) (test-seccomp target features) (test-doc target features)
test target=default-target features="": (test-unit target features) (test-isolated target features) (test-integration "rust" target features) (test-integration "c" target features) (test-doc target features)

# runs unit tests
test-unit target=default-target features="":
Expand All @@ -170,12 +170,6 @@ test-integration guest target=default-target features="":
@# run the rest of the integration tests
{{if os() == "windows" { "$env:" } else { "" } }}GUEST="{{guest}}"{{if os() == "windows" { ";" } else { "" } }} {{ cargo-cmd }} test -p hyperlight-host {{ if features =="" {''} else if features=="no-default-features" {"--no-default-features" } else {"--no-default-features -F init-paging," + features } }} --profile={{ if target == "debug" { "dev" } else { target } }} {{ target-triple-flag }} --test '*'

# runs seccomp tests
test-seccomp target=default-target features="":
@# run seccomp test with feature "seccomp" on and off
{{ cargo-cmd }} test --profile={{ if target == "debug" { "dev" } else { target } }} {{ target-triple-flag }} -p hyperlight-host test_violate_seccomp_filters --lib {{ if features =="" {''} else { "--features " + features } }} -- --ignored
{{ cargo-cmd }} test --profile={{ if target == "debug" { "dev" } else { target } }} {{ target-triple-flag }} -p hyperlight-host test_violate_seccomp_filters --no-default-features {{ if features =~"mshv2" {"--features init-paging,mshv2"} else {"--features mshv3,init-paging,kvm" } }} --lib -- --ignored

# tests compilation with no default features on different platforms
test-compilation-no-default-features target=default-target:
@# Linux should fail without a hypervisor feature (kvm, mshv, or mshv3)
Expand Down
2 changes: 0 additions & 2 deletions docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,3 @@ All communication between the host and the guest is done through a shared memory
Hyperlight provides a mechanism for the host to register functions that may be called from the guest. This mechanism is useful to allow developers to provide guests with strictly controlled access to functionality we don't make available by default inside the VM. This mechanism likely represents the largest attack surface area of this project.

To mitigate the risk, only functions that have been explicitly exposed to the guest by the host application, are allowed to be called from the guest. Any attempt to call other host functions will result in an error.

Additionally, we provide an API for using Seccomp filters to further restrict the system calls available to the host-provided functions, to help limit the impact of the un-audited or un-managed functions.
20 changes: 6 additions & 14 deletions docs/signal-handlers-development-notes.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,9 @@
# Signal Handling in Hyperlight

Hyperlight registers custom signal handlers to intercept and manage specific signals, primarily `SIGSYS` and `SIGRTMIN`. Here's an overview of the registration process:
- **Preserving Old Handlers**: When registering a new signal handler, Hyperlight first retrieves and stores the existing handler using `OnceCell`. This allows Hyperlight to delegate signals to the original handler if necessary.
Hyperlight registers custom signal handlers to intercept and manage specific signals, primarily `SIGRTMIN`. Here's an overview of the registration process:
- **Custom Handlers**:
- **`SIGSYS` Handler**: Captures disallowed syscalls enforced by seccomp. If the signal originates from a hyperlight thread, Hyperlight logs the syscall details. Otherwise, it delegates the signal to the previously registered handler.
- **`SIGRTMIN` Handler**: Utilized for inter-thread signaling, such as execution cancellation. Similar to SIGSYS, it distinguishes between application and non-hyperlight threads to determine how to handle the signal.
- **Thread Differentiation**: Hyperlight uses thread-local storage (IS_HYPERLIGHT_THREAD) to identify whether the current thread is a hyperlight thread. This distinction ensures that signals are handled appropriately based on the thread's role.

## Potential Issues and Considerations

### Handler Invalidation

**Issue**: After Hyperlight registers its custom signal handler and preserves the `old_handler`, if the host or another component modifies the signal handler for the same signal, it can lead to:
- **Invalidation of `old_handler`**: The stored old_handler reference may no longer point to a valid handler, causing undefined behavior when Hyperlight attempts to delegate signals.
- **Loss of Custom Handling**: Hyperlight's custom handler might not be invoked as expected, disrupting its ability to enforce syscall restrictions or manage inter-thread signals.

- **`SIGRTMIN` Handler**: Utilized for inter-thread signaling, such as execution cancellation.
- **Killing a sandbox**:
- To stop a sandboxed process, a `SIGRTMIN` signal must be delivered to the thread running the sandboxed code.
- The sandbox provides an interface to obtain an interrupt handle, which includes the thread ID and a method to dispatch the signal.
- Hyperlight uses the `pthread_kill` function to send this signal directly to the targeted thread.
4 changes: 1 addition & 3 deletions src/hyperlight_host/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,6 @@ windows-version = "0.1"
lazy_static = "1.4.0"

[target.'cfg(unix)'.dependencies]
seccompiler = { version = "0.5.0", optional = true }
kvm-bindings = { version = "0.14", features = ["fam-wrappers"], optional = true }
kvm-ioctls = { version = "0.24", optional = true }
mshv-bindings2 = { package="mshv-bindings", version = "=0.2.1", optional = true }
Expand Down Expand Up @@ -126,8 +125,7 @@ cfg_aliases = "0.2.1"
built = { version = "0.8.0", optional = true, features = ["chrono", "git2"] }

[features]
default = ["kvm", "mshv3", "seccomp", "build-metadata", "init-paging"]
seccomp = ["dep:seccompiler"]
default = ["kvm", "mshv3", "build-metadata", "init-paging"]
function_call_metrics = []
executable_heap = []
# This feature enables printing of debug information to stdout in debug builds
Expand Down
1 change: 0 additions & 1 deletion src/hyperlight_host/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,6 @@ fn main() -> Result<()> {
// the other features they want.
mshv2: { all(feature = "mshv2", target_os = "linux") },
mshv3: { all(feature = "mshv3", not(feature="mshv2"), target_os = "linux") },
seccomp: { all(feature = "seccomp", target_os = "linux", not(target_env = "musl")) },
}

#[cfg(feature = "build-metadata")]
Expand Down
15 changes: 0 additions & 15 deletions src/hyperlight_host/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,6 @@ pub enum HyperlightError {
#[error("Error converting CString {0:?}")]
CStringConversionError(#[from] std::ffi::NulError),

/// A disallowed syscall was caught
#[error("Seccomp filter trapped on disallowed syscall (check STDERR for offending syscall)")]
#[cfg(seccomp)]
DisallowedSyscall,

/// A generic error with a message
#[error("{0}")]
Error(String),
Expand Down Expand Up @@ -216,16 +211,6 @@ pub enum HyperlightError {
#[error("Stack overflow detected")]
StackOverflow(),

/// a backend error occurred with seccomp filters
#[error("Backend Error with Seccomp Filter {0:?}")]
#[cfg(seccomp)]
SeccompFilterBackendError(#[from] seccompiler::BackendError),

/// an error occurred with seccomp filters
#[error("Error with Seccomp Filter {0:?}")]
#[cfg(seccomp)]
SeccompFilterError(#[from] seccompiler::Error),

/// Tried to restore snapshot to a sandbox that is not the same as the one the snapshot was taken from
#[error("Snapshot was taken from a different sandbox")]
SnapshotSandboxMismatch,
Expand Down
35 changes: 1 addition & 34 deletions src/hyperlight_host/src/func/host_functions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ use hyperlight_common::flatbuffer_wrappers::function_types::{ParameterValue, Ret

use super::utils::for_each_tuple;
use super::{ParameterTuple, ResultType, SupportedReturnType};
use crate::sandbox::UninitializedSandbox;
use crate::sandbox::host_funcs::FunctionEntry;
use crate::sandbox::{ExtraAllowedSyscall, UninitializedSandbox};
use crate::{Result, new_error};

/// A sandbox on which (primitive) host functions can be registered
Expand All @@ -33,15 +33,6 @@ pub trait Registerable {
name: &str,
hf: impl Into<HostFunction<Output, Args>>,
) -> Result<()>;
/// Register a primitive host function whose worker thread has
/// extra permissive seccomp filters installed
#[cfg(seccomp)]
fn register_host_function_with_syscalls<Args: ParameterTuple, Output: SupportedReturnType>(
&mut self,
name: &str,
hf: impl Into<HostFunction<Output, Args>>,
eas: Vec<ExtraAllowedSyscall>,
) -> Result<()>;
}
impl Registerable for UninitializedSandbox {
fn register_host_function<Args: ParameterTuple, Output: SupportedReturnType>(
Expand All @@ -56,28 +47,6 @@ impl Registerable for UninitializedSandbox {

let entry = FunctionEntry {
function: hf.into().into(),
extra_allowed_syscalls: None,
parameter_types: Args::TYPE,
return_type: Output::TYPE,
};

(*hfs).register_host_function(name.to_string(), entry, &mut self.mgr)
}
#[cfg(seccomp)]
fn register_host_function_with_syscalls<Args: ParameterTuple, Output: SupportedReturnType>(
&mut self,
name: &str,
hf: impl Into<HostFunction<Output, Args>>,
eas: Vec<ExtraAllowedSyscall>,
) -> Result<()> {
let mut hfs = self
.host_funcs
.try_lock()
.map_err(|e| new_error!("Error locking at {}:{}: {}", file!(), line!(), e))?;

let entry = FunctionEntry {
function: hf.into().into(),
extra_allowed_syscalls: Some(eas),
parameter_types: Args::TYPE,
return_type: Output::TYPE,
};
Expand Down Expand Up @@ -195,13 +164,11 @@ pub(crate) fn register_host_function<Args: ParameterTuple, Output: SupportedRetu
func: impl Into<HostFunction<Output, Args>>,
sandbox: &mut UninitializedSandbox,
name: &str,
extra_allowed_syscalls: Option<Vec<ExtraAllowedSyscall>>,
) -> Result<()> {
let func = func.into().into();

let entry = FunctionEntry {
function: func,
extra_allowed_syscalls: extra_allowed_syscalls.clone(),
parameter_types: Args::TYPE,
return_type: Output::TYPE,
};
Expand Down
2 changes: 0 additions & 2 deletions src/hyperlight_host/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,6 @@ pub mod metrics;
/// outside this file. Types from this module needed for public consumption are
/// re-exported below.
pub mod sandbox;
#[cfg(seccomp)]
pub(crate) mod seccomp;
/// Signal handling for Linux
#[cfg(target_os = "linux")]
pub(crate) mod signal_handlers;
Expand Down
25 changes: 1 addition & 24 deletions src/hyperlight_host/src/metrics/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -133,11 +133,7 @@ mod tests {
if #[cfg(feature = "function_call_metrics")] {
use metrics::Label;

let expected_num_metrics = if cfg!(all(seccomp)) {
3 // if seccomp enabled, the host call duration metric is emitted on a separate thread which this local recorder doesn't capture
} else {
4
};
let expected_num_metrics = 4;

// Verify that the histogram metrics are recorded correctly
assert_eq!(snapshot.len(), expected_num_metrics);
Expand Down Expand Up @@ -185,25 +181,6 @@ mod tests {
),
"Histogram metric does not match expected value"
);

if !cfg!(all(seccomp)) {
// 4. Host call duration
let histogram_key = CompositeKey::new(
metrics_util::MetricKind::Histogram,
Key::from_parts(
METRIC_HOST_FUNC_DURATION,
vec![Label::new("function_name", "HostPrint")],
),
);
let histogram_value = &snapshot.get(&histogram_key).unwrap().2;
assert!(
matches!(
histogram_value,
metrics_util::debugging::DebugValue::Histogram(histogram) if histogram.len() == 1
),
"Histogram metric does not match expected value"
);
}
} else {
// Verify that the counter metrics are recorded correctly
assert_eq!(snapshot.len(), 1);
Expand Down
64 changes: 2 additions & 62 deletions src/hyperlight_host/src/sandbox/host_funcs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ use hyperlight_common::flatbuffer_wrappers::host_function_details::HostFunctionD
use termcolor::{Color, ColorChoice, ColorSpec, StandardStream, WriteColor};
use tracing::{Span, instrument};

use super::ExtraAllowedSyscall;
use crate::HyperlightError::HostFunctionNotFound;
use crate::func::host_functions::TypeErasedHostFunction;
use crate::mem::mgr::SandboxMemoryManager;
Expand Down Expand Up @@ -58,7 +57,6 @@ impl From<&mut FunctionRegistry> for HostFunctionDetails {

pub struct FunctionEntry {
pub function: TypeErasedHostFunction,
pub extra_allowed_syscalls: Option<Vec<ExtraAllowedSyscall>>,
pub parameter_types: &'static [ParameterType],
pub return_type: ReturnType,
}
Expand Down Expand Up @@ -119,18 +117,15 @@ impl FunctionRegistry {
fn call_host_func_impl(&self, name: &str, args: Vec<ParameterValue>) -> Result<ReturnValue> {
let FunctionEntry {
function,
extra_allowed_syscalls,
parameter_types: _,
return_type: _,
} = self
.functions_map
.get(name)
.ok_or_else(|| HostFunctionNotFound(name.to_string()))?;

// Create a new thread when seccomp is enabled on Linux
maybe_with_seccomp(name, extra_allowed_syscalls.as_deref(), || {
crate::metrics::maybe_time_and_emit_host_call(name, || function.call(args))
})
// Make the host function call
crate::metrics::maybe_time_and_emit_host_call(name, || function.call(args))
}
}

Expand All @@ -153,58 +148,3 @@ pub(super) fn default_writer_func(s: String) -> Result<i32> {
}
}
}

#[cfg(seccomp)]
fn maybe_with_seccomp<T: Send>(
name: &str,
syscalls: Option<&[ExtraAllowedSyscall]>,
f: impl FnOnce() -> Result<T> + Send,
) -> Result<T> {
use std::thread;

use crate::seccomp::guest::get_seccomp_filter_for_host_function_worker_thread;

// Use a scoped thread so that we can pass around references without having to clone them.
thread::scope(|s| {
thread::Builder::new()
.name(format!("Host Function Worker Thread for: {name:?}"))
.spawn_scoped(s, move || {
let seccomp_filter = get_seccomp_filter_for_host_function_worker_thread(syscalls)?;
seccomp_filter
.iter()
.try_for_each(|filter| seccompiler::apply_filter(filter))?;

// We have a `catch_unwind` here because, if a disallowed syscall is issued,
// we handle it by panicking. This is to avoid returning execution to the
// offending host function—for two reasons: (1) if a host function is issuing
// disallowed syscalls, it could be unsafe to return to, and (2) returning
// execution after trapping the disallowed syscall can lead to UB (e.g., try
// running a host function that attempts to sleep without `SYS_clock_nanosleep`,
// you'll block the syscall but panic in the aftermath).
match std::panic::catch_unwind(std::panic::AssertUnwindSafe(f)) {
Ok(val) => val,
Err(err) => {
if let Some(crate::HyperlightError::DisallowedSyscall) =
err.downcast_ref::<crate::HyperlightError>()
{
return Err(crate::HyperlightError::DisallowedSyscall);
}

crate::log_then_return!("Host function {} panicked", name);
}
}
})?
.join()
.map_err(|_| new_error!("Error joining thread executing host function"))?
})
}

#[cfg(not(seccomp))]
fn maybe_with_seccomp<T: Send>(
_name: &str,
_syscalls: Option<&[ExtraAllowedSyscall]>,
f: impl FnOnce() -> Result<T> + Send,
) -> Result<T> {
// No seccomp, just call the function
f()
}
Loading
Loading