diff --git a/README.md b/README.md index 9477219..d31d00b 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,7 @@ These show useful patterns you may want to copy: ## Illustrating Dstack Features - [./prelaunch-script](./prelaunch-script) - [./private-docker-image-deployment](./private-docker-image-deployment) +- [./attestation](./attestation) shows how to verify attestation for Dstack apps ## App examples - [./timelock-nts](./timelock-nts) a timelock decryption example using secure NTP (NTS) from Cloudflare as a time oracle ## Tutorial (Coming soon) diff --git a/attestation/configid-based/.gitignore b/attestation/configid-based/.gitignore new file mode 100644 index 0000000..1c8fbfa --- /dev/null +++ b/attestation/configid-based/.gitignore @@ -0,0 +1,2 @@ +/work + diff --git a/attestation/configid-based/README.md b/attestation/configid-based/README.md new file mode 100644 index 0000000..f23899f --- /dev/null +++ b/attestation/configid-based/README.md @@ -0,0 +1,150 @@ +# ConfigID-based Dstack Remote Attestation Verification + +Dstack v0.5.1 introduced [compose_hash in mr_config_id](https://github.com/Dstack-TEE/dstack/pull/190), which simplifies the verification process compared to the RTMR3-based verification. The verification doesn't need to replay the RTMRs, and only needs to compare the RTMRs and mr_config_id to known good values. + +## Steps + +This guide walks you through the process of deploying a Dstack application and verifying its attestation using ConfigID-based verification. + +### 1. Download Required Materials + +```bash +./attest.sh download +``` + +**Explanation:** This step downloads all necessary tools and images required for the verification process: +- dcap-qvl tool for verifying Intel TDX quotes +- dstack-mr tool for calculating expected measurement values +- Dstack VM image that will be used for deployment + +The downloaded files are stored in the `work/bin` and `work/images` directories. + +### 2. Prepare Application Composition + +```bash +./attest.sh compose +``` + +**Explanation:** This step generates an `app-compose.json` file based on the `docker-compose.yaml`. The app-compose.json file defines what code to run in the App and the configuration of the App. The script also calculates a compose hash, which is a SHA-256 hash of the app-compose.json file. This hash is crucial for the attestation verification process as it's used to create the mr_config_id. + +### 3. Calculate Known Good Measurement Registers + +```bash +./attest.sh calc-mrs +``` + +**Explanation:** Before deploying the application, this step calculates the expected "known good" measurement register values that should be present in a legitimate, unmodified deployment. It uses the dstack-mr tool to: +- Calculate the mr_config_id by taking the compose hash and padding it to 96 characters +- Generate expected values for RTMR[0-2] based on the VM image metadata and configuration +- Save these values to `known_good_mrs.json` for later comparison + +### 4. Deploy the Application + +```bash +./attest.sh deploy +``` + +**Explanation:** This step deploys the application to the Dstack environment: +- Uploads the app-compose.json to the dstack-vmm +- Creates a VM with the specified configuration +- Waits for the VM to boot and initialize +- Saves the instance ID for future reference + +### 5. Verify Attestation + +```bash +./attest.sh verify +``` + +**Explanation:** The final step verifies that the deployed application is running in a genuine Intel TDX environment with the expected configuration: +- Requests a quote from the application's `/quote.json` endpoint +- Verifies the quote using the dcap-qvl tool +- Compares the measurement registers in the quote with the known good values calculated earlier: + - mr_config_id (contains the app-compose.json hash) + - mr_td (TD measurement register) + - rt_mr0, rt_mr1, rt_mr2 (runtime measurement registers) +- Displays the report data, which can contain application-specific attestation information + +The verification process confirms that the application is running in a genuine TDX environment with the expected configuration and has not been tampered with. + + +## Full log + +```bash +$ ./attest.sh download +Downloading materials required for verification... +Downloading DCAP QVL tool from https://github.com/Phala-Network/dcap-qvl/releases/download/v0.2.4/dcap-qvl-linux-amd64 + % Total % Received % Xferd Average Speed Time Time Time Current + Dload Upload Total Spent Left Speed + 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 +100 7144k 100 7144k 0 0 11.8M 0 --:--:-- --:--:-- --:--:-- 11.8M +Downloading Dstack MR tool from https://github.com/kvinwang/dstack-mr/releases/download/v0.5.0/dstack-mr-linux-amd64 + % Total % Received % Xferd Average Speed Time Time Time Current + Dload Upload Total Spent Left Speed + 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 +100 6603k 100 6603k 0 0 11.6M 0 --:--:-- --:--:-- --:--:-- 104M +Downloading Dstack image... +Downloading image from https://github.com/Dstack-TEE/meta-dstack/releases/download/v0.5.1/dstack-0.5.1.tar.gz + % Total % Received % Xferd Average Speed Time Time Time Current + Dload Upload Total Spent Left Speed + 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 +100 161M 100 161M 0 0 76.3M 0 0:00:02 0:00:02 --:--:-- 98.5M +Extracting image... +Image extracted successfully +Downloads completed successfully +``` + + +```bash +$ ./attest.sh compose +Preparing app-compose.json for deployment... +Generating app-compose.json +app-compose.json has been prepared and is ready for deployment +Compose hash: eaae2351f15ecb6db16135b31039afa2a8b023e287c28200c44aeae5f0c206bd +``` + +```bash +$ ./attest.sh calc-mrs +Calculating known good MRs before deployment... +Using mr_config_id: eaae2351f15ecb6db16135b31039afa2a8b023e287c28200c44aeae5f0c206bd00000000000000000000000000000000 +Running dstack-mr to calculate MRs... +MRs calculation completed. Results saved to known_good_mrs.json +``` + +```bash +$ ./attest.sh deploy +Deploying app... +Deploying app... +✅ App deployed successfully! +VM ID: 5ba3bcd0-7344-4450-b919-e86e52dd3b6c +Waiting for the app to start... +Boot progress: booting +Boot progress: booting +Boot progress: booting +Boot progress: booting +Boot progress: initializing data disk +Boot progress: initializing data disk +Boot progress: initializing data disk +✅ App started successfully! +Instance ID: 41c97635037ba6e5ac6752be10ed1036bb4797a7 +``` + +```bash +$ ./attest.sh verify +Requesting quote from the app and verifying attestation... +Requesting quote from /quote +✅ Quote downloaded successfully! +Verifying attestation using DCAP QVL tool +Getting collateral from PCS... +Quote verified +✅ Attestation verification successful! +Comparing RTMRs with known good values... +Using TD10 report for verification +✅ mr_config_id matches known good value +✅ mr_td matches known good value +✅ rt_mr0 matches known good value +✅ rt_mr1 matches known good value +✅ rt_mr2 matches known good value +Report data: +12340000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 +``` diff --git a/attestation/configid-based/attest.sh b/attestation/configid-based/attest.sh new file mode 100755 index 0000000..a1b4b85 --- /dev/null +++ b/attestation/configid-based/attest.sh @@ -0,0 +1,416 @@ +#!/bin/bash + +# Configuration variables with default values +IMAGE_VERSION="0.5.2" +VCPU_COUNT="2" +MEM_SIZE="2048" + +# The dstack-vmm RPC endpoint +# VMM_URL="http://localhost:12000" + +# The dstack-gateway base domain +# GATEWAY_BASE_DOMAIN="app.kvin.wang:12004" + +DCAP_QVL_DL_URL="https://github.com/Phala-Network/dcap-qvl/releases/download/v0.2.4/dcap-qvl-linux-amd64" +DSTACK_MR_DL_URL="https://github.com/kvinwang/dstack-mr/releases/download/v0.5.0/dstack-mr-linux-amd64" +IMAGE_DL_URL="https://github.com/Dstack-TEE/meta-dstack/releases/download/v${IMAGE_VERSION}/dstack-${IMAGE_VERSION}.tar.gz" + +THIS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +WORK_DIR="${THIS_DIR}/work" +IMAGES_DIR="${WORK_DIR}/images" +TOOLS_DIR="${WORK_DIR}/bin" + + +# Function to display usage information +usage() { + echo "Usage: $0 " + echo "" + echo "Commands:" + echo " download Downloads the materials required for verification (image, tools)" + echo " compose Prepare app-compose.json to be deployed" + echo " calc-mrs Calculate the known good MRs before deployment" + echo " deploy Deploy the app" + echo " verify Request quote from the app and verify the attestation" + echo "" + exit 1 +} + +# Function to check if required variables are set +check_required_vars() { + local missing_vars=false + + if [ -z "$VMM_URL" ]; then + echo "Error: VMM_URL is not set" + missing_vars=true + fi + + if [ -z "$GATEWAY_BASE_DOMAIN" ]; then + echo "Error: GATEWAY_BASE_DOMAIN is not set" + missing_vars=true + fi + + if [ -z "$DCAP_QVL_DL_URL" ]; then + echo "Error: DCAP_QVL_DL_URL is not set" + missing_vars=true + fi + + if [ -z "$DSTACK_MR_DL_URL" ]; then + echo "Error: DSTACK_MR_DL_URL is not set" + missing_vars=true + fi + + if [ "$missing_vars" = true ]; then + echo "Please set the missing variables as environment variables" + exit 1 + fi +} + +# Command: download - Downloads the materials required for verification +download_command() { + echo "Downloading materials required for verification..." + + # Check if curl command exists + if ! command -v curl &> /dev/null; then + echo "Error: curl command not found. Please install it first." + exit 1 + fi + + # Create images directory + mkdir -p "${IMAGES_DIR}" "${TOOLS_DIR}" + + # Download DCAP QVL tool + echo "Downloading DCAP QVL tool from $DCAP_QVL_DL_URL" + curl -L "$DCAP_QVL_DL_URL" -o "${TOOLS_DIR}/dcap-qvl" + chmod +x "${TOOLS_DIR}/dcap-qvl" + + # Download Dstack MR tool + echo "Downloading Dstack MR tool from $DSTACK_MR_DL_URL" + curl -L "$DSTACK_MR_DL_URL" -o "${TOOLS_DIR}/dstack-mr" + chmod +x "${TOOLS_DIR}/dstack-mr" + + # Download and extract image + echo "Downloading Dstack image..." + local image_url="${IMAGE_DL_URL}" + local image_filename="$(basename "${image_url}")" + local image_path="${IMAGES_DIR}/${image_filename}" + + echo "Downloading image from ${image_url}" + curl -L "${image_url}" -o "${image_path}" + + if [ $? -eq 0 ]; then + echo "Extracting image..." + tar -xzf "${image_path}" -C "${IMAGES_DIR}" + if [ $? -eq 0 ]; then + echo "Image extracted successfully" + rm "${image_path}" + else + echo "Error: Failed to extract image" + fi + else + echo "Error: Failed to download image" + fi + + echo "Downloads completed successfully" +} + +# Command: compose - Prepare app-compose.json to be deployed +compose_command() { + echo "Preparing app-compose.json for deployment..." + + # Check if jq command exists + if ! command -v jq &> /dev/null; then + echo "Error: jq command not found. Please install it first." + exit 1 + fi + + # Check if docker-compose.yaml exists + if [ ! -f "../docker-compose.yaml" ]; then + echo "Error: docker-compose.yaml not found in the current directory." + exit 1 + fi + + # Generate app-compose.json + echo "Generating app-compose.json" + + # Create app-compose.json with proper structure + cat > "app-compose.json" << EOF +{ + "manifest_version": 2, + "name": "dstack-attestation-example", + "runner": "docker-compose", + "docker_compose_file": $(jq -Rs . < ../docker-compose.yaml), + "kms_enabled": true, + "gateway_enabled": true, + "local_key_provider_enabled": false, + "key_provider_id": "", + "public_logs": true, + "public_sysinfo": true, + "allowed_envs": [], + "no_instance_id": false, + "secure_time": false +} +EOF + # Calculate compose hash + COMPOSE_HASH=$(sha256sum app-compose.json | cut -d' ' -f1) + echo "app-compose.json has been prepared and is ready for deployment" + echo "Compose hash: ${COMPOSE_HASH}" +} + +# Command: calc-mrs - Calculate the known good MRs before deployment +calc_mrs_command() { + echo "Calculating known good MRs before deployment..." + + # Check if the tools are downloaded + if [ ! -d "$TOOLS_DIR" ] || [ -z "$(ls -A "$TOOLS_DIR")" ]; then + echo "Error: Tools not found. Run '$0 download' first." + exit 1 + fi + + # Check if app-compose.json exists + if [ ! -f "app-compose.json" ]; then + echo "Error: app-compose.json not found. Run '$0 compose' first." + exit 1 + fi + + # Check if image files exist + if [ ! -d "${IMAGES_DIR}/dstack-${IMAGE_VERSION}" ]; then + echo "Error: Dstack image not found. Run '$0 download' first." + exit 1 + fi + + # Calculate mr_config_id from app-compose.json and pad to 48 bytes (96 hex characters) + original_hash=$(sha256sum app-compose.json | cut -d' ' -f1) + # Pad with zeros if needed to reach 96 characters + mr_config_id=$(printf "01%-94s" "$original_hash" | tr ' ' '0') + echo "Using mr_config_id: $mr_config_id" + + # Run dstack-mr with metadata.json from the image + echo "Running dstack-mr to calculate MRs..." + ${TOOLS_DIR}/dstack-mr -metadata "${IMAGES_DIR}/dstack-${IMAGE_VERSION}/metadata.json" \ + -cpu ${VCPU_COUNT} \ + -memory "${MEM_SIZE}M" \ + -json > "known_good_mrs.json.tmp" + + if [ $? -ne 0 ]; then + echo "Error: Failed to calculate MRs" + exit 1 + fi + # Add mr_config_id to the JSON object + jq --arg mr_config_id "$mr_config_id" '. + {"mr_config_id": $mr_config_id}' "known_good_mrs.json.tmp" > "known_good_mrs.json" + + echo "MRs calculation completed. Results saved to known_good_mrs.json" +} + +# Command: deploy - Deploy the app +deploy_command() { + echo "Deploying app..." + + if [ ! -f "app-compose.json" ]; then + echo "Error: app-compose.json not found. Run '$0 compose' first." + exit 1 + fi + + # Generate request.json with VmConfiguration structure + cat > "request.json" << EOF +{ + "name": "dstack-attestation-example", + "image": "dstack-${IMAGE_VERSION}", + "compose_file": $(jq -Rs . < app-compose.json), + "vcpu": ${VCPU_COUNT}, + "memory": ${MEM_SIZE}, + "disk_size": 10 +} +EOF + + # Deploy the app + echo "Deploying app..." + HTTP_STATUS=$(curl -s -w "%{http_code}" -X POST "${VMM_URL}/prpc/CreateVm" \ + -H "Content-Type: application/json" \ + -d @"request.json" -o "vm-id.json") + + if [[ $HTTP_STATUS -ge 200 && $HTTP_STATUS -lt 300 ]]; then + echo "✅ App deployed successfully!" + else + echo "❌ Failed to deploy app! (HTTP Status: $HTTP_STATUS)" + cat vm-id.json + exit 1 + fi + vm_id=$(jq -r '.id' vm-id.json) + echo "VM ID: $vm_id" + echo "Waiting for the app to start..." + while true; do + HTTP_STATUS=$(curl -s -w "%{http_code}" -X GET "${VMM_URL}/prpc/GetInfo?id=$vm_id" -o vm-info.json) + if [[ $HTTP_STATUS -ge 200 && $HTTP_STATUS -lt 300 ]]; then + instance_id=$(jq -r '.info.instance_id // empty' vm-info.json) + if [ -n "$instance_id" ]; then + echo "✅ App started successfully!" + echo "Instance ID: $instance_id" + echo $instance_id > instance_id.txt + break + fi + # Check for boot errors + boot_error=$(jq -r '.info.boot_error // empty' vm-info.json) + if [ -n "$boot_error" ]; then + echo "❌ App failed to start! Error: $boot_error" + exit 1 + fi + # Check boot progress + boot_progress=$(jq -r '.info.boot_progress // empty' vm-info.json) + echo "Boot progress: $boot_progress" + # Continue waiting + fi + sleep 1 + done +} + +# Command: verify - Request quote from the app and verify the attestation +verify_command() { + echo "Requesting quote from the app and verifying attestation..." + + # Check if known good MRs exist + if [ ! -f "known_good_mrs.json" ]; then + echo "Error: Known good MRs not found. Run '$0 calc-mrs' first." + exit 1 + fi + + if [ ! -f "instance_id.txt" ]; then + echo "Error: Instance ID not found. Run '$0 deploy' first." + exit 1 + fi + + instance_id=$(cat instance_id.txt) + + # Request quote from the app + echo "Requesting quote from ${GATEWAY_BASE_URL}/quote" + HTTP_STATUS=$(curl -s -w "%{http_code}" -X GET "https://${instance_id}-8888.${GATEWAY_BASE_DOMAIN}/quote.json" -o "quote.json") + + if [[ $HTTP_STATUS -ge 200 && $HTTP_STATUS -lt 300 ]]; then + echo "✅ Quote downloaded successfully!" + else + echo "❌ Failed to download quote! (HTTP Status: $HTTP_STATUS)" + exit 1 + fi + + if [ ! -f "quote.json" ]; then + echo "Error: Failed to download report from the app" + exit 1 + fi + + jq -j '.quote' quote.json > quote.hex + + # Verify the attestation using DCAP QVL tool + echo "Verifying attestation using DCAP QVL tool" + "${TOOLS_DIR}/dcap-qvl" verify --hex quote.hex > report.json + if [ $? -ne 0 ]; then + echo "❌ Attestation verification failed!" + exit 1 + fi + echo "✅ Attestation verification successful!" + + status=$(jq -r '.status' report.json) + if [ "$status" != "UpToDate" ]; then + echo "⚠️ TCB is not up to date! status: $status" + echo "⚠️ Advisory IDs: $(jq -r '.advisory_ids[]' report.json)" + fi + + echo "Comparing RTMRs with known good values..." + + # Determine which TD report is available (TD10 or TD15) + td_path="" + if jq -e '.report.TD10' report.json > /dev/null 2>&1; then + td_path=".report.TD10" + echo "Using TD10 report for verification" + elif jq -e '.report.TD15' report.json > /dev/null 2>&1; then + td_path=".report.TD15" + echo "Using TD15 report for verification" + else + echo "❌ No TD10 or TD15 report found in the response!" + exit 1 + fi + + # Extract mr_config_id from the report + mr_config_id=$(jq -r "${td_path}.mr_config_id // empty" report.json) + if [ -n "$mr_config_id" ]; then + # Compare with known good value + known_mr_config_id=$(jq -r '.mr_config_id // empty' known_good_mrs.json) + if [ "$mr_config_id" = "$known_mr_config_id" ]; then + echo "✅ mr_config_id matches known good value" + else + echo "❌ mr_config_id does not match known good value!" + echo "Expected: $known_mr_config_id" + echo "Got: $mr_config_id" + fi + else + echo "⚠️ Could not extract mr_config_id from report" + fi + + # Compare RTMRs with known good values + for rtmr in "mr_td" "rt_mr0" "rt_mr1" "rt_mr2"; do + report_value=$(jq -r "${td_path}.${rtmr} // empty" report.json) + if [ -n "$report_value" ]; then + # Convert rtmr name to the format in known_good_mrs.json (e.g., rt_mr0 -> rtmr0) + mrs_key=${rtmr/_/} + known_value=$(jq -r ".$mrs_key // empty" known_good_mrs.json) + + if [ -n "$known_value" ]; then + if [ "$report_value" = "$known_value" ]; then + echo "✅ $rtmr matches known good value" + else + echo "❌ $rtmr does not match known good value!" + echo "Expected: $known_value" + echo "Got: $report_value" + fi + else + echo "⚠️ No known good value for $rtmr" + fi + fi + done + # Print report_data for debugging + report_data=$(jq -r "${td_path}.report_data // empty" report.json) + if [ -n "$report_data" ]; then + echo "Report data:" + echo "$report_data" + else + echo "No report data found in the response!" + fi +} + +# Main script execution +if [ $# -eq 0 ]; then + usage +fi + +mkdir -p "${WORK_DIR}" + +main() { + # Process command + case "$1" in + download) + check_required_vars + download_command + ;; + compose) + check_required_vars + compose_command + ;; + calc-mrs) + check_required_vars + calc_mrs_command + ;; + deploy) + check_required_vars + deploy_command + ;; + verify) + check_required_vars + verify_command + ;; + *) + echo "Error: Unknown command '$1'" + usage + ;; + esac +} + +(cd "${WORK_DIR}" && main "$@") +exit 0 diff --git a/attestation/configid-based/docker-compose.yaml b/attestation/configid-based/docker-compose.yaml new file mode 100644 index 0000000..0de349b --- /dev/null +++ b/attestation/configid-based/docker-compose.yaml @@ -0,0 +1,16 @@ +services: + app: + image: nginx@sha256:eee5eae48e79b2e75178328c7c585b89d676eaae616f03f9a1813aaed820745a + volumes: + - /var/run/dstack.sock:/var/run/dstack.sock + ports: + - "8888:80" + restart: always + command: > + /bin/bash -c ' + mkdir -p /usr/share/nginx/html && + echo "Fetching quote from Dstack..." && + curl --unix-socket /var/run/dstack.sock "http://dstack/GetQuote?report_data=0x1234" -s > /usr/share/nginx/html/quote.json && + echo "Quote saved to /usr/share/nginx/html/quote.json" && + nginx -g "daemon off;" + ' diff --git a/attestation/.gitignore b/attestation/rtmr3-based/.gitignore similarity index 100% rename from attestation/.gitignore rename to attestation/rtmr3-based/.gitignore diff --git a/attestation/README.md b/attestation/rtmr3-based/README.md similarity index 100% rename from attestation/README.md rename to attestation/rtmr3-based/README.md diff --git a/attestation/app-compose.json b/attestation/rtmr3-based/app-compose.json similarity index 100% rename from attestation/app-compose.json rename to attestation/rtmr3-based/app-compose.json diff --git a/attestation/report.json b/attestation/rtmr3-based/report.json similarity index 100% rename from attestation/report.json rename to attestation/rtmr3-based/report.json diff --git a/attestation/verify.py b/attestation/rtmr3-based/verify.py similarity index 100% rename from attestation/verify.py rename to attestation/rtmr3-based/verify.py