Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace MethodIDs with ImageIDs #326

Merged
merged 13 commits into from
Jan 17, 2023
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
"iosfwd": "cpp",
"vector": "cpp",
"charconv": "cpp",
"stdexcept": "cpp"
"stdexcept": "cpp",
"array": "cpp"
}
}
6 changes: 6 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@ lto = true
[profile.dev]
opt-level = 3

[profile.dev.build-override]
opt-level = 3

[profile.release]
debug = 1
lto = true

[profile.release.build-override]
opt-level = 3
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ SDK support exists for Rust, C, and C++.
First, the code to be proven must be compiled from its implementation language
into a *method*. A method is represented by a RISC-V ELF file with a special
entry point that runs the code of the method. Additionally, one can compute for
a given method its *method ID* which is a special type of cryptographic hash of
a given method its *image ID* which is a special type of cryptographic hash of
the ELF file, and is required for verification.

Next, the prover runs the method inside the zkVM. The logical RISC-V machine
Expand All @@ -68,7 +68,7 @@ The verifier can then verify the receipt and examine the log. If any tampering
was done to the journal or the seal, the receipt will fail to verify.
Additionally, it is cryptographically infeasible to generate a valid receipt
unless the output of the journal is the exactly correct output for some valid
execution of the method whose method ID matches the receipt. In summary, the
execution of the method whose image ID matches the receipt. In summary, the
receipt acts as a zero knowledge proof of correct execution.

Because the protocol is zero knowledge, the verifier cannot infer anything about
Expand All @@ -92,7 +92,7 @@ other manner of problems. Caveat emptor.
## Getting Started

To get started building applications using the zkVM in Rust, we provide a
[starter template](https://github.com/risc0/risc0-rust-starter) and a
[starter template](https://github.com/risc0/risc0-rust-starter) and a
number of [working examples](https://github.com/risc0/risc0-rust-examples/).

## Example
Expand Down
13 changes: 5 additions & 8 deletions risc0/build/risc0.ld
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,9 @@ EXTERN(__start)

/* Must match risc0/zkvm/platform/src/memory.rs */
MEMORY {
stack : ORIGIN = 0x00000000, LENGTH = 16M
data (RW) : ORIGIN = 0x01000000, LENGTH = 16M
heap : ORIGIN = 0x02000000, LENGTH = 32M
input : ORIGIN = 0x04000000, LENGTH = 32M
stack : ORIGIN = 0x02000000, LENGTH = 16M
data (RW) : ORIGIN = 0x03000000, LENGTH = 16M
heap : ORIGIN = 0x04000000, LENGTH = 32M
prog (X) : ORIGIN = 0x06000000, LENGTH = 32M
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason not to put prog @ 0x08... - 0x0c...? I was of the impression that system memory starts only after 0x0c and that nondet was now below @0x02. It seems like 64M for heap and 64M for code would be better than 32M for heap and 96M for program. Or maybe even more room for heap.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That all makes great sense. I was also thinking maybe prog should live above the system region? Like maybe reading from the PC could use a special memory op.

}

Expand All @@ -47,17 +46,15 @@ SECTIONS {

. = ALIGN(4);

.bss (NOLOAD) : {
.bss (NOLOAD) : {
__bss_begin = .;
*(.sbss*)
*(.gnu.linkonce.sb.*)
*(.bss .bss.*)
*(.gnu.linkonce.b.*)
*(COMMON)
. = ALIGN(4);
__result = .;
/* Result is 9 words * 4 = 36 bytes, 8 words for output, and 1 word for output size*/
__bss_end = . + 36;
__bss_end = .;
} >data

__bss_size = __bss_end - __bss_begin;
Expand Down
78 changes: 18 additions & 60 deletions risc0/build/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@
//!
//! In order for the host to execute guest code in the [RISC Zero
//! zkVM](risc0_zkvm), the host must be provided a compiled RISC-V ELF file and
//! the corresponding [MethodID](risc0_zkvm::MethodId). This crate
//! the corresponding ImageID. This crate
//! contains the functions needed to take zkVM guest code, build a corresponding
//! ELF file and MethodID, and make the MethodID and a path to the ELF file
//! ELF file and ImageID, and make the ImageID and a path to the ELF file
//! available for the host to use.
//!
//! ## Using risc0-build to build guest methods
Expand Down Expand Up @@ -57,10 +57,10 @@
//! include!(concat!(env!("OUT_DIR"), "/methods.rs"));
//! ```
//!
//! This process will generate a method ID (`*_ID`) and the contents of an ELF
//! This process will generate an image ID (`*_ID`) and the contents of an ELF
//! file (`*_ELF`). The names will be derived from the name of the ELF
//! binary, which will be converted to ALL_CAPS to comply with rust naming
//! conventions. Thus, if a method binary is named `multiply`, the method ID
//! conventions. Thus, if a method binary is named `multiply`, the image ID
//! will be named `methods::MULTIPLY_ID` and the contents of the ELF file will
//! be named `methods::MULTIPLY_ELF`. These are included at the beginning
//! of the host-side code:
Expand All @@ -83,10 +83,10 @@ use std::{

use cargo_metadata::{MetadataCommand, Package};
use downloader::{Download, Downloader};
use risc0_zkp::adapter::TapsProvider;
use risc0_zkvm::{MethodId, DEFAULT_METHOD_ID_LIMIT};
use risc0_zkvm::{sha::Digest, MemoryImage, Program};
use risc0_zkvm_platform::{memory::MEM_SIZE, PAGE_SIZE};
use serde::Deserialize;
use sha2::{Digest, Sha256};
use sha2::{Digest as ShaDigest, Sha256};
use tempfile::tempdir_in;
use zip::ZipArchive;

Expand All @@ -112,7 +112,7 @@ struct Risc0Method {
}

impl Risc0Method {
fn make_method_id(&self, code_limit: usize) -> MethodId {
fn make_image_id(&self) -> Digest {
if !self.elf_path.exists() {
eprintln!(
"RISC-V method was not found at: {:?}",
Expand All @@ -121,57 +121,21 @@ impl Risc0Method {
std::process::exit(-1);
}

let method_id_path = self.elf_path.with_extension("id");
let elf_sha_path = self.elf_path.with_extension("sha");
let elf_contents = std::fs::read(&self.elf_path).unwrap();

let elf_sha = Sha256::new()
// Method ID calculation is slow, so only recalculate it if we
// actually get a different ELF file.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I correct to assume that imageId calculation is much more efficient so we no longer need to cache it or should we still consider caching the imageID to avoid recalculating?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be pretty fast without having to cache it. Caching comes with all sorts of complexity and potential for bugs. If we find out it's still too slow we can bring caching back.

.chain_update(&elf_contents)
// Take into account the current circuit; if the circuit
// changes, the Method ID will change as well.
.chain_update(format!(
"{:?}",
risc0_circuit_rv32im::CircuitImpl.get_taps()
))
// If the Method ID version changes, we need to recreate the Method ID.
.chain_update(format!("{}", MethodId::VERSION))
.finalize();

let elf_sha_hex: String = elf_sha
.as_slice()
.iter()
.map(|x| format!("{:02x}", x))
.collect();

if method_id_path.exists() {
if let Ok(cached_sha) = std::fs::read(&elf_sha_path) {
if cached_sha == elf_sha.as_slice() {
println!("MethodID for {} ({}) up to date", self.name, elf_sha_hex);
let buf = std::fs::read(&method_id_path).unwrap();
return MethodId::from_slice(&buf).unwrap();
}
}
}

println!("Computing MethodID for {} ({:})!", self.name, elf_sha_hex);
let method_id = MethodId::compute_with_limit(&elf_contents, code_limit).unwrap();
std::fs::write(method_id_path, method_id.as_slice()).unwrap();
std::fs::write(elf_sha_path, elf_sha).unwrap();
method_id
let elf = fs::read(&self.elf_path).unwrap();
let program = Program::load_elf(&elf, MEM_SIZE as u32).unwrap();
let image = MemoryImage::new(&program, PAGE_SIZE);
image.root
}

fn rust_def(&self, code_limit: usize) -> String {
fn rust_def(&self) -> String {
let elf_path = self.elf_path.display();
let upper = self.name.to_uppercase();
let method_id = self.make_method_id(code_limit);
let method_id = method_id.as_slice();
let image_id = self.make_image_id();
let elf_contents = std::fs::read(&self.elf_path).unwrap();
format!(
r##"
pub const {upper}_ELF: &'static [u8] = &{elf_contents:?};
pub const {upper}_ID: &'static [u8] = &{method_id:?};
pub const {upper}_ID: &'static str = r#"{image_id:?}"#;
pub const {upper}_PATH: &'static str = r#"{elf_path}"#;
"##
)
Expand Down Expand Up @@ -497,11 +461,6 @@ fn build_guest_package<P>(
/// Options defining how to embed a guest package in
/// [`embed_methods_with_options`].
pub struct GuestOptions {
/// The maximum number of cycles (in units of powers of 2) supported by a
/// MethodID. The bigger this value is the longer it takes to compute a
/// MethodID.
pub code_limit: usize,

/// Features for cargo to build the guest with.
pub features: Vec<String>,

Expand All @@ -512,7 +471,6 @@ pub struct GuestOptions {
impl Default for GuestOptions {
fn default() -> Self {
GuestOptions {
code_limit: DEFAULT_METHOD_ID_LIMIT,
features: vec![],
std: true,
}
Expand Down Expand Up @@ -554,7 +512,7 @@ pub fn embed_methods_with_options(mut guest_pkg_to_options: HashMap<&str, GuestO

for method in guest_methods(&guest_pkg, &out_dir) {
methods_file
.write_all(method.rust_def(guest_options.code_limit).as_bytes())
.write_all(method.rust_def().as_bytes())
.unwrap();
}
}
Expand All @@ -575,7 +533,7 @@ pub fn embed_methods_with_options(mut guest_pkg_to_options: HashMap<&str, GuestO
/// listing the relative paths that contain riscv guest method
/// packages.
///
/// To access the generated method IDs and ELF filenames, include the
/// To access the generated image IDs and ELF filenames, include the
/// generated methods.rs:
///
/// ```text
Expand All @@ -584,7 +542,7 @@ pub fn embed_methods_with_options(mut guest_pkg_to_options: HashMap<&str, GuestO
///
/// To conform to rust's naming conventions, the constants are mapped
/// to uppercase. For instance, if you have a method named
/// "my_method", the method ID and elf contents will be defined as
/// "my_method", the image ID and elf contents will be defined as
/// "MY_METHOD_ID" and "MY_METHOD_ELF" respectively.
pub fn embed_methods() {
embed_methods_with_options(HashMap::new())
Expand Down