Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions crates/cfsctl/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Command-line control utility for composefs repositories and images.
//!
//! `cfsctl` provides a comprehensive interface for managing composefs repositories,
//! creating and mounting filesystem images, handling OCI containers, and performing
//! repository maintenance operations like garbage collection.

use std::{
fs::create_dir_all,
path::{Path, PathBuf},
Expand Down
8 changes: 8 additions & 0 deletions crates/composefs-boot/src/bootloader.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
//! Bootloader entry parsing and manipulation.
//!
//! This module provides functionality to parse and manipulate Boot Loader Specification
//! entries and Unified Kernel Images (UKIs). It supports Type 1 BLS entries with separate
//! kernel and initrd files, Type 2 UKI files, and traditional vmlinuz/initramfs pairs
//! from /usr/lib/modules. Key types include `BootLoaderEntryFile` for parsing BLS
//! configuration files and `BootEntry` enum for representing different boot entry types.

use core::ops::Range;
use std::{
collections::HashMap, ffi::OsStr, os::unix::ffi::OsStrExt, path::PathBuf, str::from_utf8,
Expand Down
7 changes: 7 additions & 0 deletions crates/composefs-boot/src/cmdline.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
//! Kernel command line parsing and manipulation.
//!
//! This module provides utilities for parsing and generating kernel command line arguments,
//! with specific support for composefs parameters. It handles the kernel's simple quoting
//! mechanism and provides functions to extract and create composefs= arguments with optional
//! insecure mode indicators.

use anyhow::{Context, Result};
use composefs::fsverity::FsVerityHashValue;

Expand Down
7 changes: 7 additions & 0 deletions crates/composefs-boot/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
//! Boot integration for composefs filesystem images.
//!
//! This crate provides functionality to transform composefs filesystem images for boot
//! scenarios by extracting boot resources, applying SELinux labels, and preparing
//! bootloader entries. It supports both Boot Loader Specification (Type 1) entries
//! and Unified Kernel Images (Type 2) for UEFI boot.

#![deny(missing_debug_implementations)]

pub mod bootloader;
Expand Down
7 changes: 7 additions & 0 deletions crates/composefs-boot/src/os_release.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
//! Parsing and handling of os-release files.
//!
//! This module provides functionality to parse os-release files according to the
//! freedesktop.org specification. It handles shell-style quoting and variable assignment,
//! extracting common fields like PRETTY_NAME, VERSION_ID, and ID for use in boot labels.
//! The `OsReleaseInfo` type provides methods to generate appropriate boot entry titles.

use std::collections::HashMap;

// We could be using 'shlex' for this but we really only need to parse a subset of the spec and
Expand Down
7 changes: 7 additions & 0 deletions crates/composefs-boot/src/selabel.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
//! SELinux security context labeling for filesystem trees.
//!
//! This module implements SELinux policy parsing and file labeling functionality.
//! It reads SELinux policy files (file_contexts, file_contexts.subs, etc.) and applies
//! appropriate security.selinux extended attributes to filesystem nodes. The implementation
//! uses regex automata for efficient pattern matching against file paths and types.

use std::{
collections::HashMap,
ffi::{OsStr, OsString},
Expand Down
7 changes: 7 additions & 0 deletions crates/composefs-boot/src/uki.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
//! Unified Kernel Image (UKI) parsing and metadata extraction.
//!
//! This module provides functionality to parse PE (Portable Executable) format UKI files
//! and extract embedded sections like .osrel and .cmdline. It implements the Boot Loader
//! Specification Type 2 requirements for UKI boot entries, including extraction of boot
//! labels from os-release information embedded in the UKI binary.

use thiserror::Error;
use zerocopy::{
little_endian::{U16, U32},
Expand Down
7 changes: 7 additions & 0 deletions crates/composefs-boot/src/write_boot.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
//! Boot entry writing and installation functionality.
//!
//! This module provides functions to write boot entries to the filesystem, handling both
//! Boot Loader Specification Type 1 entries (separate kernel/initrd files) and Type 2
//! Unified Kernel Images. It manages file placement, directory creation, and command line
//! argument injection for composefs boot scenarios.

use std::{
fs::{create_dir_all, write},
path::Path,
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs-fuse/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! FUSE filesystem implementation for composefs trees.
//!
//! This crate provides a userspace filesystem implementation that exposes composefs
//! directory trees through FUSE. It supports read-only access to files, directories,
//! symlinks, and extended attributes, with data served from a composefs repository.

use std::{
collections::HashMap,
ffi::OsStr,
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs-http/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! HTTP-based download functionality for composefs splitstreams and objects.
//!
//! This crate provides an asynchronous downloader that can fetch splitstreams and their
//! referenced objects from HTTP servers. It handles recursive fetching of nested splitstream
//! references and verifies content integrity using fsverity checksums.

use std::{
collections::{HashMap, HashSet},
fs::File,
Expand Down
10 changes: 10 additions & 0 deletions crates/composefs-oci/src/image.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,13 @@
//! OCI image processing and filesystem construction.
//!
//! This module handles the conversion of OCI container image layers into composefs filesystems.
//! It processes tar entries from container layers, handles overlayfs semantics like whiteouts,
//! and constructs the final filesystem tree that can be mounted or analyzed.
//!
//! The main functionality centers around `create_filesystem()` which takes an OCI image configuration
//! and builds a complete filesystem by processing all layers in order. The `process_entry()` function
//! handles individual tar entries and implements overlayfs whiteout semantics for proper layer merging.

use std::{ffi::OsStr, os::unix::ffi::OsStrExt, rc::Rc};

use anyhow::{ensure, Context, Result};
Expand Down
12 changes: 12 additions & 0 deletions crates/composefs-oci/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,15 @@
//! OCI container image support for composefs.
//!
//! This crate provides functionality for working with OCI (Open Container Initiative) container images
//! in the context of composefs. It enables importing, extracting, and mounting container images as
//! composefs filesystems with fs-verity integrity protection.
//!
//! Key functionality includes:
//! - Pulling container images from registries using skopeo
//! - Converting OCI image layers from tar format to composefs split streams
//! - Creating mountable filesystems from OCI image configurations
//! - Sealing containers with fs-verity hashes for integrity verification

pub mod image;
pub mod skopeo;
pub mod tar;
Expand Down
10 changes: 10 additions & 0 deletions crates/composefs-oci/src/skopeo.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,13 @@
//! Container image pulling and registry interaction via skopeo/containers-image-proxy.
//!
//! This module provides functionality to pull container images from various registries and import them
//! into composefs repositories. It uses the containers-image-proxy library to interface with skopeo
//! for image operations, handling authentication, transport protocols, and image manifest processing.
//!
//! The main entry point is the `pull()` function which downloads an image, processes its layers
//! asynchronously with parallelism control, and stores them in the composefs repository with proper
//! fs-verity integration. It supports various image formats and compression types.

use std::{cmp::Reverse, process::Command, thread::available_parallelism};

use std::{iter::zip, sync::Arc};
Expand Down
11 changes: 11 additions & 0 deletions crates/composefs-oci/src/tar.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,14 @@
//! TAR archive processing and split stream conversion.
//!
//! This module handles the conversion of tar archives (container image layers) into composefs split streams.
//! It provides both synchronous and asynchronous tar processing, intelligently deciding whether to store
//! file content inline in the split stream or externally in the object store based on file size.
//!
//! Key components include the `split()` and `split_async()` functions for converting tar streams,
//! `get_entry()` for reading back tar entries from split streams, and comprehensive support for
//! tar format features including GNU long names, PAX extensions, and various file types.
//! The `TarEntry` and `TarItem` types represent processed tar entries in composefs format.

use std::{
cell::RefCell,
collections::BTreeMap,
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs-setup-root/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Root filesystem setup utility for composefs-based boot systems.
//!
//! This utility is designed to run during early boot to mount and configure
//! the root filesystem using composefs images. It handles overlay mounts for
//! writable directories, state management, and system integration.

use std::{
ffi::OsString,
fmt::Debug,
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/dumpfile.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Writing composefs dumpfile format from filesystem trees.
//!
//! This module provides functionality to serialize filesystem trees into
//! the composefs dumpfile text format, handling file metadata, extended
//! attributes, and hardlink tracking.

use std::{
collections::HashMap,
ffi::{OsStr, OsString},
Expand Down
5 changes: 5 additions & 0 deletions crates/composefs/src/erofs/composefs.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
//! Composefs-specific EROFS structures and overlay metadata.
//!
//! This module defines EROFS structures specific to composefs usage,
//! particularly overlay metadata for fs-verity integration.

use zerocopy::{FromBytes, Immutable, IntoBytes, KnownLayout};

use crate::fsverity::FsVerityHashValue;
Expand Down
5 changes: 5 additions & 0 deletions crates/composefs/src/erofs/debug.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
//! Debug utilities for analyzing EROFS images.
//!
//! This module provides tools for inspecting and debugging EROFS filesystem
//! images, including detailed structure dumping and space usage analysis.

use std::{
cmp::Ordering,
collections::BTreeMap,
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/erofs/format.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! EROFS on-disk format definitions and data structures.
//!
//! This module defines the binary layout of EROFS filesystem structures
//! including superblocks, inodes, directory entries, and other metadata
//! using safe zerocopy-based parsing.

// This is currently implemented using zerocopy but the eventual plan is to do this with safe
// transmutation. As such: all of the structures are defined in terms of pure LE integer sizes, we
// handle the conversion to enum values separately, and we avoid the TryFromBytes trait.
Expand Down
5 changes: 5 additions & 0 deletions crates/composefs/src/erofs/mod.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
//! EROFS (Enhanced Read-Only File System) format support for composefs.
//!
//! This module provides functionality to read and write EROFS filesystem images,
//! which are used as the underlying storage format for composefs images.

pub mod composefs;
pub mod debug;
pub mod format;
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/erofs/reader.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! EROFS image reading and parsing functionality.
//!
//! This module provides safe parsing and navigation of EROFS filesystem
//! images, including inode traversal, directory reading, and object
//! reference collection for garbage collection.

use core::mem::size_of;
use std::collections::{BTreeSet, HashSet};
use std::ops::Range;
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/erofs/writer.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! EROFS image generation and writing functionality.
//!
//! This module provides functionality to generate EROFS filesystem images
//! from composefs tree structures, handling inode layout, directory blocks,
//! and metadata serialization.

use std::{
cell::RefCell,
collections::{BTreeMap, HashMap},
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/filesystem_ops.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! High-level filesystem operations for composefs trees.
//!
//! This module provides convenience methods for common operations on
//! FileSystem objects, including computing image IDs, committing to
//! repositories, and generating dumpfiles.

use anyhow::Result;

use crate::{
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/fs.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Reading and writing filesystem trees to/from disk.
//!
//! This module provides functionality to read filesystem structures from
//! disk into composefs tree representations and write them back, including
//! handling of hardlinks, extended attributes, and repository integration.

use std::{
cell::RefCell,
collections::{BTreeMap, HashMap},
Expand Down
5 changes: 5 additions & 0 deletions crates/composefs/src/fsverity/digest.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
//! Userspace fs-verity digest computation.
//!
//! This module implements the fs-verity Merkle tree algorithm in userspace,
//! allowing computation of fs-verity digests without kernel support.

use core::{cmp::min, mem::size_of};

use sha2::Digest;
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/fsverity/hashvalue.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Hash value types and trait definitions for fs-verity.
//!
//! This module defines the FsVerityHashValue trait and concrete implementations
//! for SHA-256 and SHA-512 hash values, including parsing from hex strings
//! and object pathnames.

use core::{fmt, hash::Hash};

use hex::FromHexError;
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/fsverity/ioctl.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Low-level ioctl interfaces for fs-verity kernel operations.
//!
//! This module provides safe wrappers around the Linux fs-verity ioctls
//! for enabling and measuring fs-verity on files, handling the conversion
//! between kernel and userspace data structures.

#![allow(unsafe_code)]

use core::mem::size_of;
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/fsverity/mod.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Linux fs-verity support for integrity verification.
//!
//! This module provides complete fs-verity functionality including userspace
//! digest computation, kernel ioctl interfaces for enabling and measuring
//! verity, and hash value types for SHA-256 and SHA-512.

mod digest;
mod hashvalue;
mod ioctl;
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Rust bindings and utilities for working with composefs images and repositories.
//!
//! Composefs is a read-only FUSE filesystem that enables efficient sharing
//! of container filesystem layers by using content-addressable storage
//! and fs-verity for integrity verification.

pub mod dumpfile;
pub mod dumpfile_parse;
pub mod erofs;
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/mount.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Modern Linux mount API support for composefs.
//!
//! This module provides functionality to mount composefs images using the
//! new mount API (fsopen/fsmount) with overlay filesystem support and
//! fs-verity verification.

use std::{
io::Result,
os::fd::{AsFd, BorrowedFd, OwnedFd},
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/mountcompat.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Compatibility helpers for older Linux kernel mount APIs.
//!
//! This module provides fallback implementations for mount operations
//! on kernels that don't support the modern mount API, including
//! loopback device setup and temporary mount handling.

use std::{
io::Result,
os::fd::{AsFd, BorrowedFd, OwnedFd},
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/repository.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Content-addressable repository for composefs objects.
//!
//! This module provides a repository abstraction for storing and retrieving
//! content-addressed objects, splitstreams, and images with fs-verity
//! verification and garbage collection support.

use std::{
collections::HashSet,
ffi::CStr,
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/splitstream.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Split Stream file format implementation.
//!
//! This module implements the Split Stream format for efficiently storing
//! and transferring data with inline content and external object references,
//! supporting compression and content deduplication.

/* Implementation of the Split Stream file format
*
* See doc/splitstream.md
Expand Down
6 changes: 6 additions & 0 deletions crates/composefs/src/util.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Utility functions and types used throughout the composefs crate.
//!
//! This module provides common functionality including error handling helpers,
//! I/O utilities for reading data streams, SHA256 digest parsing, and
//! filesystem operations like atomic symlink replacement.

use rand::{distr::Alphanumeric, Rng};
use std::{
io::{Error, ErrorKind, Read, Result},
Expand Down
6 changes: 6 additions & 0 deletions crates/erofs-debug/src/main.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
//! Debug utility for analyzing EROFS filesystem images.
//!
//! This tool produces detailed, diff-friendly dumps of EROFS images that can be used
//! to examine the internal structure and identify differences between images.
//! The output format is deterministic and suitable for automated comparison.

use std::{fs::File, io::Read, path::PathBuf};

use clap::Parser;
Expand Down
Loading