Skip to content

How To Replicate

Jump to bottom
NtinosTheGamer2324 edited this page Feb 11, 2026 · 1 revision

MDFS – How to Replicate / Port (Reference Guide)

This page is a practical guide for implementing an MDFS (ModularFS) reader/writer in another environment:

  • your own custom OS
  • a Linux userspace tool (FUSE-style)
  • a Windows driver (or parsing library)
  • a BSD driver

The goal is to document what is on disk and the minimum logic needed to interoperate.

MDFS is inspired by EXT2 (inodes + bitmaps + inode table layout) and exFAT (directory entry sets + checksums).

0) Quick facts (v2)

From include/moduos/fs/MDFS/mdfs.h:

  • Magic: MDFS_MAGIC = 0x5346444D (ASCII "MDFS" little-endian)
  • Version: MDFS_VERSION = 2
  • Block size: MDFS_BLOCK_SIZE = 4096
  • Inode size: MDFS_INODE_SIZE = 256
  • Max direct blocks: MDFS_MAX_DIRECT = 12
  • Directory record size: MDFS_DIR_REC_SIZE = 32
  • Max name bytes: MDFS_MAX_NAME = 255 (UTF-8 bytes)

1) On-disk layout

1.1) Canonical on-disk structs (C reference)

These structs are the authoritative reference for MDFS v2. They are copied from include/moduos/fs/MDFS/mdfs.h.

#define MDFS_MAGIC 0x5346444Du /* 'MDFS' little-endian */
#define MDFS_VERSION 2
#define MDFS_BLOCK_SIZE 4096u
#define MDFS_INODE_SIZE 256u
#define MDFS_MAX_DIRECT 12u
#define MDFS_MAX_NAME 255u

#define MDFS_DIR_REC_SIZE 32u
#define MDFS_DIRREC_PRIMARY 1u
#define MDFS_DIRREC_NAME    2u

#define MDFS_DIRFLAG_VALID   0x01u
#define MDFS_DIRFLAG_DELETED 0x02u

typedef struct __attribute__((packed)) {
    uint16_t mode;        // 0x4000 dir, 0x8000 file
    uint16_t _pad0;
    uint32_t uid;
    uint32_t gid;
    uint64_t size_bytes;
    uint32_t link_count;
    uint32_t flags;
    uint64_t direct[MDFS_MAX_DIRECT];
    uint64_t indirect1;
    uint8_t  _pad[MDFS_INODE_SIZE - 2 - 2 - 4 - 4 - 8 - 4 - 4 - (8*MDFS_MAX_DIRECT) - 8];
} mdfs_inode_t;

typedef struct __attribute__((packed)) {
    uint8_t  rec_type;      // MDFS_DIRREC_PRIMARY
    uint8_t  flags;         // MDFS_DIRFLAG_*
    uint8_t  entry_type;    // 1=file,2=dir
    uint8_t  record_count;  // total records in entry set (including this primary)
    uint32_t inode;
    uint16_t name_len;      // UTF-8 bytes
    uint16_t _rsv0;
    uint32_t checksum;      // CRC32 over entry set with this field zero
    uint8_t  _pad[32 - 1 - 1 - 1 - 1 - 4 - 2 - 2 - 4];
} mdfs_dir_primary_t;

typedef struct __attribute__((packed)) {
    uint8_t rec_type; // MDFS_DIRREC_NAME
    uint8_t name_bytes[31];
} mdfs_dir_name_t;

typedef struct __attribute__((packed)) {
    uint32_t magic;
    uint32_t version;
    uint32_t block_size;
    uint32_t _reserved0;

    uint64_t total_blocks;
    uint64_t free_blocks;
    uint64_t total_inodes;
    uint64_t free_inodes;

    uint64_t block_bitmap_start;
    uint64_t block_bitmap_blocks;
    uint64_t inode_bitmap_start;
    uint64_t inode_bitmap_blocks;
    uint64_t inode_table_start;
    uint64_t inode_table_blocks;

    uint64_t root_inode;

    uint8_t  uuid[16];
    uint32_t features;
    uint32_t checksum; /* CRC32 over superblock with this field zero */

    uint8_t  pad[MDFS_BLOCK_SIZE - (4*4) - (6*8) - (1*8) - 16 - 4 - 4];
} mdfs_superblock_t;

MDFS is block-based. “Block N” means block index in units of 4096 bytes.

The superblock (block 1) tells you where metadata lives:

  • block_bitmap_start / block_bitmap_blocks
  • inode_bitmap_start / inode_bitmap_blocks
  • inode_table_start / inode_table_blocks
  • root_inode

Required reads to mount

  1. Read block 1 into mdfs_superblock_t.
  2. Validate:
    • magic == MDFS_MAGIC
    • version == 2
    • block_size == 4096
  3. Optionally validate superblock.checksum (CRC32) if non-zero.

2) CRC32 algorithm

Reference implementation (copied from src/fs/MDFS/mdfs_disk.c):

#include <stdint.h>
#include <stddef.h>

uint32_t mdfs_crc32_buf(const void *data, size_t len) {
    const uint8_t *p = (const uint8_t*)data;
    uint32_t crc = 0xFFFFFFFFu;
    for (size_t i = 0; i < len; i++) {
        uint32_t x = (crc ^ p[i]) & 0xFFu;
        for (int b = 0; b < 8; b++) {
            x = (x >> 1) ^ (0xEDB88320u & (-(int)(x & 1u)));
        }
        crc = (crc >> 8) ^ x;
    }
    return ~crc;
}

MDFS uses CRC32 with the common reversed polynomial 0xEDB88320.

Reference implementation exists in ModuOS:

  • src/fs/MDFS/mdfs_disk.c: mdfs_crc32_buf()

Usage:

  • Superblock checksum: CRC32 over the superblock with checksum field set to 0.
  • Directory entry set checksum: CRC32 over the full entry set with the primary’s checksum field set to 0.

3) Inodes

Inode location

C-like pseudocode:

// byte offset of inode inside inode table region
uint64_t byte_off = (uint64_t)ino * (uint64_t)MDFS_INODE_SIZE;
uint64_t block = sb->inode_table_start + (byte_off / MDFS_BLOCK_SIZE);
uint64_t off   = byte_off % MDFS_BLOCK_SIZE;

read_block(block, tmp4096);
memcpy(out_inode, tmp4096 + off, sizeof(mdfs_inode_t));

Inode ino is stored in the inode table region:

  • byte offset = ino * MDFS_INODE_SIZE
  • block = inode_table_start + (byte_off / 4096)
  • offset inside block = byte_off % 4096

See ModuOS reference:

  • src/fs/MDFS/mdfs_disk.c: mdfs_disk_read_inode()

Inode format (mdfs_inode_t)

Key fields:

  • mode: 0x4000 directory, 0x8000 regular file
  • size_bytes
  • direct[12]: block pointers
  • indirect1: reserved for future expansion

Current implementation supports direct blocks plus indirect1/2/3 (single/double/triple-indirect).

4) Allocation bitmaps

MDFS uses EXT2-like bitmaps.

Inode bitmap

  • bitmap is at block inode_bitmap_start
  • inode i is free if bit i is 0

Block bitmap

  • bitmap is at block block_bitmap_start
  • block b is free if bit b is 0

Note: ModuOS uses multi-block bitmap scanning via block_bitmap_blocks / inode_bitmap_blocks (see mdfs_api.c), allowing large volumes.

5) Directories (entry sets, exFAT-like)

Directories are files whose data blocks contain an array of 32-byte records.

Record types

  • Primary record: rec_type = MDFS_DIRREC_PRIMARY (1)
  • Name record: rec_type = MDFS_DIRREC_NAME (2)

Primary record (mdfs_dir_primary_t)

Fields of interest:

  • flags: includes VALID and DELETED
  • entry_type: 1=file, 2=dir
  • record_count: total record count (primary + name records)
  • inode: inode number
  • name_len: UTF-8 byte length
  • checksum: CRC32 over the entire entry set

Name records (mdfs_dir_name_t)

  • each name record stores 31 bytes of UTF-8 payload
  • long names are concatenated across multiple name records

Parsing a directory

Algorithm (compatible with ModuOS mdfs_dir.c):

  1. Read the directory inode.
  2. For each direct block pointer in direct[]:
    • read block
    • walk records at offsets 0..4096 in steps of 32
    • if rec_type == 0, stop (end-of-directory marker)
    • if record is not a valid primary, skip using record_count when possible
    • when you find a valid primary:
      • verify checksum (optional but recommended)
      • read record_count-1 name records and reconstruct UTF-8 name

Lookup

Compare the reconstructed name with the query name (case-sensitive in current ModuOS logic).

Deletion

ModuOS marks entries as deleted by setting the DELETED flag and rewriting the checksum.

6) Path resolution

MDFS path resolution is hierarchical:

  • start at root_inode
  • split /a/b/c into components
  • for each component:
    • look up entry in current directory
    • ensure it’s a directory if there are remaining components

See ModuOS reference:

  • src/fs/MDFS/mdfs_api.c: mdfs_lookup_path()

7) Reading files

  1. Resolve path to inode.
  2. Read inode to get size_bytes and direct blocks.
  3. Read blocks in order until you have size_bytes.

Current ModuOS implementation reads using direct[] plus indirect1/2/3 via the mdfs_get_block_ptr() helper (single/double/triple-indirect, EXT2-style).

8) Writing files (ModuOS behavior)

ModuOS supports both whole-file overwrite and offset writes:

  • mdfs_write_file_by_path() (offset 0)
  • mdfs_write_file_at_by_path() (offset-aware, used by the FD layer)

ModuOS mdfs_write_file_at_by_path() does:

  • resolves parent directory + basename
  • if file exists:
    • overwrites file contents in-place (allocating blocks as needed)
  • if file does not exist:
    • allocates inode
    • adds directory entry set
    • allocates blocks and writes data

Important notes:

  • unlink/rmdir do free previously allocated blocks (direct + indirect) in the current implementation.
  • Truncation/block-reclamation on overwrite is partially implemented:
    • mdfs_create_file_trunc(..., truncate=1, ...) has a TODO to free blocks (so true truncation is not complete yet).
    • Normal writes support direct[] + indirect1/2/3 and include a fast sequential preallocation path (batched bitmap alloc + mdfs_set_block_ptr_range()).

If you’re implementing a driver elsewhere, decide whether you want to match this behavior or implement full block reclamation.

9) Minimal “portable” implementation plan

A) Read-only library (fastest)

  • Implement superblock read/verify
  • Implement inode read
  • Implement directory parse (entry sets)
  • Implement path lookup
  • Implement file read

This is enough for:

  • Linux userspace tool
  • FUSE read-only filesystem
  • Windows/BSD read-only driver

B) Read-write (more work)

Add:

  • bitmap allocation/free
  • inode write
  • directory entry set add/remove
  • data block write

And decide on crash-consistency strategy (MDFS is not journaled).

10) Tips for specific platforms

Linux (FUSE)

  • Create a FUSE filesystem that maps MDFS inodes to FUSE inodes.
  • Cache directory listings and inode reads for performance.
  • Implement readdir, lookup, read first.

Windows

  • Consider starting with a userspace parser (WinFsp) before writing a kernel driver.
  • If writing a kernel driver, be careful with paging/IRQL rules.

BSD

  • Implement as a VFS module; start with lookup/read-only.

11) Reference code in ModuOS

  • Structures: include/moduos/fs/MDFS/mdfs.h
  • Disk IO + CRC: src/fs/MDFS/mdfs_disk.c
  • Directory logic: src/fs/MDFS/mdfs_dir.c
  • Path + VFS-facing API: src/fs/MDFS/mdfs_api.c

12) Compatibility and versioning

  • This doc describes MDFS v2.
  • If you change on-disk layout/record formats, bump version and update this doc.

A small test image generator is included so external implementations have a known-good corpus.

Known-good corpus generator

Script:

  • tools/mdfs_make_test_image.py

It creates an MBR-partitioned disk image with a single MDFS v2 partition and a few deterministic sample files.

Run:

python tools/mdfs_make_test_image.py --size-mib 64 --out dist/mdfs_test.img

Sample contents:

  • /test.txt => MDFS OK\n
  • /hello/world.txt => Hello from ModularFS (MDFS)!\n
  • /big.bin => deterministic pattern (used for verifying reads)

This image is intended for driver/FUSE/parser bring-up.

Clone this wiki locally