MSCS — Safe Serialization for Python

v2.5.0 | Changelog | PyPI

Status: Beta — API is stable but the format may evolve. Not yet battle-tested in large-scale production.

A secure, fast, binary serialization library. Drop-in replacement for pickle that does not execute arbitrary code during deserialization of unregistered classes.

Built for AI/ML workflows — native support for NumPy arrays and PyTorch tensors with zero-copy performance.

Why not pickle?

# pickle: arbitrary code execution on load
data = pickle.loads(untrusted_bytes)  # can run os.system("rm -rf /")

# mscs: only reconstructs explicitly registered classes
data = mscs.loads(untrusted_bytes)    # MSCSecurityError if class not registered

Comparison with Alternatives

Feature	mscs	pickle	safetensors	torch.save
No arbitrary code execution	Partial*	No	Yes	No
HMAC authentication	Yes	No	No	No
Custom class support	Yes (registry)	Yes	No	Yes
NumPy arrays	Yes	Yes	Yes	Yes
PyTorch tensors	Yes	Yes	Yes	Yes
Circular references	Yes	Yes	No	Yes
Zero dependencies	Yes	Yes	Yes (Rust)	No
Compression built-in	Yes (zlib)	No	No	No

* mscs executes __setstate__ on registered classes. See Security Model for details.

When to use safetensors instead: If you only need to serialize tensors and arrays (model weights, embeddings), safetensors is the industry standard — it's written in Rust, truly zero-code-execution, and widely adopted. Use mscs when you need to serialize mixed Python objects (configs, custom classes, nested structures) alongside tensors.

Install

pip install mscs              # core (no dependencies)
pip install mscs[numpy]       # + numpy support
pip install mscs[torch]       # + numpy + PyTorch tensor support
pip install mscs[all]         # everything

Quick Start

import mscs

# Primitives, collections, nested structures — just works
data = {"model": "v5.2", "lr": 0.001, "layers": [64, 128, 256]}
encoded = mscs.dumps(data)
decoded = mscs.loads(encoded)

# NumPy arrays
import numpy as np
arr = np.random.randn(100, 100).astype(np.float32)
encoded = mscs.dumps(arr)

# PyTorch tensors — no .numpy() conversion needed
import torch
weights = torch.randn(256, 256)
encoded = mscs.dumps(weights)  # safe, no pickle involved

# Full model checkpoints
checkpoint = {
    "epoch": 100,
    "model_state": {k: v for k, v in model.state_dict().items()},
    "optimizer_lr": 0.0003,
}
mscs.dump(checkpoint, open("checkpoint.mscs", "wb"))
restored = mscs.load(open("checkpoint.mscs", "rb"))

Custom Classes

import mscs
from dataclasses import dataclass

@mscs.register
@dataclass
class Config:
    state_size: int = 256
    lr: float = 0.001

config = Config(512, 0.0003)
data = mscs.dumps(config)
restored = mscs.loads(data)  # Config(state_size=512, lr=0.0003)

# Unregistered classes raise MSCSecurityError in strict mode
mscs.loads(data_with_unknown_class)  # MSCSecurityError

# Or get a dict fallback in non-strict mode
mscs.loads(data_with_unknown_class, strict=False)  # {'__class__': '...', '__state__': {...}}

Backward Compatibility with Renamed Classes

mscs.register_alias("my_module.OldConfig", Config)

Register All Classes in a Module

import my_models
mscs.register_module(my_models)

Compression & Integrity

# zlib compression
with open("data.mscs.z", "wb") as f:
    mscs.dump_compressed(large_obj, f)

with open("data.mscs.z", "rb") as f:
    obj = mscs.load_compressed(f)

# CRC32 integrity check (detects accidental corruption, NOT tamper-proof)
data = mscs.dumps(obj, with_crc=True)
mscs.loads(data)  # verifies CRC, raises MSCDecodeError if corrupted

# HMAC-SHA256 authentication (cryptographic, tamper-proof)
key = b'your-secret-key-here'
data = mscs.dumps(obj, hmac_key=key)
mscs.loads(data, hmac_key=key)          # verifies HMAC, raises MSCSecurityError if tampered
mscs.loads(data)                         # MSCSecurityError: no key provided for signed payload
mscs.loads(unsigned_data, hmac_key=key)  # MSCSecurityError: anti-downgrade protection

API Reference

Core

Function	Description
dumps(obj, *, with_crc=False, hmac_key=None) -> bytes	Serialize to bytes
loads(data, *, strict=True, hmac_key=None) -> Any	Deserialize from bytes
dump(obj, file, *, with_crc=False, hmac_key=None)	Serialize to file (binary mode)
load(file, *, strict=True, hmac_key=None) -> Any	Deserialize from file
dump_compressed(obj, file, level=6)	Serialize with zlib compression
load_compressed(file) -> Any	Deserialize compressed data

Registry

Function	Description
register(cls) -> cls	Register class as safe (also works as decorator)
register_alias(old_path, cls)	Map old class path to new class
register_module(module) -> list	Register all classes in a module

Utilities

Function	Description
inspect(data) -> dict	Get metadata without deserializing
benchmark(obj, rounds=100) -> dict	Measure encode/decode performance
copy(obj) -> obj	Deep copy via serialization round-trip

Supported Types

Type	Notes
None, bool, int, float, complex	Ints up to 8192 bytes (~19,700 digits)
str, bytes, bytearray	UTF-8, ref-tracked
list, tuple, dict, set, frozenset, deque	Circular refs supported (including cycles through tuples); deque preserves maxlen
datetime, date, time, timedelta	ISO 8601
Decimal, UUID, Path	Lossless
Enum	Must be registered
numpy.ndarray	dtype whitelist enforced
torch.Tensor	Auto CPU transfer, preserves requires_grad
dataclass (incl. frozen), __slots__, __dict__ objects	Must be registered; circular refs supported (self/mutual references, cycles through tuples). __slots__ handled in full: string form, inheritance, private (name-mangled) slots, and hybrid slots+__dict__ classes. One exception fails closed: a custom __setstate__ whose state contains a reference to a tuple still under construction raises MSCDecodeError — break such cycles through a mutable container

Performance

Benchmarked on a single machine (results may vary by hardware and payload):

Payload: state_dict with 4 tensors (~57K parameters, dominated by contiguous float32 buffers):

Method	Roundtrip	Size
mscs	~0.1 ms	~65 KB
pickle	~0.6 ms	~68 KB
torch.save	~0.4 ms	~67 KB

mscs is fast for tensor-heavy payloads because it writes raw buffers with minimal framing overhead. For small, nested Python structures (dicts, strings, configs), the speedup is smaller. Always benchmark with your actual data.

Run python tests/benchmark.py to reproduce on your machine.

Security Model

mscs provides a defense-in-depth approach, but it is not a sandbox. Understand the boundaries:

What mscs prevents

1. No dynamic imports: Class names in the binary stream are only used as registry lookup keys — never passed to importlib
2. Explicit registry: Custom classes must be registered before deserialization; unregistered classes raise MSCSecurityError
3. NumPy dtype whitelist: Blocks object, void, and structured dtypes that could execute code
4. Total-size and depth limits: loads()/load() reject any blob larger than max_size (default MAX_SIZE=512MB) before decoding — this bounds the sum of all fields, not just each field individually (every decoded string/collection lives inside the input). The limit is on the whole encoded blob (data + framing), so the largest round-trippable single field is slightly below max_size; pass a larger max_size to load trusted data at that edge. load() reads at most max_size+1 bytes, so an oversized file is never materialized. Other per-field caps: MAX_COLLECTION=10M elements per collection, MAX_STRING=100MB per string/bytes, MAX_INT_BYTES=8192, MAX_DEPTH=256 nesting. Peak memory is a multiple of max_size (Python object overhead is ~6× for tiny primitives, plus transient copies during decompression), so set max_size conservatively for untrusted input — e.g. loads(data, max_size=50*1024*1024)
5. Anti zip-bomb: load_compressed bounds both the compressed read and the decompressed output to max_size (the compressed read additionally capped by MAX_COMPRESSED) and decompresses incrementally, aborting the moment the output crosses the limit — a bomb cannot exhaust memory before the size check, and lowering max_size lowers peak memory on both sides
6. Compressed-container integrity: load_compressed verifies the zlib stream ended cleanly (checksum validated), rejects trailing bytes and concatenated streams, and checks the decompressed length against the declared header — a truncated, forged, or padded container fails closed with MSCDecodeError instead of yielding partial data
7. Path null byte rejection: Paths containing null bytes are rejected
8. CRC32 corruption detection: Optional checksum to detect accidental data corruption (not cryptographic — an attacker can forge CRC32)
9. HMAC-SHA256 authentication: Optional cryptographic signature to detect intentional tampering. Anti-downgrade protection rejects any payload lacking a valid HMAC when a key is supplied — including the legacy v1 format — so an attacker cannot strip or version-downgrade the signature.
10. Trailing bytes rejection: Payloads with unexpected bytes after the serialized object are rejected
11. Integer size limit: Ints larger than MAX_INT_BYTES (8192 bytes, ~19,700 digits) are rejected to prevent CPU exhaustion attacks
12. Forged forward-ref rejection: Payloads referencing a container still under construction from an unpatchable position (dict key, set/frozenset member, self-referential root tuple) fail closed with MSCDecodeError — and loads() verifies no unresolved placeholder survives decoding. The encoder can never produce these shapes

What mscs does NOT prevent

1. __setstate__ execution: If you register a class that implements __setstate__, that method will execute during deserialization. Only register classes you trust.
2. Path traversal: Deserialized Path objects may contain ../ sequences. The consumer must validate paths before using them for file I/O.
3. Malicious registered classes: The security boundary is the registry. If you register a class with dangerous behavior in __init__, __setstate__, or property setters, mscs cannot protect you.
4. Bounded-but-not-tiny memory: the size limit bounds the input, but peak memory is a multiple of it — decoding expands compact bytes into Python objects (~6× for tiny primitives), and decompression holds transient copies. Budget for a few times max_size, and lower max_size for untrusted input.

Configuring limits

Pass max_size / max_depth per call — these are the supported, thread-safe knobs:

obj = mscs.loads(untrusted, max_size=50 * 1024 * 1024, max_depth=64)
mscs.dumps(obj, max_depth=64)

Note: rebinding the module constant (mscs.MAX_DEPTH = 1) has no effect — it is a re-exported name, not the global the encoder/decoder read. Use the parameters above.

Rule of thumb: mscs is safe for deserializing untrusted data as long as your registry only contains trusted classes and you set max_size to fit your memory budget.

Binary Format

┌──────────┬─────────┬───────┬──────────┬────────────────┬──────────────┬──────────────┐
│ Magic(4) │ Ver.(1) │ Fl(1) │ Tag(1)   │ Payload(var)   │ CRC32(4)?    │ HMAC(32)?    │
│ "MSCS"   │ 0x02    │ bits  │ type tag │ type-dependent  │ if flag 0x01 │ if flag 0x02 │
└──────────┴─────────┴───────┴──────────┴────────────────┴──────────────┴──────────────┘

Header (6 bytes fixed):

• Bytes 0-3: Magic MSCS (0x4D534353)
• Byte 4: Format version (currently 0x02)
• Byte 5: Flags (bit 0 = CRC32 appended, bit 1 = HMAC-SHA256 appended)

CRC32 and HMAC are mutually exclusive (HMAC is strictly superior).

Payload: Recursive type-length-value encoding. Each value starts with a 1-byte type tag:

Tag	Type	Payload format
0x00	None	(empty)
0x01	bool	1 byte (0x00/0x01)
0x02	int	<H> byte count + signed little-endian bytes
0x03	float	<d> IEEE 754 double
0x04	str	<I> byte count + UTF-8
0x05	bytes	<I> byte count + raw
0x06	list	<I> item count + items
0x07	tuple	<I> item count + items
0x08	dict	<I> pair count + key/value pairs
0x09	set	<I> item count + items (sorted)
0x0A	ndarray	str(meta) + <I> data size + raw buffer
0x0B	object	str(class_path) + encoded(state)
0x0C	complex	<dd> real, imag
0x0D	frozenset	<I> item count + items (sorted)
0x0E	datetime	<H> str len + ISO 8601 string
0x0F	date	<HBB> year, month, day
0x10	time	<H> str len + ISO 8601 string
0x11	timedelta (legacy)	<iiI> days, seconds, microseconds
0x12	Decimal	<H> str len + decimal string
0x13	Enum	str(class_path) + encoded(value)
0x14	bytearray	<I> byte count + raw
0x15	ref	<I> reference ID
0x16	UUID	16 bytes raw
0x17	Path	<I> str len + UTF-8 path string
0x18	Tensor	str(meta) + <I> data size + raw buffer
0x19	timedelta2	<iiI> days, seconds, microseconds
0x1A	deque	<i> maxlen (-1 if None) + <I> item count + items

ndarray meta: "{dtype}|{shape}" where shape is "dim0xdim1x..." (e.g., "float32|100x100").

Tensor meta: "{dtype}|{shape}|{requires_grad}" (e.g., "float32|256x256|0").

Reference tracking: Mutable containers (list, dict, set, etc.), strings, bytes, and arrays are assigned incrementing IDs. Tag 0x15 refers back to a previously seen object by ID, enabling circular reference support.

License

MIT