Skip to content

Rainwalker-Hello/rPickle

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

rPickle

A safe and efficient Python serialization library.

Features

  • 🔒 Safe - No arbitrary code execution, unlike pickle
  • 📦 Compact - Small serialized size
  • 🔄 Circular references - Handles self-referential structures
  • 🎨 Extensible - Custom type support via extensions

🔒 Safety

rPickle is safe in the sense that it does not execute arbitrary code during deserialization.

Unlike pickle (which can call arbitrary functions during loads()), rPickle only reconstructs data structures.
It does not use __reduce__(), no implicit imports, and no function calls.

rPickle.loads(data) does not execute any code — it only restores data.
pickle.loads(data) may execute arbitrary code defined in __reduce__.

However, safety is not absolute:

  • If you use custom extensions (Extension), you are responsible for the code you provide — the library author is not liable.
  • rPickle does not encrypt data or prevent tampering. If you need integrity or confidentiality, take additional measures.
  • rPickle is not a sandbox. If you load untrusted data with malicious extensions, it can execute arbitrary code.

It is recommended to only load data from trusted sources and review any custom extensions before use.

Requirements

  • Python 3.15+ (preview)
  • Python 3.11+ (full)

Installation

pip install rPickle

Quick Start

import rPickle

# Serialize
data = {'name': 'Alice', 'scores': [95, 87, 92]}
packed = rPickle.dumps(data)

# Deserialize
restored = rPickle.loads(packed)
print(restored)  # {'name': 'Alice', 'scores': [95, 87, 92]}

API

Core Function

Function Description
dumps(obj) Serialize object to bytes
loads(data) Deserialize from bytes
dump(obj, file) Serialize to file
load(file) Deserialize from file

Extensions

You can use built-in extensions to support additional types like datetime, Decimal, UUID, etc.

For example, to serialize a dictionary containing a datetime object:

from datetime import datetime

data = {'created': datetime.now()}
packed = rPickle.dumps(data, extensions=rPickle.ext.datetime_ext)
restored = rPickle.loads(packed, extensions=rPickle.ext.datetime_ext)

# Using more extensions at the same time
exts = rPickle.ext.datetime_ext | rPickle.ext.Path_ext | rPickle.ext.Decimal_ext
packed = rPickle.dumps(data, extensions=exts)

Custom Extensions

Add support for your own types using the extensions parameter. For example, to support datetime.datetime:

import rPickle
from datetime import datetime

# 1. Define dump function (type → bytes)
def dump_datetime(dt: datetime) -> bytes:
    return dt.timestamp().to_bytes(8, 'little')

# 2. Define load function (bytes → type)
def load_datetime(data: bytes) -> datetime:
    timestamp = int.from_bytes(data, 'little')
    return datetime.fromtimestamp(timestamp)

# 3. Register your extension
my_extensions = rPickle.ext.Extension(typ=datetime, load_func=load_datetime, dump_func=dump_datetime)

# 4. Use it
data = {'created': datetime.now()}
packed = rPickle.dumps(data, extensions=my_extensions)
restored = rPickle.loads(packed, extensions=my_extensions)

You can also combine your extension with built-in ones or custom ones using |=. For example, to combine your datetime extension with the built-in one:

import rPickle
from datetime import datetime

def dump_datetime(dt: datetime) -> bytes:
    return dt.timestamp().to_bytes(8, 'little')

def load_datetime(data: bytes) -> datetime:
    timestamp = int.from_bytes(data, 'little')
    return datetime.fromtimestamp(timestamp)

my_extensions = rPickle.ext.Extension(typ=datetime, load_func=load_datetime, dump_func=dump_datetime)

# Add a built-in extension or others
my_extensions |= rPickle.ext.datetime_ext

packed = rPickle.dumps(data, extensions=my_extensions)
restored = rPickle.loads(packed, extensions=my_extensions)

⚠️Note: For safety, avoid using eval(), exec(), or __import__() in your custom extensions.

Overriding Built-in Types With Custom Extensions

You can override built-in types with your own extensions. For example, if you want to change how list is serialized:

import rPickle
from rPickle.ext import Extension

def dump_list_plus(lst: list) -> bytes:
    return bytes(lst)

def load_list_plus(data: bytes) -> list:
    return list(data)

list_plus = Extension(typ=list, load_func=load_list_plus, dump_func=dump_list_plus)

data = [1, 2, 5, 3, 4, 5, 8, 9, 7, 5, 9, 5]
packed = rPickle.dumps(data, extensions=list_plus)
restored = rPickle.loads(packed, extensions=list_plus)

⚠️Note: For safety, avoid using eval(), exec(), or __import__() in your enhancement extensions

Supported Types

  • None, bool, int, float, complex, str
  • bytes, bytearray
  • list, tuple, set, frozenset
  • dict, frozendict (Python 3.15+)
  • range, slice
  • Ellipsis, NotImplemented
  • sentinel (Python 3.15+)
  • Custom types

Built-in Extensions

rPickle comes with ready-to-use extensions for common types: see the list below.

Name Type
datetime_ext datetime.datetime
Decimal_ext decimal.Decimal
UUID_ext uuid.UUID
Fraction_ext fractions.Fraction
Path_ext pathlib.Path
defaultdict_ext collections.defaultdict
date_ext datetime.date
time_ext datetime.time

Links

License

MIT

About

A safe and efficient Python serialization library.

Resources

License

Stars

1 star

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages