markdown-body entry-content container-lg" itemprop="text">

Relational File System, ongoing development version

Overview

The Relational File System (RelFS) is a proposed new way to store files and represent knowledge. Although the hierarchical file model is ubiquitous, RelFS is intended complement or replace hierarchies as the preferred way to organize files.

RelFS combines features from classic hierarchical file systems, relational databases, and content-addressable storage systems. Viewed in another way, RelFS takes the hierarchical model and removes undesirable features and limitations. Then it adds new features to support desirable use cases.

The RelFS proposal defines:

The concept of immutable, hash-addressed files
The basic operations supported by file repositories
How to define schemas to represent metadata
How metadata values are serialized to / deserialized from bytes
Sketches of practical metadata and practical queries

The RelFS proposal does not cover:

The meaning of metadata types and values
What application should read/write what metadata
Repository storage formats and low-level details like file packing, fragmentation, delta encoding, indexes, etc.
Multi-user concerns such as file permissions, storage quotas, etc.
Administrative details such as replication, fault handling, etc.
Support for file change notifications, publish/subscribe, etc.

Background and motivation

The RelFS proposal was born out of Nayuki's frustration with trying to organize personal files into hierarchies. The most general way to store information is the hierarchy of directories and files, and this mechanism is what every major operating system provides. However, this ubiquitous hierarchical file model is full of undesirable features:

Contrast with hierarchical file systems
Names suck, paths suck, length limitations, no structure (e.g. integer, fields)
General metadata formats are brittle
Brittle references
Dependent metadata
Poor unions
Poor deduplication
Poor sharing
Poor propietary media libraries
Different metadata per file format
Nayuki's first article on this topic published in Mar 2017: Designing better file organization around tags, not hierarchies
Inspiration from Git version control, Danbooru image board, BitTorrent file sharing, eDonkey2000 hash links, IPFS, Perkeep (formerly Camlistore), relational database model

RelFS main concepts

Files are plain blobs
Files have no inherent name
Files are immutable
Files are addressed by hash
Metadata can reference nonexistent files
Metadata can have metadata
Everything is a file
Location is independent of queries

Notes about RelFS

Repositories are mutable
Versioning has many possible incarnations
File system implementation can store more copies
Can implement stuff like file permissions, even hierarchies
Interfaces poorly with path-based systems, but also these systems don't take advantage of hash features
Is not a graph database, but is more general

License

Licensed as Creative Commons Attribution-ShareAlike 4.0 (CC BY-SA 4.0).

nayuki/Relational-File-System

About

Resources

Stars

Watchers

Forks

Releases

Packages 0

Packages