-
Notifications
You must be signed in to change notification settings - Fork 1.3k
FileStorage
This documentation is valid only for
v4.x
version.
To keep its memory profile slim, LiteDB has a limited document size of 1Mb. For text documents, this is a huge size. But for many binary files, 1Mb is too small. LiteDB therefore implements FileStorage
, a custom collection to store files and streams.
LiteDB uses two special collections to split file content in chunks:
-
_files
collection stores file reference and metadata only
{
_id: "my-photo",
filename: "my-photo.jpg",
mimeType: "image/jpg",
length: { $numberLong: "2340000" },
uploadDate: { $date: "2015-01-01T00:00:00Z" },
metadata: { "key1": "value1", "key2": "value2" }
}
-
_chunks
collection stores binary data in 1MB chunks.
{
_id: "my-photo\00001",
data: { $binary: "VHlwZSAob3Igc ... GUpIGhlcmUuLi4" }
}
{
_id: "my-photo\00002",
data: { $binary: "pGaGhlcmUuLi4 ... VHlwZSAob3Igc" }
}
{
...
}
Files are identified by an _id
string value, with following rules:
- Starts with a letter, number,
_
,-
,$
,@
,!
,+
,%
,;
or.
- If contains a
/
, must be sequence with chars above
To better organize many files, you can use _id
as a directory/file_id
. This will be a great solution to quickly find all files in a directory using the Find
method. The mimetype
is derived from the filename.
Example: $/photos/2014/picture-01.jpg
The FileStorage
collection contains simple methods like:
-
Upload
: Send file or stream to database. Can be used with file orStream
. If file already exists, file content is overwritten. -
Download
: Get your file from database and copy toStream
parameter -
Delete
: Delete a file reference and all data chunks -
Find
: Find one or many files in_files
collection. ReturnsLiteFileInfo
class, that can be download data after. -
SetMetadata
: Update stored file metadata. This method doesn't change the value of the stored file. It updates the value of_files.metadata
. -
OpenRead
: Find file by_id
and returns aLiteFileStream
to read file content as stream
// Upload a file from file system
db.FileStorage.Upload("$/photos/2014/picture-01.jpg", @"C:\Temp\picture-01.jpg");
// Upload a file from a Stream
db.FileStorage.Upload("$/photos/2014/picture-01.jpg", "picture-01.jpg", stream);
// Find file reference only - returns null if not found
LiteFileInfo file = db.FileStorage.FindById("$/photos/2014/picture-01.jpg");
// Now, load binary data and save to file system
file.SaveAs(@"C:\Temp\new-picture.jpg");
// Or get binary data as Stream and copy to another Stream
file.CopyTo(Response.OutputStream);
// Find all files references in a "directory"
var files = db.FileStorage.Find("$/photos/2014/");
FileStorage
does not support transactions to avoid putting all of the file in memory before storing it on disk. Transactions are used per chunk. Each uploaded chunk is committed in a single transaction.
FileStorage
keeps _files.length
at 0
when uploading each chunk. When all chunks finish uploading FileStorage
updates _files.length
by aggregating the size of each chunk. If you try to download a zero bytes length file, the uploaded file is corrupted. A corrupted file doesn't necessarily mean a corrupt database.
Data Modeling
- Data Structure
- BsonDocument
- Object Mapping
- Relationships with Document References
- Collections
- FileStorage
Index
Query
Database
Version 4 changes
Shell