Digital Object Storage System
Java Other
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.settings
bin
client
doc
project
resources-test
resources/doss/migrations
src/doss
test/doss
.gitignore
.travis.yml
LICENSE
README.md
doss.thrift
nla-deploy.sh
pom.xml

README.md

DOSS 2

UBC Library Robot

DOSS 2 will be an object storage abstraction layer with transactional write-once semantics. Objects are uniquely and persistently identified and can never be overwritten only created and (soft) deleted.

Build Status (Huboard: Tasks, Backlog) (Javadoc) (Test coverage)

Status

Feature Goals

  • write once blob storage semantics
  • concurrent distributed random read access
  • multiple storage pools and devices (with different policies)
  • two-phase commit write transactions
  • soft deletes ("rubbish bin")
  • consistency and integrity checks
  • packs small files into containers for efficient bulk processing

Java API

See java-api.md for more examples.

Ingesting files

try (BlobStore bs = LocalBlobStore.open("/doss-devel");
    BlobTx tx = bs.begin()) {    
   
    Blob blob1 = tx.put(Paths.get("/tmp/myimage.jpg"));
    
    // if a digest is available, it's a good idea to verify it
    Blob blob2 = tx.put(Paths.get("/tmp/mytext.txt"));
    if (!blob2.digest("SHA1").equals("2aae6c35c94fcfb415dbe95f408b9ce91ee846ed")) {
        // something got corrupt so bail out (will automatically rollback)
        throw new RuntimeException("ingest failed: digest mismatch");
    }
    
    tx.commit();
    return blob1.id();
}

Random access

try (BlobStore bs = LocalBlobStore.open("/doss-devel")) {
    Blob blob = bs.get(273);
    try (Channel channel = blob.openChannel()) {
        // do something with the channel
    }
}

Non-Java read access

See doc/native-interface.md.

dossfs /doss-devel /mnt
grep -i endeavour /mnt/962b6910
convert /mnt/014908b0 /tmp/kangaroo.jpg

Building

Requirements:

  • Java 8
  • Maven 3

Compile by typing:

mvn package

To build both the server and remote client:

cd project
mvn package

License

Copyright 2013 National Library of Australia

DOSS 2 is free software; you can redistribute it and/or modify it under the terms of the Apache License, Version 2.0:

http://www.apache.org/licenses/LICENSE-2.0

See the file LICENSE for details.