Skip to content
VDMS: Your Favourite Visual Data Management System
C++ Python Jupyter Notebook Other
Branch: develop
Clone or download
luisremis Merge pull request #120 from IntelLabs/remove-forced-read
Do not push 'read' operation on Video constructor
Latest commit 08a6981 Aug 5, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
client Add return blob to cpp client Jul 23, 2019
docker Update Dockerfiles to reduce image size Jun 4, 2019
include/vcl
src Merge pull request #120 from IntelLabs/remove-forced-read Aug 5, 2019
tests
utils
INSTALL.md Update INSTALL.md with protobuf version Aug 1, 2019
LICENSE
README.md Update README.md Nov 26, 2018
SConstruct Merge VideoData and Video classes Jul 2, 2019
config-vdms.json Fix directories handling Jan 14, 2019
run_server.sh

README.md

VDMS: Your Favourite Visual Data Management System

VDMS is a storage solution for efficient access of big-”visual”-data that aims to achieve cloud scale by searching for relevant visual data via visual metadata stored as a graph and enabling machine friendly enhancements to visual data for faster access. We use an in-persistent-memory graph database developed in our team called Persistent Memory Graph Database (PMGD) as the metadata tier and we are exploring the use of an array data manager, TileDB and other formats for images, visual descriptors, and videos as part of our Visual Compute Library (VCL). VDMS is run as a server listening for client requests and we provide client side bindings to enable communication between ( python, C++) applications and the server. Hence, it also has a Request Server component defined to implement the VDMS API, handle concurrent client requests, and coordinate the request execution across its metadata and data components to return unified responses. This project aims to research the use of a scalable multi-node graph based metadata store as part of a hierarchical storage framework specifically aimed at processing visual data, and also it includes an investigation into the right hardware and software optimizations to store and efficiently access large scale (pre-processed) visual data.

Motivation

Data access is swiftly becoming a bottleneck in visual data processing, providing an opportunity to influence the way visual data is treated in the storage system. To foster this discussion, we identify two key areas where storage research can strongly influence visual processing run-times: efficient metadata storage and new storage formats for visual data. We propose a storage architecture designed for efficient visual data access that exploits next generation hardware and give preliminary results showing how it enables efficient vision analytics.

Get Started

To get started, take a look at the INSTALL.md file, where you will find instructions on how to install and run the server.

Also, visit our wiki to learn more about the VDMS API, and take a look at some of the examples/tutorials.

Academic Papers

Conference Links, Cite Description
Learning Systems @ NIPS 2018 Paper, Cite Systems for Machine Learning Workshop @ NIPS
HotStorage @ ATC 2017 Paper, Presentation, Cite Positioning Paper at USENIX ATC 2017 Workshop
You can’t perform that action at this time.