VT Bundle: Make database subsystem more robust

[dakoop]

Use SQLAlchemy to allow integration with different database backends and improve robustness of database storage. This includes updates to the bundle structure (#962).

The bundle structure is designed to be more customizable. Specifically, we have the following core classes:

• Bundle: stores content as BundleObj objects and mappings in a dictionary of BundleObjMapping; a bundle is typed according to its root object
• BundleObj: wraps a VisTrails object with type and identifier fields
• BundleObjMapping: defines how BundleObjs of a specific type are organized in a Bundle; e.g., is there a single object in the bundle or can there be many. If there are many, how are they identified?
• BundleSerializer: defines how a Bundle is serialized to a specific backend; contains individual Serializer objects that define serializations for individual BundleObj types

Note that the separation between bundle organization and serialization is intentional. We want to be able to access a bundle's objects independently of how they are stored. We also want to be able to store a bundle via any serialization; one might take a .vt file and save it to a sqlite database.

The bundles are versioned. Thus, if someone opens a v2.0 bundle, it can be parsed correctly even if the v2.1 bundle structure restructures content. In practice, this requires the version to define default bundle structures and mappings as well as default bundle serialization methods. See the 1.0.5 persistence/__init__.py source for how this works. However, the structures should also allow users to define their own bundle types (for example, to include data files or package definitions) and register the corresponding mappings and serializations.

The point of the BundleObjMapping is to define the mapping from in-memory raw object to a BundleObj. Specifically, what are the obj_type and obj_id mapped from. File reference mappings are not good mappings because they are tied to a specific serialization; this is something the serializer needs to interpret. Thus, the goals of individual mappings are to set up methods to extract/sync ids and types from objects and manage the access to collections of objects with the same type.

TODOs

• Check translations between versions
• Update DB serialization routines
• Make sure mapping <-> serializer match (link these better?)
• Create core methods to register bundle types
• Make bundles and serializers user extensible