Permalink
Browse files

new SchemaDef plugin type to define site-specific schemas (#342)

* new SchemaDef plugin type to define site-specific schemas
* add missing _update_plugin_source for pkg_resources; force schemadefs to load
* add SchemaDef plugin test
* add method to extend all three manifest plugin lists at once
* cleaner save/restore of existing _MANIFEST
* schemadef plugin classes added to otio.schemadef namespace
* Explicitly include our source files in MANIFEST.in

Travis-CI is having some problems reconciling the file list between
sdist and the repo and suggested adding this line:
recursive-include opentimelineio *
so I'm adding it to see if that works.
The particular problem is that it is not picking up one of my latest
additions, namely the schemadef directory and the __init__.py file
in it.  That's the only source file in that directory -- maybe that's
the problem?

* add UnknownSchema object to pass through undefined schemas

Old behavior was to through an exception if any object being read
(or created by instance_from_schema) had an unregistered schema name.
New behavior is to replace the schema of such objects with the
UnknownSchema schema type and save the original schema label in a
field called "UnknownSchemaOriginalLabel".  Such objects are then
available within the OTIO data structure.  The new property
"is_unknown_schema" will be True for these objects and False for all
other SerializableObjects.  When an UnknownSchema is serialized,
the json_serializer will replace the UnknownSchema label with the
original label, so that (for instance) otiocat will pass such
unregistered objects through unchanged.  This is part of the schemadef
project because schemadef plugins imply that we will start to see
site or studio-specific OTIO schema types, and it would be more polite
to pass these through quietly if you don't understand them.

* improve docs and tests for schemadef and unknown schema
  • Loading branch information...
peachey authored and ssteinbach committed Oct 24, 2018
1 parent a13284e commit 9459b2c3d3b2820091b6d864027f11e1af362634
@@ -1,5 +1,6 @@
include README.md CHANGELOG.md LICENSE.txt NOTICE.txt
recursive-include examples *
recursive-include opentimelineio *
recursive-exclude docs *
prune docs
@@ -14,4 +15,4 @@ exclude opentimelineio_contrib/adapters/Makefile
exclude Makefile
recursive-exclude opentimelineio_contrib/adapters/tests *
recursive-exclude tests *
recursive-exclude tests *
@@ -47,6 +47,7 @@ Tutorials
tutorials/wrapping-otio
tutorials/write-an-adapter
tutorials/write-a-media-linker
tutorials/write-a-schemadef
Use Cases
------------
@@ -71,4 +72,3 @@ Indices and tables
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
@@ -0,0 +1,118 @@
# Writing an OTIO SchemaDef Plugin
OpenTimelineIO SchemaDef plugins are plugins that define new schemas within the
otio type registry system.
You might want to do this to add new schemas that are specific to your own
internal studio workflow and shouldn't be part of the generic OpenTimelineIO
package.
To write a new SchemaDef plugin, you create a Python source file that
defines and registers one or more new classes subclassed from
``otio.core.SerializeableObject``. Multiple schema classes can be defined
and registered in one plugin, or you can use a separate plugin for each of them.
Here's an example of defining a very simple class called ``MyThing``:
```
import opentimelineio as otio
@otio.core.register_type
class MyThing(otio.core.SerializableObject):
"""A schema for my thing."""
_serializable_label = "MyThing.1"
_name = "MyThing"
def __init__(
self,
arg1=None,
argN=None
):
otio.core.SerializableObject.__init__(self)
self.arg1 = arg1
self.argN = argN
arg1 = otio.core.serializable_field(
"arg1",
doc = ( "arg1's doc string")
)
argN = otio.core.serializable_field(
"argN",
doc = ( "argN's doc string")
)
def __str__(self):
return "MyThing({}, {})".format(
repr(self.arg1),
repr(self.argN)
)
def __repr__(self):
return "otio.schema.MyThing(arg1={}, argN={})".format(
repr(self.arg1),
repr(self.argN)
)
```
In the example, the ``MyThing`` class has two parameters ``arg1`` and ``argN``,
but your schema class could have any number of parameters as needed to
contain the data fields you want to have in your class.
One or more class definitions like this one can be included in a plugin
source file, which must then be added to the plugin manifest as shown below:
## Registering Your SchemaDef Plugin
To create a new SchemaDef plugin, you need to create a Python source file
as shown in the example above. Let's call it ``mything.py``.
Then you must add it to a plugin manifest:
```
{
"OTIO_SCHEMA" : "PluginManifest.1",
"schemadefs" : [
{
"OTIO_SCHEMA" : "MyThing.1",
"name" : "mything",
"execution_scope" : "in process",
"filepath" : "mything.py"
}
]
}
```
The same plugin manifest may also include adapters and media linkers, if desired.
Then you need to add this manifest to your `$OTIO_PLUGIN_MANIFEST_PATH` environment variable (which is "`:`" separated).
## Using the New Schema in Your Code
Now that we've defined a new otio schema, how can we create an instance of the
schema class in our code (for instance, in an adapter or media linker)?
SchemaDef plugins are magically loaded into a namespace called ``otio.schemadef``,
so you can create a class instance just like this:
```
import opentimelineio as otio
mine = otio.schemadef.my_thing.MyThing(arg1, argN)
```
An alternative approach is to use the ``instance_from_schema``
mechanism, which requires that you create and provide a dict of the parameters:
```
mything = otio.core.instance_from_schema("MyThing", 1, {
"arg1": arg1,
"argN": argN
})
```
This ``instance_from_schema`` approach has the added benefit of calling the
schema upgrade function to upgrade the parameters in the case where the requested
schema version is earlier than the current version defined by the schemadef plugin.
This seems rather unlikely to occur in practice if you keep your code up-to-date,
so the first technique of creating the class instance directly from
``otio.schemadef`` is usually preferred.
@@ -37,6 +37,7 @@
exceptions,
core,
schema,
schemadef,
plugins,
adapters,
algorithms,
@@ -27,7 +27,7 @@
# flake8: noqa
from . import (
serializable_object,
serializable_object
)
from .serializable_object import (
SerializableObject,
@@ -61,3 +61,7 @@
from .media_reference import (
MediaReference,
)
from . import unknown_schema
from .unknown_schema import (
UnknownSchema
)
@@ -34,6 +34,8 @@
type_registry,
)
from .unknown_schema import UnknownSchema
from .. import (
exceptions,
opentime,
@@ -94,6 +96,20 @@ def _encoded_serializable_object(input_otio):
return result
def _encoded_unknown_schema_object(input_otio):
orig_label = input_otio.data.get(UnknownSchema._original_label)
if not orig_label:
raise exceptions.InvalidSerializableLabelError(
orig_label
)
# result is just a dict, not a SerializableObject
result = {}
result.update(input_otio.data)
result["OTIO_SCHEMA"] = orig_label # override the UnknownSchema label
del result[UnknownSchema._original_label]
return result
def _encoded_time(input_otio):
return {
"OTIO_SCHEMA": "RationalTime.1",
@@ -125,6 +141,7 @@ def _encoded_transform(input_otio):
opentime.RationalTime: _encoded_time,
opentime.TimeRange: _encoded_time_range,
opentime.TimeTransform: _encoded_transform,
UnknownSchema: _encoded_unknown_schema_object,
SerializableObject: _encoded_serializable_object,
}
@@ -140,6 +140,12 @@ def schema_version(cls):
cls._serializable_label
)
@property
def is_unknown_schema(self):
# in general, SerializableObject will have a known schema
# but UnknownSchema subclass will redefine this property to be True
return False
def __copy__(self):
result = self.__class__()
result.data = copy.copy(self.data)
@@ -48,6 +48,12 @@ def schema_version_from_label(label):
return int(label.split(".")[1])
def schema_label_from_name_version(schema_name, schema_version):
"""Return the serializeable object schema label given the name and version."""
return "{}.{}".format(schema_name, schema_version)
def register_type(classobj, schemaname=None):
""" Register a class to a Schema Label.
@@ -106,9 +112,14 @@ def instance_from_schema(schema_name, schema_version, data_dict):
"""Return an instance, of the schema from data in the data_dict."""
if schema_name not in _OTIO_TYPES:
raise exceptions.NotSupportedError(
"OTIO_SCHEMA: '{}' not in type registry.".format(schema_name)
)
from .unknown_schema import UnknownSchema
# create an object of UnknownSchema type to represent the data
schema_label = schema_label_from_name_version(schema_name, schema_version)
data_dict[UnknownSchema._original_label] = schema_label
unknown_label = UnknownSchema._serializable_label
schema_name = schema_name_from_label(unknown_label)
schema_version = schema_version_from_label(unknown_label)
cls = _OTIO_TYPES[schema_name]
@@ -0,0 +1,43 @@
#
# Copyright 2017 Pixar Animation Studios
#
# Licensed under the Apache License, Version 2.0 (the "Apache License")
# with the following modification; you may not use this file except in
# compliance with the Apache License and the following modification to it:
# Section 6. Trademarks. is deleted and replaced with:
#
# 6. Trademarks. This License does not grant permission to use the trade
# names, trademarks, service marks, or product names of the Licensor
# and its affiliates, except as required to comply with Section 4(c) of
# the License and to reproduce the content of the NOTICE file.
#
# You may obtain a copy of the Apache License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the Apache License with the above modification is
# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the Apache License for the specific
# language governing permissions and limitations under the Apache License.
#
"""
Implementation of the UnknownSchema schema.
"""
from .serializable_object import SerializableObject
from .type_registry import register_type
@register_type
class UnknownSchema(SerializableObject):
"""Represents an object whose schema is unknown to us."""
_serializable_label = "UnknownSchema.1"
_name = "UnknownSchema"
_original_label = "UnknownSchemaOriginalLabel"
@property
def is_unknown_schema(self):
return True
@@ -84,6 +84,7 @@ class Manifest(core.SerializableObject):
def __init__(self):
core.SerializableObject.__init__(self)
self.adapters = []
self.schemadefs = []
self.media_linkers = []
self.source_files = []
@@ -92,16 +93,31 @@ def __init__(self):
type([]),
"Adapters this manifest describes."
)
schemadefs = core.serializable_field(
"schemadefs",
type([]),
"Schemadefs this manifest describes."
)
media_linkers = core.serializable_field(
"media_linkers",
type([]),
"Media Linkers this manifest describes."
)
def extend(self, another_manifest):
"""
Extend the adapters, schemadefs, and media_linkers lists of this manifest
by appending the contents of the corresponding lists of another_manifest.
"""
if another_manifest:
self.adapters.extend(another_manifest.adapters)
self.schemadefs.extend(another_manifest.schemadefs)
self.media_linkers.extend(another_manifest.media_linkers)
def _update_plugin_source(self, path):
"""Track the source .json for a given adapter."""
for thing in (self.adapters + self.media_linkers):
for thing in (self.adapters + self.schemadefs + self.media_linkers):
thing._json_path = path
def from_filepath(self, suffix):
@@ -140,6 +156,12 @@ def adapter_module_from_name(self, name):
adp = self.from_name(name)
return adp.module()
def schemadef_module_from_name(self, name):
"""Return the schemadef module associated with a given schemadef name."""
adp = self.from_name(name, kind_list="schemadefs")
return adp.module()
_MANIFEST = None
@@ -165,8 +187,7 @@ def load_manifest():
"contrib_adapters.plugin_manifest.json"
)
)
result.adapters.extend(contrib_manifest.adapters)
result.media_linkers.extend(contrib_manifest.media_linkers)
result.extend(contrib_manifest)
except ImportError:
pass
@@ -194,15 +215,19 @@ def load_manifest():
manifest_stream.read().decode('utf-8')
)
manifest_stream.close()
filepath = pkg_resources.resource_filename(
plugin.module_name,
'plugin_manifest.json'
)
plugin_manifest._update_plugin_source(filepath)
except Exception:
logging.exception(
"could not load plugin: {}".format(plugin_name)
)
continue
result.adapters.extend(plugin_manifest.adapters)
result.media_linkers.extend(plugin_manifest.media_linkers)
result.extend(plugin_manifest)
else:
# XXX: Should we print some kind of warning that pkg_resources isn't
# available?
@@ -221,9 +246,11 @@ def load_manifest():
continue
LOCAL_MANIFEST = manifest_from_file(json_path)
result.adapters.extend(LOCAL_MANIFEST.adapters)
result.media_linkers.extend(LOCAL_MANIFEST.media_linkers)
result.extend(LOCAL_MANIFEST)
# force the schemadefs to load and add to schemadef module namespace
for s in result.schemadefs:
s.module()
return result
Oops, something went wrong.

0 comments on commit 9459b2c

Please sign in to comment.