Skip to content

Code organization

Jump to bottom
Josh Tomlinson edited this page Jul 7, 2015 · 5 revisions

For better or worse, we use the 'dpa' namespace for most of our pipeline code. What follows is a list of the modules in there:

dpa.action

Houses classes for defining and registering actions that can be performed within the pipeline. Actions are subclassed from an abstract base where, at a minimum, they define their interface (within pipeline and via command line) as well as 'execute' and 'undo' methods. Action classes are then registered for use within the pipeline and become, optionally, accessible via that registry within python or the 'dpa' command line tool.

dpa.app

The 'app' module defines classes and methods related to content creation packages. APIs include:

  • session abc which defines an interface for actions common to dcc applications. these include import, export, open, save, etc.
  • an app server class that facilitates communication with a dcc session running in another process (uses rpyc)
  • a command line interface action class abc that, when subclassed, allows dcc sessions to startup without a UI, running a remote session server
  • entity abstract base class which provides an api for mapping items within a dcc session to files on disk. geometry cache is an example of an entity type that we'd define for a given dcc app to facilitate export/import via the session objects.

dpa.cli

Defines an action subclass that is itself an abc for implementing command line tools as pipeline actions. The 'dpa' command line tool is itself an action.

dpa.config

Our Config objects are a subclass of python's OrderedDict. These objects read/write YAML files. We have custom YAML Loader and Dumper classes that allow our Config objects to maintain order as well as contain other Config objects that correspond to the hierarchy defined in the YAML files they read.

We also have a 'composite' method in the Config api that allows config files to be combined in a number of ways so that overrides can be applied as you move further down into the production hierarchy.

video demo: vimeo.com/123305980

dpa.data

This directory isn't really python. It contains some other files that should be available when we pip install the repo. Included are some default config files, icons, and app-specific plugins.

dpa.env

Wrapper APIs around environment variables and collections of environment variables. The 'EnvVar' class is an API for getting/setting/unsetting a single environment variable, but also houses a default value. 'PathVar' is a subclass of 'EnvVar' with list-like methods for manipulating variables like PYTHONPATH. Finally, the 'Env' class manages getting/setting a collection of 'EnvVars'. We use these APIs for manipulating the context as users move around on production.

There's also a dpa.env.vars class which provides easy access to common env variables. The point is to never hardcode an env variable in production code, and use this api instead.

video demo: vimeo.com/123340074

dpa.frange

A frame range API written by one of our students. I believe it works, but we're not really using it much yet. I'd like to revisit it as time permits. I'd really like to re-implement it using github.com/josh-t/OpenRange (shameless plug) as a base class.

dpa.houdini

Houdini-specific APIs. Currently a first pass at a session implementation for Houdini. Untested.

dpa.imgres

Image resolution API written by a student. It handles parsing reolution specifications like "1920x1080", then providing access to width, height, aspect ratio, and other properties. Also has 'half' and 'double' properties for calculating alternative resolutions.

dpa.location

Interface to our dpa locations api. Locations are stored in the db and we define them at filesystem boundaries. This is a simple read-only api.

dpa.logging

A simple logger object that I'm not super happy with. I need to revisit soon.

dpa.maya

Maya-specific APIs. Houses the maya session class as well as a python API for building python shelves. Also has one entity subclass currently for importing and exporting maya work files (references essentially).

dpa.notify

A placeholder mostly for what we hope will eventually be a really nice notification system/api. Currently the notification api just sends email.

Also defines some notification actions we use like 'ooto' (out of the office) and 'fail'.

dpa.open

An action for opening various types of files. Basically, a config file defines how various file types are opened. Typing 'dpa open ' performs this action which then opens the file with the corresponding application.

dpa.product

Our APIs that handle things we write to disk and subsequently share between artists working on production. In here are product related actions that can be performed like create, info, list, and update. There are also APIs for product categories, groupings, representations, subscriptions, versions, and location statuses.

Subscription actions include 'create', 'edit', 'list', 'refresh', 'remove', and 'update'.

dpa.ptask

PTasks, or Production Tasks, represent the various levels of our production hierarchy and also provide the context for the work a user is doing. Here we house the primary ptask API which retrieves meta data from the database. You'll also find the PTaskArea and PTaskEnv APIs which handle translating the context to a location in the filesystem and managing it's environment variables respectively. There is also a PTaskSpec API that we use to define a pipeline location independent specification for referring to these contexts.

PTask actions include 'clone', 'complete' (tab-completion of spec), 'create', 'env', 'info', 'list', 'source', 'sync', 'transfer', and 'version'.

The PTask versioning APIs also live here.

dpa.restful

Here we define the client for restful communication with our django server. Also defined is a base class for any objects using data returned from the server. We have mixin classes that allow us to quickly add any of the create, delete, get, set, list, or update methods to our data objects.

dpa.shell

First, a formatter base class that provides an API used to format shell-specific commands. We've implemented a BashFormatter so that we can set our context using bash. In theory, we could quickly implement other shell formatter types and our context setting would work there as well.

We also have an output API for pretty printing stuff to the terminal. The API primarily is used for printing tables and lists of data while taking into account the terminal width and adjusting the table's column widths accordingly. There are a bunch of options for formatting the output as well.

Also included are a bunch of class methods for prompting the user for answers of a given type. The 3rd party colorama package is also exposed here for making use of colored output.

dpa.singleton

A very simple singleton base class for when you need it.

dpa.sync

Pipeline action for rsync'ing files.

dpa.ui

Collecitions of qt widgets for use within the pipeline.

dpa.user

A simple read-only restful api into our django users data.

#########################

We also have a 'dpa_site' package that defines a custom import hook. We aren't using it currently, but we may turn it on at some point. The hook allows modules under the 'dpa' namespace to exist in multiple locations along the python path. The default behavior of python would be to expect any module under 'dpa' to exists where the first 'dpa' package was found along the PYTHONPATH. With this hook 'dpa.user' might exist in one python directory, while 'dpa.config' lives in another. This has proven useful in the past when you want to test a subset of code. A better solution for us may be to break out our codebase into smaller packages that are easy to install. That's why we haven't turned this on yet.

Clone this wiki locally