Support 3rd party procedural DSOs

[bentoogood]

Some users may have need to call custom subprocedurals from within gaffer, using gaffer to assign shaders and attributes to the procedurals.

For example, users may wish to use a fur or crowd system that provides its own render procedural (rather than caching out results into a format gaffer reads).

Investigate ways in which a location could be tagged to be substituted with the relevant calls to a procedural. Would some more generalised solution that could inject arbitrary Rib/Ass statements be helpful?

[bentoogood]

One example of use would be to render a Yeti fur cache. As these caches contain all the information required to describe the fur, there is no need to query data from parts of the gaffer scene (i.e. no need to provide information about the base mesh).
An example of calling the yeti procedural and providing parameters is included here for reference (in .ass format)

procedural
{
 name pgYetiMaya1Shape
 dso "pgYetiArnold.so"
 min -2.0 -1.0 -1.4
 max 1.5 2.0 1.8
 visibility 255
 receive_shadows on
 self_shadows on
 matrix
 1 0 0 0
 0 1 0 0
 0 0 1 0
 0 0 0 1
 shader "myHairShader"
 opaque on
 id 982594017
 declare filename constant STRING
 filename "/path/to/my/cache.0001.fur"
 declare frame constant INT
 frame 1
 declare verbosity constant INT
 verbosity 1
 declare density constant FLOAT
 density 10
 declare width constant FLOAT
 width 1
 declare length constant FLOAT
 length 1
 declare threads constant INT
 threads 0
 declare min_pixel_width constant FLOAT
 min_pixel_width 0
 declare mode constant INT
 mode 0
 declare imageSearchPath constant STRING
 imageSearchPath ""
 declare samples constant ARRAY FLOAT
 samples 1
}

questions

• what bounds would be provided? Possibly there is some way of querying the cache to find its bounds... if not, option to just provide inf bounds?
• is it simple to map attributes attached to a location to custom declared ai attributes?

[bentoogood]

Example yeti cache for reference
https://drive.google.com/file/d/0B16CnRwtGChuSXE4NnJPejJxUkk/edit?usp=sharing
I believe it's a hdf5 file so should be browseable with hdfview

[johnhaddon]

I think this is pretty doable without too much effort. All objects in the scene must derive from IECore::Object, so we'll need some new "DSOProcedural" object which stores the name of the DSO and its parameters so it can be passed through the graph. Instances of these could be generated by a new ObjectSource derived class in Gaffer which uses a CompoundDataPlug to provide the arbitrary list of parameters. It would then be straightforward for a user to make an empty one, and add on the parameters necessary for a Yeti procedural, or any other procedural for that matter.

Then we need to consider how a DSOProcedural actually gets rendered via the IECore::Renderer abstraction layer - currently we don't have a mechanism for DSOs here. Going out to Arnold is easy, because we can convert an arbitrary list of parameters into Arnold parameters with existing code, but going out to RenderMan is a pain, because everything must get packed into a single string, and each DSO will have its own convention for how those are packed. I'm tempted to suggest that the RenderMan backend should only support a single string parameter, and its up the user to rig up some expressions to convert all the parameters into an appropriate string. That could all be boxed up for the common procedural types so that once it was done, you could just interact with the friendlier representation and forget about the string.

[colindoncaster]

Hi there -

Ben poked me about this and I figure it would be more useful to respond here.

We have quite a few studios using Yeti with Katana and doing exactly what you describe - the parameters to the DSO's are documented and you can still adjust fur density etc. We even have 2 studios ligthing/rendering Yeti caches within XSI!

As of Yeti 1.3 we also store previews for the fur, this could be used for visualizing the cache in the viewport potentially?

The format is indeed hdf5 and pretty close to Alembic though written prior to Alembic being released. The actual groom data is also accessible, some studios have been reading this groom data into houdini for unifying the dynamics with cloth sims and then overwriting the dataset.

Cheers

[colindoncaster]

Oh, and FWIW our renderman DSO uses the boost/program_options library to parse the string as if it's a command line assuming it's unix style which makes it easy to create/re-create these options.

[johnhaddon]

Quick note to say that we'd also want this mechanism to support .rib and .ass archives - I think this would map naturally onto what we've already discussed anyway, but just wanted it down in writing somewhere.

[mcsterpro]

Hello friends.

I have a question I hope you can help me, the question is how can I export fibres from yeti as .ass file in DSO mode "procedurally" to make populations in a new scene without generate yeti nodes with the stand-ins files?
can anyone help me with some step to step guide how to work with DSOs between yeti and arnold?

Thank you very much.

[andrewkaufman]

Hi @mcsterpro , I wonder if you'd mind re-posting your question to gaffer-dev@googlegroups.com ? I have a feeling more people will be following along there (though I could be wrong).

Also, to clarify, are you trying to use a fully baked Yeti render (.ass file) and the Gaffer ExternalProcedural node to render it via Gaffer as a sort of "AssArchive"? Or are you trying to use a Yeti cache (.fur file) and use the Gaffer ExternalProcedural with Yeti's pgYetiArnold.so to actually run Yeti evaluation at render time?