Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serialize/deserialize shader groups #381

Merged
merged 3 commits into from
Jun 26, 2014

Conversation

lgritz
Copy link
Collaborator

@lgritz lgritz commented Jun 19, 2014

I'll introduce this set of changes by illustrating the "killer app":

$ myrenderer myscene.scene
...weird shader behavior or an error...

$ OSL_OPTIONS="archive_groupname=mygroup" myrenderer myscene.scene
$ ls mygroup*
mygroup.tar.gz
$ tar tf mygroup.tar.gz
shadergroup
a.oso
b.oso

This tar file contains a serialized description (in "shadergroup") of the shader group network -- shaders, instance parameter values, connections -- and the oso of every shader referenced by the group. These can be reconstituted to debug the shader from testshade or any other debugging-conducive environment, which does not have to be the original renderer.

This involves three parts:

Part 1:

Added a new variety of ShaderGroupBegin() that accepts a string containing "commands" that easily and human-readably describe the entire setup of a ShaderGroup, equivalent to a set of Parameter, Shader, and ConnectShaders calls. Here is an example:

ShaderGroupBegin ("groupname", "surface",
                      /*concatenated string follows: */
                      "param float fin 3.14; " 
                      "param float val 42.0 [[int lockgeom=0]]; " 
                      "shader a alayer;"
                      "shader b blayer;"
                      "connect alayer.f_out blayer.f_in;");

Added --group command to testshade so this can be tested and used from the
command line. Here's an example:

testshade -v -group "param float fin 3.14, shader a alayer, shader b blayer, connect alayer.f_out blayer.f_in"

(Commas are an allowed substitute for semicolons for statement separators, and the last statement separator is optional if it's at the end of the string; this makes it easier for command line use, where semicolons have special meaning in some contexts like our test scripts.)

Part 2:

The mirror image of this, where we can serialize (turn into a command string) an existing ShaderGroup. This is done by retrieving the group attribute "pickle":

std::string groupstring;
shadingsys->getattribute (groupptr, "pickle", groupstring);
std::cout << groupstring << "\n";

will print

param float fin 3.14 ; 
param float val 42.0 [[int lockgeom=0]] ; 
shader a alayer ;
shader b blayer ;
connect alayer.f_out blayer.f_in ;

Part 3:

shadingsys->archive_shadergroup(group,filename) will create an archive file (tar, tar.gz, or zip, depending on the file extension) containing a serialized description of the shader group as well as oso files for every shader referenced in the group.

Also, new ShadingSystem attributes "archive_groupname" and "archive_filename" will cause the named group to be automatically archived to the given filename, right after it is set up.

@lgritz
Copy link
Collaborator Author

lgritz commented Jun 19, 2014

Actually, I'm hoping to make it work on Windows, too. I just need somebody to tell me a reliable command line to invoke a zip'er. The rest of the file system manipulation ought to just work.

@fpsunflower
Copy link
Contributor

I don't think there's any command line zipper built into windows. I think the only portable solution would require a third party library.

@lgritz
Copy link
Collaborator Author

lgritz commented Jun 19, 2014

Then the shell-out will fail at runtime. I'm tempted to leave it in, so that if they have a "tar" or "zip" implementation in the path, it will just work. Otherwise, it will simply make the directory, copy the files, fail the zip command, then wipe clean the directory, no harm no foul.

…r group

This is half the work to allow serialization of the set of Parameter, Shader,
and ConnectShaders calls that fully assemble a shader group.  Here is an
example:

    ShaderGroupBegin ("groupname", "surface",
                          "param float fin 3.14; " /*concatenate string*/
                          "shader a alayer;"
                          "shader b blayer;"
                          "connect alayer.f_out blayer.f_in;");

Add --group command to testshade so this can be tested and used from the
command line. Here's an example:

    testshade -v -group "param fin 3.14, shader a alayer, shader b blayer, connect alayer.f_out blayer.f_in"

(Commas are an allowed substitute for semicolons for statement separators,
and the last statement separator is optional if it's at the end of the
string; this makes it easier for command line use, where semicolons have
special meaning.)
…ribute

for the group. This will turn the full ShaderGroup specification into a
string in the same format that's accepted by the new ShaderGroupBegin
syntax.
1. shadingsys->archive_shadergroup(group,filename) will create an archive
file (tar, tar.gz, or zip, depending on the file extension) containing
a serialized description of the shader group as well as oso files for every
shader referenced in the group. Great for debugging!

2. SS attribute "archive_groupname", "archive_filename" will cause the
named group to be automatically archived to the given filename, right after
it is set up.

You can thus dump everything necessary to restore a shader group, even from
a renderer that has OSL buried deep within, by running with environment
variable OSL_OPTIONS="archive_groupname=mygroup,archive_filename=debug.tgz",
and then the archived oso files and shadergroup description can be
reconstituted to debug the shader from testshade or any other
debugging-conducive environment, which does not have to be the original
renderer.
lgritz added a commit that referenced this pull request Jun 26, 2014
Serialize/deserialize shader groups
@lgritz lgritz merged commit dbbf6dc into AcademySoftwareFoundation:master Jun 26, 2014
@lgritz lgritz deleted the lg-pickle branch June 26, 2014 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants