Additive and subtractive extraction

[mottosso]

@BigRoy I mentioned during a previous chat something about "additive and subtractive extraction", and in light of this plug-in: validate_keys_none.py, I thought this might be a good time to elaborate on this concept.

Another way to put it is inclusive versus exclusive extraction, in which a plug-in either excludes unwanted data or includes only what is wanted.

For example, in validate_keys_none.py, animation keys are sought after and if any exists the Instance is invalid.

Instead, data can be excluded during extraction.

data

Let's assume this is our data.

data = {"vertices": list(), "userDefinedAttributes": list(), "keys": list()}

Inclusion

By not allowing certain data to be included, we would throw an error here.

if "keys" in data:
    raise Exception
else:
    export(data)

exclusion

By selectively include only wanted data, we would get the same output, without the need to throw an error.

export({
    "vertices": data["vertices"],
    "userDefinedAttributes": data["userDefinedAttributes"]
})

This technique might also be applicable to:

• validate_no_construction_history.py
• validate_no_unknown_nodes.py

Implementation

In the case of Maya, cmds.file has an option to exclude e.g. constructionHistory.

cmds.file(exportSelected=True, constructionHistory=False)

Benefits

• Simpler implementations
• No risk of uncaught, yet invalid data during validation
• Less user-facing errors

Disadvantages

• The data in a user's scene differ from what is outputted

[BigRoy]

@mottosso This is very much in line with what I had in mind. I've found that most of the checks I considered important could become less crucial by improving the method of extraction.

For example if I don't want any non-default shader assigned on my exported mesh at first I thought Validating whether it didn't have any non-default shaders assigned. Nevertheless it's much less intrusive for the Artist to just 'fix' it upon export (temporarily assign default shader) because it's not related data for the export anyway. Note that maya's export allows to exclude shaders, but the annoying thing is that it won't even get default shader assigned when imported in a new scene. You would only have the wireframe then.

So I have been setting up a more elaborate Extractor that ensures correct output to allow more freedom for the Artist (less restrictive validations) and still have a fully valid output. Would you also say that's the way to go?

[BigRoy]

Just wanted to add a little note to the example given at the top about validate_keys_none.py.

Excluding keyframes from Extraction might be beneficial for avoiding to have to raise an error but it's likely that a keyframe might mean an Extraction from different frame will result in a different output. With this in mind it might be possible that being on a different frame is an unintentional difference from the artist.

We could set up the Extractor so that it always extracts from the first frame so its consistent, but this might be confusing to the artist because the output is different than what he sees in the current scene on the current frame. Instead it would be clearer if at least a warning is raised.

This shows that it's important to ensure it's clear to artists what will be extracted from the scene and how such extractions are performed. Maybe it could be crucial here to have warnings for the artists that could be 'passed' if the artist is aware of what is happening to his scene instead of having him/her run into more limiting errors.

What do you think?

[mottosso]

Excluding keyframes from Extraction might be beneficial for avoiding to have to raise an error but it's likely that a keyframe might mean an Extraction from different frame will result in a different output.

Hm, I'm not following here. Are you thinking that it would exclude individual keys, as opposed to whole curves? If so, then no, that would be too fragile and not to mention unnecessary. :S

[BigRoy]

I meant if you were to extract a model without any animation from a scene by excluding any keyframes it might have then it would have a different output based on from which frame it would be extracted.

A mesh moving or deforming from origin up to 10 in y-axis would change on different frames.

Does that make more sense?

[mottosso]

It does, but I can't see how that's a problem. Wouldn't a model with keyframes be broken regardless? How do you rig something like that?

[BigRoy]

Just wanted to add a little note to the example given at the top aboutvalidate_keys_none.py.

With this I meant that specifically this example would be tricky one. Doing an Exclusive extraction (just ditching the keys) would be worse than having a Validator in place for this.

So exactly, a Model should be validated for keyframes to help the artist know what's going on. Extraction can still be made Exclusive of keys to make that safe as well.

[mottosso]

Ah, now I see what you mean. Well put. I even got tricked into proving your point. :)

[BigRoy]

I think the Additive vs Subtractive remains an interesting topic and could even be a main Wiki page for Pyblish to describe up/downsides and introduce anyone with it so they at least think about it.

I think the discussion has proven useful but it's hard to keep this open "just as a reminder" to keep this in our minds.

It's not a particular "issue" currently present so could be closed?

[mottosso]

You're welcome to close anything you feel is not achievable or is already achieved.

[BigRoy]

I can't seem to be able to close this, only those that I opened have the close button. 😄

[mottosso]

Oh! I'll see if I can fix that.

[mottosso]

Ok, should be fixed now.