Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving how file paths are mapped to colorspaces #893

Closed
doug-walker opened this issue Nov 27, 2019 · 4 comments
Closed

Improving how file paths are mapped to colorspaces #893

doug-walker opened this issue Nov 27, 2019 · 4 comments

Comments

@doug-walker
Copy link
Collaborator

doug-walker commented Nov 27, 2019

OCIO currently supports a workflow where users may embed a ColorSpace name string in their file paths and this allows applications to automatically associate imported files with the correct ColorSpace.

However some users do not have the flexibility to alter the paths to the media files in this way. There is a need for some additional capabilities that would allow simple functionality such as:
-- map all paths that contain the string "texture" to a specific ColorSpace
-- map all files with the extension "exr" to a specific ColorSpace

In addition, it would be very helpful for a config to always specify what ColorSpace to use by default when an application is importing a file and the normal rules are not matched. Some applications cannot work in strictparsing mode, i.e., they cannot always put up a dialog box to ask users what ColorSpace to use, and so there must be a fallback. It is better for the config to specify this explicitly rather than an app making a random choice (e.g. the first ColorSpace in the config).

This issue proposes a new feature called "File Rules" to provide these capabilities.

First a quick recap of what is currently in place. OCIO currently offers the parseColorSpaceFromString method on the Config class that takes a file path string and searches it to find a ColorSpace name from the loaded config. If a match is found it returns the ColorSpace name. If no match is found and the strictparsing token in the config is false, it returns the ColorSpace name specified by the default role. Otherwise it returns an empty string.

We propose adding a new section to the config called "file_rules". Here is an example:

file_rules:
    !<Rule> {name: LogC, extension="*", pattern="*LogC*", colorspace=ARRI LogC}
    !<Rule> {name: OpenEXR, extension="[eE][xX][rR]", pattern="*", colorspace=ACEScg}
    !<Rule> {name: TIFF, regex=".*\.TIF?F$", colorspace=sRGB}
    !<Rule> {name: ColorSpaceNamePathSearch}
    !<Rule> {name: Default, colorspace=Raw}

The File Rules are a set of mappings that are evaluated from the top down. The first rule to match is what determines which ColorSpace is returned.

There are four types of rules available. Each rule type has a name key that may be used by applications to refer to that rule. Name values must be unique. The other keys depend on the rule type.

  1. Basic Rules
    This is the basic rule type that uses Unix glob style pattern matching and is thus very easy to use. It contains the keys:
    name: Name of the rule
    pattern: Glob pattern to be used for the main part of the name/path.
    extension: Glob pattern to be used for the file extension. Note that if glob tokens are not used, the extension will be used in a non-case-sensitive way by default.
    colorspace: ColorSpace name to be returned.

  2. Regex Rules
    This is similar to the basic rule but allows additional capabilities for power-users. It contains the keys:
    name: Name of the rule
    regex: Regular expression to be evaluated.
    colorspace: ColorSpace name to be returned.

  3. OCIO v1 style Rule
    This rule allows the use of the OCIO v1 style, where the string is searched for ColorSpace names from the config. This rule may occur 0 or 1 times in the list. The position in the list prioritizes it with respect to the other rules. It has the key:
    name: Must be "ColorSpaceNamePathSearch".

  4. Default Rule
    The file_rules must always end with this rule. If no prior rules match, this rule specifies the ColorSpace applications will use. It has the keys:
    name: must be "Default".
    colorspace: ColorSpace name to be returned.

Roles may be used rather than ColorSpace names in the rules.

The existing parseColorSpaceFromString method will remain in the API and function like before. New API calls will be added to use the new rule system.

The strictparsing token will remain in the config for applications that want to enable that style of working. This token will not affect the behavior of the new API calls, in other words, evaluating the rules will always result in a ColorSpace being available to an application. However, the API will also allow the application to know which rule was the matching one. So apps that want to work in "strict" mode would first check if strictparsing is true and if so check to see if the matching rule was the Default Rule. If so, it could then notify the user and take whatever action is appropriate. (A helper method could also be provided to bundle this set of steps into a single call.)

If the file_rules section is not present in the config, the new API will try to use the default role in place of the Default Rule. If both are present, the Default Rule takes precedence. If both the file_rules section and the default role are missing, an exception will be thrown when loading the config.

Side Note: Currently, the default role is used not only for parseColorSpaceFromString but also for all other color space name searches (e.g. getIndexForColorSpace). In v2, the default role will only be used for parseColorSpaceFromString, which seems to be the intent of the OCIO documentation.

In order to facilitate extensibility, it will be legal for any of the rules to have additional key:value pairs where the value may be an arbitrary string. The API will provide access to getting/setting these additional pairs and will preserve them on a Config read/write.

The parseColorSpaceFromString method is currently not case sensitive and would remain that way. The other rules would be case-sensitive. Some OS and file systems obviously have issues with respect to case-sensitivity, but the API won't consider that since it is simply working on the provided strings.

@hodoulp hodoulp added the v2.0 label Nov 27, 2019
@michdolan
Copy link
Collaborator

This would be really helpful to many studios.

Another approach could be a Rule class hierarchy, to avoid some names being reserved, which can be less clear and more prone to human error. That could also avoid bloat with added arguments down the road, adding new derived classes instead.

file_rules:
    !<BasicRule> {name: LogC, extension="*", pattern="*LogC*", colorspace=ARRI LogC}
    !<BasicRule> {name: OpenEXR, extension="[eE][xX][rR]", pattern="*", colorspace=ACEScg}
    !<RegexRule> {name: TIFF, regex=".*\.TIF?F$", colorspace=sRGB}
    !<ColorSpaceNameRule>
    !<DefaultRule> {colorspace=Raw}

@michdolan
Copy link
Collaborator

Alternatively, we could use an Enum to type a single Rule class. Something like:

file_rules:
    !<Rule> {type: basic, name: LogC, extension="*", pattern="*LogC*", colorspace=ARRI LogC}
    !<Rule> {type: basic, name: OpenEXR, extension="[eE][xX][rR]", pattern="*", colorspace=ACEScg}
    !<Rule> {type: regex, name: TIFF, pattern=".*\.TIF?F$", colorspace=sRGB}
    !<Rule> {type: colorspace_name}
    !<Rule> {type: default, colorspace=Raw}

In either of these cases, the name field may be unnecessary as well, since the rules wouldn't be exposed in a UI. Just some thoughts...

@doug-walker
Copy link
Collaborator Author

Thanks for the suggestions Michael. Actually, we are anticipating that some apps may want to expose the rules in the UI. That was part of the thinking behind the proposed design. Since the name string might get exposed in UIs we wanted the names of the Default and ColorSpaceNamePathSearch rules to be fixed/uniformly presented. Hence the approach of making the name serve as both the description and the rule type.

@michdolan
Copy link
Collaborator

michdolan commented Dec 4, 2019

In that case, I would at least propose the use of some string constants to avoid having to define those strings to create the specialized rules (maybe this is planned already). Similar to the behavior of the built-in roles. Sounds great though!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants