OCIO fails parsing configs with non-C locale

[jeremyselan]

When OCIO is run on a Ubuntu 12.04 machine with a German locale, parsing of srgb.spi1d fails and we get an "Invalid 'From' Tag" exception. The issue is sscanf() there is attempting to parse the string using the German locale.

This should be fixed inside OCIO, the external user should not have to mess with locales before using.

See this thread on ocio-dev for additional information:
http://groups.google.com/forum/?fromgroups=#!topic/ocio-dev/xH47bLSIgwE

[devernay]

Also observed on OS X, with a French locale (fr_FR.UTF-8).
setting LANG=C fixes parsing, but IMHO OCIO should always parse the files using the "C" locale

[dictoon]

This problem has also been reported several times in appleseed:
appleseedhq/appleseed#2567

[JenusL]

I just got bit by this when writing a plugin for 3ds Max that uses OCIO.
I can't believe this has been open since 2012.
Will this ever get fixed?

[lgritz]

I went through this a year or two ago with OpenImageIO. I had to ruthlessly find all printf, scanf, and iostream stuff and replace it all with a new set of calls that I guarantee to be locale-independent.
It's not super hard and you are welcome to steal what you need from OIIO (I can point out what is necessary), but it can be an extensive undertaking.

[Dithermaster]

An alternative to changing the calls is to fix the locale around the call. We fixed this specific OCIO issue on the host side by saving the thread locale, setting it to "C" locale, calling the OCIO function, and then restoring the locale. It's all wrapped in an auto-style C++ class, so it's one line of code in the places we needed it. Yes, I reported it at the time, but we needed a fix ASAP so that's how we did it.

I recall it was a beautiful looking bug -- if you were running French language OS, one of our color managed looks got all posterized. It turned out to be because the LUT read only got integers instead of floating-point numbers (because decimal separator in French is not "."), so the curve because stair-steps.

[KelSolaar]

Oh we got that one at Duran Dubois when compiling Qt for Maya. All the artists got Minecraft like assets as everything was rounded to next integer. The panic it caused was unbelievable :)

…

On Wed, 26 Jun 2019, 06:19 Dithermaster, ***@***.***> wrote: An alternative to changing the calls is to fix the locale around the call. We fixed this specific OCIO issue on the host side by saving the thread locale, setting it to "C" locale, calling the OCIO function, and then restoring the locale. It's all wrapped in an auto-style C++ class, so it's one line of code in the places we needed it. Yes, I reported it at the time, but we needed a fix ASAP so that's how we did it. I recall it was a beautiful looking bug -- if you were running French language OS, one of our color managed looks got all posterized. It turned out to be because the LUT read only got integers instead of floating-point numbers (because decimal separator in French is not "."), so the curve because stair-steps. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#297>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAYLQ4766A5RUADV2YUQQ3P4JOS3ANCNFSM4ACPFBWA> .

[JenusL]

What bugs me is that I spent a long time trying to figure out what was wrong. But after I found out it was locale, the fix was easy in the host (3ds Max in this case).
Took me about 10 minutes to code it. But in case someone doesn't want to reinvent the wheel, here's what I did.

// OCIO has a , vs . bug depending on system locale
// https://github.com/imageworks/OpenColorIO/issues/297
const char* curlocale = strdup(setlocale(LC_NUMERIC, NULL));  // save current numeric locale
setlocale(LC_NUMERIC, "C");  // set numeric locale to C

// Put your affected OCIO code here
processor = config->getProcessor(transform);

setlocale(LC_NUMERIC, curlocale);  // restore numeric locale

It's a very manual approach but it was quick. So I would love to see how @Dithermaster solved it :)
Keep in mind I only override LC_NUMERIC. Maybe some other languages/locales need to override more.

[Dithermaster]

Our solution was based on the same APIs, but wrapped in C++ and with Windows support:

class AutoSetAndRestoreThreadLocale
{
public:
    AutoSetAndRestoreThreadLocale();
    ~AutoSetAndRestoreThreadLocale();

private:
#ifndef _WIN32
    locale_t oldLocale;
    locale_t currentLocale;
#else
    SoStringA ssaLocale;
    int previousThreadConfig;
#endif
};

and

AutoSetAndRestoreThreadLocale::AutoSetAndRestoreThreadLocale()
{

#ifndef _WIN32
    // set to C locale, saving the old one (returned from useLocale)
    currentLocale = newlocale(LC_ALL_MASK,"C",NULL);
    oldLocale = uselocale(currentLocale);
#else
    // set locale will only change locale on the current thread
    previousThreadConfig = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);

    // get and store current locale
    ssaLocale.convert(setlocale(LC_ALL, NULL));

    // set to "C" locale
    setlocale(LC_ALL, "C");
#endif

}

AutoSetAndRestoreThreadLocale::~AutoSetAndRestoreThreadLocale()
{

#ifndef _WIN32
    // restore the previous locale and freeing the created locale
    uselocale(oldLocale);
    freelocale(currentLocale);
#else
    // thread specific
    setlocale(LC_ALL, ssaLocale.c_str());

    // set back to global settings]
    _configthreadlocale(previousThreadConfig);
#endif

}

Then in a block where you're making OCIO calls, have
AutoSetAndRestoreThreadLocale locale;
When it hits that line it saves and sets the locale, then when the object destructs it restores the old locale.