consider setting proper locale for hooks environments

[achiang]

A mongodb-relation-changed hook called out to a python3 script to import some data into mongodb.

The python script encountered a unicode error:

2014-06-19 20:23:17 INFO mongodb-relation-changed Traceback (most recent call last):
2014-06-19 20:23:17 INFO mongodb-relation-changed File "/opt/openmotion/import.py", line 194, in
2014-06-19 20:23:17 INFO mongodb-relation-changed parse_bikes(mongo_uri)
2014-06-19 20:23:17 INFO mongodb-relation-changed File "/opt/openmotion/import.py", line 155, in parse_bikes
2014-06-19 20:23:17 INFO mongodb-relation-changed stations = parser1
2014-06-19 20:23:17 INFO mongodb-relation-changed File "/opt/openmotion/import.py", line 81, in parse_valencia_bikes
2014-06-19 20:23:17 INFO mongodb-relation-changed json_data = open(basepath + 'data/bikes/Valenbisi.JSON').read()
2014-06-19 20:23:17 INFO mongodb-relation-changed File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
2014-06-19 20:23:17 INFO mongodb-relation-changed return codecs.ascii_decode(input, self.errors)[0]
2014-06-19 20:23:17 INFO mongodb-relation-changed UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 272: ordinal not in range(128)
2014-06-19 20:23:17 ERROR juju.worker.uniter uniter.go:482 hook failed: exit status 1

Running the same script from a terminal on the same node succeeded.

After further debugging, I determined that the mongodb-relation-changed hook does not set any locale.

Adding:

export LANG=en_US.UTF-8

to the hook allowed the import script to succeed.

Note: although this enabled me to make progress on writing my hook, I think upstream needs to take a closer look. I am not an expert on locales, and I wonder if forcing en_US.UTF-8 on all juju users is appropriate or not...

[setharnold]

LANG=c.UTF-8 may make sense but I'm very afraid of deviating from a hard-coded locale for all hooks. If a hook author wants to set a different locale, that's fine, but to allow arbitrary LANG choice is a recipe for instability and difficult-to-debug problems.

Thanks

[marcoceppi]

What if it just inherited the default lang for the system? This is very much a "flies in the face of new comers" when a script works flawlessly on a fresh cloud image but not in a hook context.

[marcoceppi]

that's fine, but to allow arbitrary LANG choice is a recipe for instability and difficult-to-debug problems.

Without a LANG set we're seeing that right now. Seems like there's no way to win?

[axw]

Without a LANG set we're seeing that right now. Seems like there's no way to win?

Hook authors can explicitly set the LANG environment variable from within the hook. That said, I think en_US.UTF8 would be a reasonable default.

[waigani]

Bug added to lp: https://bugs.launchpad.net/juju-core/+bug/1334482

[squidsoup]

This just bit me with a python 3 library calling codecs.open() which falls back to ASCII if no LANG is set. In this case it was reading from a UTF-8 rst, and blew up. Inheriting from the system LANG seems sensible and the least surprising behaviour.

[anastasiamac]

We track our bug through launchpad. Please use filed bug above for comments/additions/suggestions.