Output missing with diff command / missing utf-8 locale

[jneubert]

With a mapping file created externally, I get no output from the wdmapper diff command:

$ wdmapper diff P227 P2428 < /opt/repec_ras/var/pers_examples/example2/map/gnd_ras_mapping.example.csv
$ head  /opt/repec_ras/var/pers_examples/example2/map/gnd_ras_mapping.example.csv
source, target, annotation
171255054, phe112, "anthony giles heyes"
114008787, pwy2, "Charles Wyplosz"
137272472, pli522, "Stephen Littlechild"
171244133, pka87, "Sandeep Kapur"
120931761, pba2, "Söhnke M. Bartram"
171128656, pde16, "Rommert Dekker"
171110358, par215, "Anil Arya"
160912946, pqu66, "Agnes Reynes Quisumbing"
171827759, phy7, "Ari Hyytinen"

The complete input file is available on github

I've checked this on the current master and dev branch.

[nichtich]

Try wdmapper diff P227 P2428 -i /opt/repec_ras/var/pers_examples/example2/map/gnd_ras_mapping.example.csv. Reading Unicode from STDIN seems to be broken.

[jneubert]

Hmm - looks like reading from file is affected, too:

[nbt@7b2e4fc879c2 wdmapper]$ python wdmapper.py diff P227 P2428 -i /opt/repec_ras/var/pers_examples/example2/map/gnd_ras_mapping.example.csv
+ 106773011, pfl107, Peter Flaschel
+ Traceback (most recent call last):
  File "wdmapper.py", line 8, in <module>
    main()
  File "/opt/wdmapper/wdmapper/cli.py", line 84, in main
    run(*args)
  File "/opt/wdmapper/wdmapper/cli.py", line 73, in run
    wdmapper.wdmapper(args)
  File "/opt/wdmapper/wdmapper/wdmapper.py", line 218, in wdmapper
    command_diff(args)
  File "/opt/wdmapper/wdmapper/wdmapper.py", line 97, in command_diff
    writer.write_link(link)
  File "/opt/wdmapper/wdmapper/format/csv.py", line 96, in write_link
    print(self.separator.join(row), file=self.stream)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 25: ordinal not in range(128)

Commit is f963d7b

[jneubert]

Obviously, the issue was caused by the fact that in my centos7-on-docker instance the locale was not set by default. localectl isn't available there either, but with

LC_CTYPE=en_US.utf8
LANG=en_US.utf8

set, the command works nicely. Sorry for the false alarm!

UPDATE: See #22 (comment) for the correct settings