Unicode Error

[whoot]

Hey there, i hope that this is the right repo for my issue.
I´m using the Shodan CLI and installed it with "pip install shodan".
Im using the following command for requesting hosts:

shodan host <ip address>

However, when there is an umlaut character in the city name (in this case 'Düsseldorf'), the output is broken and the following error message is shown:

shodan host REDACTED
REDACTED
Hostnames:               REDACTED.com
Traceback (most recent call last):
  File "/usr/local/bin/shodan", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/dist-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python2.7/dist-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python2.7/dist-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python2.7/dist-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/shodan/__main__.py", line 479, in host
    click.echo('{:25s}{}'.format('City:', host['city']))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 1: ordinal not in range(128)

[achillean]

Fixed the problem by making sure that the generated strings are unicode.

[whoot]

A similar problem occurs when downloading shodan results and parsing it. Umlaute are represended as ?
Example:

shodan download output 'org:"REDACTED"'
  Search query:			org:"REDACTED" 
  Total number of results:	REDACTED
  Query credits left:		0
  Output file:			test.json.gz
  [####################################]  100%             
  Saved REDACTED results into file test.json.gz


shodan parse --fields location.city test.json.gz
  D?sseldorf
  [...]

[achillean]

Good point! Re-opening the issue to comb over other parts of the CLI that may output special characters.

[achillean]

Please try the new release (1.10.0) and make sure it looks good! I updated all of the code I could find to ensure it returns Unicode so the terminal can print it properly. The issue affected parse, stream and search.

[whoot]

Works like a charm! Thanks a lot!