[Fix] decode error(#1806) #1809

mingchuno · 2016-07-30T08:51:18Z

Short Description:

[Fix] decode error(#1806)

Fixes:

fix exception like "UnicodeDecodeError: 'ascii' codec can't decode byte 0xe7 in position 31: ordinal not in range(128)"

gregtampa · 2016-07-30T09:19:16Z

someone test and comment please

vieira · 2016-07-30T09:23:11Z

Not working for me.

$ python pokecli.py -cf config.json 
[10:20:29] PokemonGO Bot v1.0
[10:20:29] Configuration initialized
[10:20:29] 
[10:20:29] Not logged in!
Traceback (most recent call last):
  File "pokecli.py", line 464, in <module>
    main()
  File "pokecli.py", line 79, in main
    report_summary(bot)
  File "pokecli.py", line 87, in report_summary
    metrics.capture_stats()
  File "/home/vieira/projects/PokemonGo-Bot/pokemongo_bot/metrics.py", line 75, in capture_stats
    response_dict = self.bot.api.call()
  File "/home/vieira/projects/PokemonGo-Bot/pokemongo_bot/api_wrapper.py", line 57, in call
    if not self._can_call():
  File "/home/vieira/projects/PokemonGo-Bot/pokemongo_bot/api_wrapper.py", line 29, in _can_call
    raise NotLoggedInException()
pgoapi.exceptions.NotLoggedInException

As soon as I make decoded_str = string instead of decoded_string = string.decode('utf-8') it starts again but the original problem is still there, of course.

mingchuno · 2016-07-30T09:26:05Z

I am running it myself and it works
@vieira for your case, It looks like totally different issue from your stacktrace

vieira · 2016-07-30T09:27:35Z

The fix that works for me is:

--- a/pokemongo_bot/cell_workers/move_to_fort_worker.py
+++ b/pokemongo_bot/cell_workers/move_to_fort_worker.py
@@ -40,7 +40,8 @@ class MoveToFortWorker(object):
         )

         if dist > Constants.MAX_DISTANCE_FORT_IS_REACHABLE:
-            logger.log('Moving towards fort {}, {} left'.format(fort_name, format_dist(dist, unit))
+            logger.log(u'Moving towards fort {}, {} left'
+                    .format(fort_name.decode('utf-8'), format_dist(dist, unit)))

             step_walker = StepWalker(
                 self.bot,

@mingchuno Fell free the use the fix above in your PR if you see fit.

vieira · 2016-07-30T09:29:27Z

@mingchuno It's the same problem, the Unicode exception is probably getting caught by a generic except that just says unable to login. It's probably because my first name as a special character 😉

mingchuno · 2016-07-30T09:30:10Z

@vieira We should solve it in lower layer since the logger is shared by the whole app.
Adding .decode('utf-8') inside the logger impl is better than adding .decode('utf-8') every where in the code

vieira · 2016-07-30T09:31:20Z

@mingchuno Yes, I agree but in that case you have the fix every call to the logger that are already passing utf-8 encoded strings. Forget about my first name, it's probably the initial location.

vieira · 2016-07-30T09:33:57Z

@mingchuno Have a look at https://github.com/PokemonGoF/PokemonGo-Bot/blob/dev/pokemongo_bot/__init__.py#L456

mingchuno · 2016-07-30T09:37:06Z

@vieira Well, I am not actually know much on Python.
Isn't that u'test'.decode('utf-8') still give a valid unicode string?

Di9 · 2016-07-30T09:49:10Z

it works for me fine, thx! But Instead of Cyrillic title i got in terminal pokestot name with wrong encoding anyway (before it also was with not-cyrilic encoding, just another one) :)

vieira · 2016-07-30T09:59:46Z

@mingchuno You encode a sequence of unicode characters into a str, e.g. print(u'ç'.encode('utf-8')). u'ç'.decode('utf-8') won't work. You use decode, for instance, to bring a some encoded str back to unicode, you have to know how the str was encoded and decode it with the same codec, e.g. 'utf-8'. I am no expert in this, either.

vieira · 2016-07-30T10:02:34Z

@mingchuno Can you reproduce my problem if you set your initial location to something with special chars, e.g. "ç" or "ü"?

vieira · 2016-07-30T10:12:15Z

@mingchuno Without changing anything else, I think this works:

def log(string, color='white'):
    try:
        string = string.decode('utf8')
    except UnicodeEncodeError:
        pass

<rest of the original code>

Sometimes log is called with an already decoded str, other with an unicode sequence. If it is already a string decode will fail but it is safe to continue.

sgitkene · 2016-07-30T10:33:25Z

@vieira your fix seems to do it for me: (I used to have an error due to the "ü" in Sihlbrücke)

[12:31:10] PokeStop on cooldown. Time left: 4 minutes, 59 seconds
[12:31:13] Moving towards fort 1866 Sihlbrücke, 0.11km left
[12:31:19] -- Recycled 3x Pokeball (keeps only 50 maximum)
[12:31:20] -- Recycled 1x Super Potion (keeps only 1 maximum)
[12:31:20] Moving towards fort 1866 Sihlbrücke, 0.10km left
[12:31:25] Moving towards fort 1866 Sihlbrücke, 0.08km left
[12:31:31] Something rustles nearby!

Edit: Apparently somethng's still wrong: (Why do we use ascii???)

Traceback (most recent call last):
File "pokecli.py", line 464, in
main()
File "pokecli.py", line 68, in main
bot.tick()
File "/home/max/PokemonGo-Bot/pokemongo_bot/init.py", line 84, in tick
if worker.work() == WorkerResult.RUNNING:
File "/home/max/PokemonGo-Bot/pokemongo_bot/cell_workers/seen_fort_worker.py", line 32, in work
logger.log('Now at Pokestop: {0}'.format(fort_name), 'cyan')
File "/home/max/PokemonGo-Bot/pokemongo_bot/logger.py", line 28, in log
print('[' + time.strftime("%H:%M:%S") + '] ' + u'\033[' + color_hex[color] + string.decode('utf-8') + '\033[0m')
File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 28: ordinal not in range(128)

Edit2:

I put @CapCap 's fix in too and now it works. I believe nowhere should we still use ascii.
reload(sys)
sys.setdefaultencoding('utf8')

vieira · 2016-07-30T10:46:27Z

@sgitkene Ooops, missed an u'' in line 28. I will open a PR with both fixes.

Should be working now (Now at Pokestop: Quiosque Cais do Sodré):

[11:45:01] Moving towards fort Quiosque Cais do Sodré , 0.05km left
[11:45:06] Moving towards fort Quiosque Cais do Sodré , 0.04km left
[11:45:11] Now at Pokestop: Quiosque Cais do Sodré 
[11:45:11] Spinning ...

vieira · 2016-07-30T10:54:49Z

@sgitkene Could you help testing #1824? Thanks!

WebberTakkn · 2016-07-30T14:12:35Z

Please approve

DayBr3ak · 2016-07-30T16:12:21Z

190640f fixed this

sgitkene · 2016-07-30T18:57:01Z

sorry @vieira I was away. I assume it is no longer needed?

vieira · 2016-07-30T20:00:39Z

@sgitkene No problem at all! I guess #1839 should have fixed this but I can't test right now. If it works with:

initial location with special characters
moving to fort with special characters
arrived at pokestop with special characters

then we can close #1824.

marksweb · 2016-08-01T15:41:01Z

@vieira I've come up against a couple more UnicodeDecode errors with some French place names. It seems that setting the encoding when creating the string passed to logger.log() proves a better solution, e.g. 'Now at Pokestop: {0}'.format(unicode(fort_name, 'utf-8'))

I've done this with spin_fort.py and move_to_fort.py & had it running for an hour or so with no issues. Maybe it's worth another PR?

elicwhite · 2016-08-03T01:48:08Z

The logger has been deprecated and replaced with websockets. Hopefully this isn't an issue anymore. If it is, please re-open against the new approach.

[Fix] decode error(PokemonGoF#1806)

da36859

gregtampa added the Help to Test label Jul 30, 2016

mercuriete mentioned this pull request Jul 30, 2016

bot dies printing unicode to console #1813

Closed

mercuriete mentioned this pull request Jul 30, 2016

[dev] Nidoran Error because of special character #1855

Closed

elicwhite closed this Aug 3, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fix] decode error(#1806) #1809

[Fix] decode error(#1806) #1809

mingchuno commented Jul 30, 2016

gregtampa commented Jul 30, 2016

vieira commented Jul 30, 2016

mingchuno commented Jul 30, 2016

vieira commented Jul 30, 2016

vieira commented Jul 30, 2016

mingchuno commented Jul 30, 2016

vieira commented Jul 30, 2016

vieira commented Jul 30, 2016

mingchuno commented Jul 30, 2016

Di9 commented Jul 30, 2016

vieira commented Jul 30, 2016

vieira commented Jul 30, 2016

vieira commented Jul 30, 2016

sgitkene commented Jul 30, 2016 •

edited

Loading

vieira commented Jul 30, 2016 •

edited

Loading

vieira commented Jul 30, 2016

WebberTakkn commented Jul 30, 2016

DayBr3ak commented Jul 30, 2016

sgitkene commented Jul 30, 2016

vieira commented Jul 30, 2016

marksweb commented Aug 1, 2016

elicwhite commented Aug 3, 2016

[Fix] decode error(#1806) #1809

[Fix] decode error(#1806) #1809

Conversation

mingchuno commented Jul 30, 2016

gregtampa commented Jul 30, 2016

vieira commented Jul 30, 2016

mingchuno commented Jul 30, 2016

vieira commented Jul 30, 2016

vieira commented Jul 30, 2016

mingchuno commented Jul 30, 2016

vieira commented Jul 30, 2016

vieira commented Jul 30, 2016

mingchuno commented Jul 30, 2016

Di9 commented Jul 30, 2016

vieira commented Jul 30, 2016

vieira commented Jul 30, 2016

vieira commented Jul 30, 2016

sgitkene commented Jul 30, 2016 • edited Loading

vieira commented Jul 30, 2016 • edited Loading

vieira commented Jul 30, 2016

WebberTakkn commented Jul 30, 2016

DayBr3ak commented Jul 30, 2016

sgitkene commented Jul 30, 2016

vieira commented Jul 30, 2016

marksweb commented Aug 1, 2016

elicwhite commented Aug 3, 2016

sgitkene commented Jul 30, 2016 •

edited

Loading

vieira commented Jul 30, 2016 •

edited

Loading