Arabic output garbled from dictionary creation

[mzeidhassan]

Hi @mammothb,

I am using your code to create a frequency dictionary in Arabic.
My corpus file is in utf-8 format.

Here is the output I am getting:

طµظپطھظٹظ 2
515 1
ظٹط²ظٹظٹظ 4
طھظˆط³ظٹظ 12
ظپظƒطœ 2
ظٹظˆظƒظٹطھط 14
ط³ظٹط³ظˆط 5

This looks like corrupt characters, so I am not sure what is causing this.

I even tried to print out to file and used 'encoing='utf-8', but again I am getting the same result as you can see in the attached screen shot.

Any idea how can I fix this or what is causing this issue? I am using Anaconda with Python 3..5.2 by the way.

Thanks

[mzeidhassan]

I created a pull request that fixed this issue for me. I am not so familiar with pr submission, so I hope I did it right. Here is the link to the pr.
#29

Maybe, it's just easier to make the change at your end directly.

[mzeidhassan]

The newly created frequency dictionary looks fine now.