PyPDF2 failing to read unicode character

[SharmileeS]

I have a PDF which PDFFileReader is unable to read the text , instead this is the output:

u'\n˘ˇˆ˘ˇ˙˝˛˛˚˜ !!"#$%&"˝˛˝˘˛˘˛˚˙˘ˇ˝˛˘˛$\'(˘%˘ˇ˘ˆ˘)_)˛\'+,-)"˛./0"0!123˛"4˙"5)46)!6"˙˘˘˘,˘ˇˆ˙˙ˆ˝˛˚˜ !˘ˇˆ˙˝"" ˜#˝$˛˚˜ ˆ˙˝"" ˜ %˛˚˜ !˛˚ˇ!"#$%˘ˇ&ˆ˙˝˛˝ˆ˙&˚˝\'˛˚&\'()_ˇ+˙˝"" ˜#˝$˜#( ˛˚(ˇ+,˘˘˘ˇˆˆˆˇ,ˆ--ˆˇˇ˙˝˝% ˜)˜#_#˝$$˜  ˙ ˝_˛˚ˆ-&ˆ!ˆˇ&˘+$ˆ(˙˝+˚˜,!˛˚./&0ˆˆ+$ˆ(˙˝-˛-,&˘˝ˆ. ˚%˝% ˜)˜#\* ˜!˛˚&ˆˇ%ˆ!&(12+3ˇ˙˝,˜ˆ/˛˚%#"+3("ˆˇ.!ˆˇ43ˇ(˙-,&53ˇ6ˆˇ,˝˝% ˜)˜#\* ˜!˛˚(77777777777˜#( 0123& ˜"" ˜ %˛˚˜ 77777777777˜#( _ˆ_˛ ,4+#(56˝% ˜)˜#\* ˜!7  56 _˜ˆ(  %!_ˆ_˛ ˆ˙&˚˝\'586"ˇ+((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((&\'()_&\'(_&\'()˘536((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((&\'&\' &\'˜ ˜˙˚ˆ-",ˇˆˇ!ˆ-ˆ,ˆ&ˆ!ˆˇ&53ˇ6ˆˇ,(˙˚&ˆ!-ˇ!6ˆˇ,˘ 8-ˇˆ-˙˝˝% ˜)˜#_ ˜!7  ˛˚(˙˚9ˇˇˆ-6ˆˇ,:;ˇˇˆ-<ˆˆ-ˇ&\' ,,˘˘ˇˇˆ-(9ˆˇˆ-!˘ˇˆ9˘ˆˇ˘˘(\n\n'

This is the output after Extract Text and it doesnot throw any error message.

A similar issue has been posted here:

http://stackoverflow.com/questions/15583535/how-to-extract-text-from-a-pdf-file-in-python
I am using windows so the solution in link is not helpful

[mstamy2]

This issue is larger than PyPDF2; the text of the PDF is likely not stored in a 'reasonably exportable way'. Can you successfully copy/paste text from the PDF as you view it in a PDF viewer? If not, and the text that gets pasted is unreadable or in a binary format, then the above is true.

The description here http://stackoverflow.com/questions/12703387/pdf-font-encoding
explains how most tools fail to extract text from PDFs such as this. Unfortunately, the options given in the answer on that page are rather limited.

If you can successfully copy/paste text from the PDF, then the above may not be the case.

[Rob1080]

I think I have an issue related to this (if it doesn't seem related,please let me know and I'll create a new issue).
When writing a final doc (merged some pages from other pdfs) I get an error

UnicodeDecodeError: 'utf8' codec can't decode byte 0xae in position 13: invalid start byte

The it's caused by this: /ARX#20CoSign\xae'

The code where the issue occurs is in generic.py, line 471

return NameObject(name.decode('utf-8'))

To "fix" it, I changed the code to

    try:
            name = name.decode('utf-8')
   except Exception, e:
            pass

   return NameObject(name)

I know it's a hack, but it seemed to resolve the issue I was having. The pdf generates after making this change.

[caxap]

Hey @Rob1080, thanks for your fix. I have same problem with '/ARX#20CoSign\xae' name. Are you found any problems with this fix? Or is there a better solution?

Also I think this fix can be slightly improved:

    try:
        name = name.decode('utf-8')
    except UnicodeDecodeError:
        pass
    return NameObject(name)

[Rob1080]

@caxap It's working fine so far. Fix was merged here: https://github.com/mstamy2/PyPDF2/pull/52/files

[caxap]

@Rob1080 thanks again. Updated to new version.

[hongquan]

This PDF is generated by xhtml2pdf 0.0.6, but is not read properly by PyPDF2 https://drive.google.com/file/d/0ByKUrmVuTXp5RndEVWYxYWM2dGc/view?usp=sharing

The Unicode text there is distorted. The u'Nguy\u1ec5n H\u1ed3ng Qu\xe2n' text is read as Nguy\u2026n H\u2014ng Qu\u2013n

[rahatUlAin]

I am also getting result like this while trying to read a pdf file with unicode urdu using pyPDF2 .It(text) get copied and pasted but the words get inverted like 'urdu' become 'udru'(this for your convenience other wise its more like یقین->نیقی )
\n\n\n\n\n \n \n \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n2|14|\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\

[AzizieAbuduaini]

what i have tried is to replace specific " ' " unicode with "’" which solves this issue. Please let me know if u still failed to generate pdf with this approach.
text = text.replace("'", "’")

[MartinThoma]

We recently improved the situation a lot with #924 (release will be today). Without a specific PDF file, I cannot verify if this issue is solved. I assume it is. Please open another issue if that is not the case.