New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement request : parametrize internal decode of VARCHAR2 data in Python 3 #162

Closed
shoyu63 opened this Issue Mar 20, 2018 · 5 comments

Comments

Projects
None yet
3 participants
@shoyu63
Copy link

shoyu63 commented Mar 20, 2018

When there are corrupted data in a VARCHAR2 (for example \x90 in a WINDOWS-1252 database), we can't fetch the corrupted record because of a cx_Oracle internal decode error.
For example:

(<class 'UnicodeDecodeError'>, UnicodeDecodeError('charmap', b'ELLE REVIENDRA VE
RS NOUS - BIEN PLAC\x90\xc9S - ELLE ATTEND SON CLIENT', 36, 37, 'character maps
to <undefined>'), <traceback object at 0x0000000003C6B5C8>)

With Python 3, there is no way in cx_Oracle (v6.2) to specify the "errors" parameter to the decode() that takes place internally.

It might be nice to be able to

  1. set this parameter.
    and/or
  2. bypass the internal "decode" to have a direct access to the binary data of the STRING and FIXED_CHAR fields.

Thank you.
Stéphane

@cjbj cjbj added the enhancement label Mar 20, 2018

anthony-tuininga added a commit that referenced this issue Jun 19, 2018

Add support for specifying the "errors" parameter to the decode() tha…
…t takes

place internally when fetching strings from the database
(#162).
@anthony-tuininga

This comment has been minimized.

Copy link
Member

anthony-tuininga commented Jun 19, 2018

Using code like the following you can now work around bad data in your database. Let me know if that works for you!

def OutputTypeHandler(cursor, name, defaultType, size, precision, scale):
    if defaultType == cx_Oracle.STRING:
        return cursor.var(defaultType, size, arraysize = cursor.arraysize,
                encodingErrors = "replace")

cursor.outputtypehandler = OutputTypeHandler

cursor.execute("select column1, column2 from SomeTableWithBadData")
@shoyu63

This comment has been minimized.

Copy link
Author

shoyu63 commented Jun 20, 2018

Hello,
It looks great for me. I will use it as soon as the new version is available.
Thank you.
Stéphane.

@cjbj

This comment has been minimized.

Copy link
Member

cjbj commented Jun 20, 2018

@anthony-tuininga is there any performance impact by having this always enabled?

@anthony-tuininga

This comment has been minimized.

Copy link
Member

anthony-tuininga commented Jun 20, 2018

The default error handler is "strict" but there are other error handlers besides "replace" which can be seen here: https://docs.python.org/3/library/codecs.html#error-handlers. There is also the possibility of implementing your own error handler. So I'm not sure what "always enabled" would look like!

@anthony-tuininga

This comment has been minimized.

Copy link
Member

anthony-tuininga commented Jul 2, 2018

cx_Oracle 6.4 has been released with this enhancement included in it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment