Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Batch-inserting via executemany() fails with NotSupportedError: Python value cannot be converted to a database value #236
Batch-inserting via executemany() fails with
@anthony-tuininga thank you. I appreciate that very much.
I managed to reproduce the issue on a complete seperate environment (seperate from our corporate Windows Environment). On my linux home computer, I setup a linux container and installed the free Oracle Express Edition Server 11.2.0-1.0 on Archlinux with Kernel 4.14.78-1-lts (amd64).
As client, I used Archlinux on another machine with python 3.7.1, cx_Oracle 7.0.0 and Oracle Instant Client (Basic Package) 22.214.171.124.0
Note: Using the dataset from above on linux, I ran into some utf-8 encoding issues, that I don't think is relevant here, so I replaced the problematic contents with randomized data and uploaded a new dataset
I now understand why this is happening. In your first batch two of your columns never have any value other than None. cx_Oracle defers type assignment until the end of the batch and then simply assumes None is the only value possible (string with maximum length 1). In your second batch at least one of the rows actually has a value, but that value isn't compatible. The error message is very misleading, however, so I'll see if I can address that. You have a few options:
(1) perform all inserts in a single batch (but depending on size this might not be usable)
Of those three options, the second option is the simplest and will only add a small bit of overhead. The third option is the one you'll want to use for the best performance.
Just for clarity, here is a simple test case that demonstrates the issue:
import cx_Oracle conn = cx_Oracle.connect("cx_Oracle/welcome") cursor = conn.cursor() cursor.execute("truncate table TestTempTable") sql = "insert into TestTempTable values (:1, :2)" cursor.executemany(sql, [(1, None), (2, None)]) cursor.executemany(sql, [(3, None), (4, "Testing")])
The second call to executemany() will fail with the error you reported.
added a commit
Nov 30, 2018
@anthony-tuininga thank you very much. I don't have the possibility to install cx_Oracle from source in our environment, but I will happily await till it gets packaged for my distribution.
meanwhile I will try out your workaround number 2) and create a new cursor for each batch.
Thanks a lot :-)