SQLITE_BUSY in CSVWriter

[tballison]

WARN [ForkJoinPool.commonPool-worker-1] 08:13:20,805
org.apache.tika.pipes.reporters.jdbc.JDBCPipesReporter problem writing
to the db. Will try to reconnect
org.sqlite.SQLiteException: [SQLITE_BUSY] The database file is locked
(database is locked)
at org.sqlite.core.DB.newSQLException(DB.java:1135)
~[sqlite-jdbc-3.40.0.0.jar:?]
at org.sqlite.core.DB.newSQLException(DB.java:1146)
~[sqlite-jdbc-3.40.0.0.jar:?]
at org.sqlite.core.DB.throwex(DB.java:1106)
~[sqlite-jdbc-3.40.0.0.jar:?]
at org.sqlite.core.DB.executeBatch(DB.java:907)
~[sqlite-jdbc-3.40.0.0.jar:?]
at org.sqlite.core.DB.lambda$executeBatch$0(DB.java:872)
~[sqlite-jdbc-3.40.0.0.jar:?]
at org.sqlite.core.SafeStmtPtr.safeRun(SafeStmtPtr.java:128)
~[sqlite-jdbc-3.40.0.0.jar:?]
at org.sqlite.core.DB.executeBatch(DB.java:872)
~[sqlite-jdbc-3.40.0.0.jar:?]
at org.sqlite.core.CorePreparedStatement.lambda$executeLargeBatch$1(CorePreparedStatement.java:76)
~[sqlite-jdbc-3.40.0.0.jar:?]
at org.sqlite.jdbc3.JDBC3Statement.withConnectionTimeout(JDBC3Statement.java:454)
~[sqlite-jdbc-3.40.0.0.jar:?]
at org.sqlite.core.CorePreparedStatement.executeLargeBatch(CorePreparedStatement.java:72)
~[sqlite-jdbc-3.40.0.0.jar:?]
at org.sqlite.core.CorePreparedStatement.executeBatch(CorePreparedStatement.java:58)
~[sqlite-jdbc-3.40.0.0.jar:?]
at org.apache.tika.pipes.reporters.jdbc.JDBCPipesReporter$ReportWorker.reportNow(JDBCPipesReporter.java:232)
~[tika-pipes-reporter-jdbc-2.6.1-20221128.201713-28.jar:?]
at org.apache.tika.pipes.reporters.jdbc.JDBCPipesReporter$ReportWorker.run(JDBCPipesReporter.java:211)
~[tika-pipes-reporter-jdbc-2.6.1-20221128.201713-28.jar:?]
at java.util.concurrent.CompletableFuture$AsyncRun.run(Unknown
Source) ~[?:?]
at java.util.concurrent.CompletableFuture$AsyncRun.exec(Unknown
Source) ~[?:?]
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) ~[?:?]
at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown
Source) ~[?:?]
at java.util.concurrent.ForkJoinPool.scan(Unknown Source) ~[?:?]
at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) ~[?:?]
at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) ~[?:?]

[tballison]

Not sure why this is happening. The cause may be that the main pipes application has two connections open, one from the emitter at the pipes level and one for the reporter. Apparently two writing connections within a jvm is "not good"(TM) for sqlite.

I had tried h2 with AUTO_SERVER=TRUE, but that failed for a reason I can't remember. I'm wondering if we should perhaps start a local h2 server and have the pipes connect to that. IIRC, I think the problem was that we started a connection and then shut it down from the main thread(?), leaving some unpleasant competition among the other processes?