I strongly suspect the appender api silently fail when error

[jrouaix]

Hello

I had no Err() when using append_row(params![None])) when the target column is indeed NOT NULL.
And did not find a way to check how many lines were actually inserted ...

Does it sound like something fishy here or was it the exepected/normal behavior ?

Got it in duckdb-rs 0.8 but could not really reproduce with latest version cause it has other issues (Assertion 'types == input.GetTypes()' failed panicking from cpp side)

duckdb = { version = "0.8", features = [
  "bundled",
  "modern-full",
  "parquet",
  "json",
  "r2d2",
] }
libduckdb-sys = { version = "0.8", features = ["bundled", "parquet", "json"] }

[[package]]
name = "duckdb"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "393c174261032fbed2d1a67ae78fd3aa5e9a42687a23bb105b2a00a646a36dda"

[[package]]
name = "libduckdb-sys"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "84f1e0bc90f97fb7fe64f5d1d8450f4eef8db725efcf7a29a30b7fdfd2705d81"

[erikvullings]

@jrouaix I agree with your suspicion. I've used the appender to insert 650_000 simulated persons into duckdb, and noticed a couple of things:

• Loading was VERY slow (compared to sqlite)
• After calling appender's flush method, the data should have been written to the database. However, it was not (as confirmed by a select count query on the table).
• When inserting data using sqlite, the unique ID constraint was violated several times. Either appender was too slow, or it never reported it (or the underlying C implementation never checked it).

[jrouaix]

So we didn't yet updated further than 8.0, but perhaps version 10 is freed from this bug.
Will try ... one day soon 🤞🏼

[erikvullings]

@jrouaix Unfortunately, I have tried this with the latest version, 0.10.1.

[era127]

The appender api should error as documented in this issue

However it should not be much slower than SQLite, is it possible to share a mwe.

[Swoorup]

I found if you are using the arrow version, try flushing it every <2048 len elements to work.