DuckDB erroneously pushes incomplete results to S3 following query failure

[onderkalaci]

What happens?

Even-though a query fails, the httpfs extension pushing the incomplete results to S3.

To Reproduce

This is a simplified example where the generate_series fail for negative values, but due to destructor pushing results to S3, some incomplete data is written to S3.

For parquet:

COPY (
	SELECT sqrt(generate_series.generate_series)
	FROM generate_series(1000000,-12, -1)
     ) TO 's3://mybucket/tmp/ft5/ft202.parquet';
Out of Range Error: cannot take square root of a negative number

SELECT count(*) FROM
read_parquet('s3://mybucket/tmp/ft5/ft202.parquet');
Invalid Input Error: No magic bytes found at end of file 's3://mybucket/tmp/ft5/ft202.parquet'

for csv/json, see that results are wrong:

D COPY (SELECT sqrt(generate_series.generate_series) FROM generate_series(20000,-20000, -1)) TO 's3://mybucket/tmp/ft5/ft17.json';
Out of Range Error: cannot take square root of a negative number
D select count(*) FROM read_json('s3://mybucket/tmp/ft5/ft17.json');
┌──────────────┐
│ count_star() │
│    int64     │
├──────────────┤
│        18432 │
└──────────────┘
D 

OS:

Both MacOs, and Ubuntu 22.04.4 LTS on WSL2

DuckDB Version:

0.10.2, main

DuckDB Client:

CLI

Full Name:

Onder KALACI

Affiliation:

CrunchyData

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a source build

Did you include all relevant data sets for reproducing the issue?

Yes

Did you include all code required to reproduce the issue?

• Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

• Yes, I have