depr(duckdb): deprecate read_in_memory

[gforsyth]

Description of changes

This functionality is available via create_table (and also available
across all backends instead of just DuckDB).

[gforsyth]

ugh, I forgot this is also in polars and maybe datafusion

[cpcloud]

By deprecating this are we now requiring that people effectively copy their data into backends that can operate directly on the in-memory data?

[gforsyth]

I don't think so. It's currently only available on the DuckDB backend.

The suggested alternative works on all backends (except Druid) where a user can pass an in-memory object to create_table and operate on it from there.

[cpcloud]

Oh, also I forgot about memtable 😅 which definitely does not use create_table with the DuckDB backend!

[gforsyth]

@cpcloud -- anything else you'd like to see here?

[cpcloud]

Nope, this LGTM!

For some reason I incorrectly assumed this was a breaking change, but it's not.

[acuitymd-filip]

It seems like a breaking change. Our use-case is registering Arrow tables and using DuckDB as a strictly in-memory query engine for Arrow data. Using register_in_memory we can pass in a pointer to Arrow data and register it with DuckDB without doing memory copies, etc.

The original functionality was basically equivalent to duckdb.register which created a view inside of DuckDB, effectively, which then referenced the pointer to the arrow table.

The recommended upgrade path creates a table in DuckDB, which involves memory copies, even if the temp flag is set to true.

[acuitymd-filip]

@gforsyth Would you prefer if I create an issue for this ^?

[gforsyth]

Hey @acuitymd-filip -- you can open an issue if you'd like to discuss this further (that will probably be better for searches, anyway).

I can update the options in the upgrade path. If you use ibis.memtable with the DuckDB backend, we don't make any copies of the data and DuckDB doesn't make copies when accessing it.