NoMetadata error when reading detlatable

[finlaydotb]

Environment

Delta-rs version: 0.13.0

Binding:

Environment:

• Cloud provider: Local
• OS: MacOS
• Other:

---

Bug

What happened:

• I write to a detla table,
• I can successfully read the table and query using datafusion
• I repeat the same write in the first step
• Now when I try to read using the same code that initially worked, I now get the error

thread 'main' panicked at 'called Result::unwrap() on an Err value: NoMetadata', /Users/me/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deltalake-0.13.1/src/delta_datafusion.rs:432:35
stack backtrace:
0: rust_begin_unwind

What you expected to happen:

To be able to read the delta lake table

How to reproduce it:

Write to a delta table path using code similar to this

    let table = DeltaTableBuilder::from_uri(path)
        .with_storage_options(storage_options)
        .build()
        .unwrap();
    
    let ops = DeltaOps::from(table);

    ops.write(vec![...]).await?;

The query using datafusion with code similar to this

    let mut table = deltalake::open_table_with_storage_options("s3://path/to/table", get_storage_options())
            .await?;

    let mut ctx = SessionContext::new();
    ctx.register_table("demo", Arc::new(table)).unwrap();
    let df = ctx.sql("select * from test").await?;
    df.show().await?;

now repeat the write, and then the reading, the reading will now fail with the above error.

[Blajda]

I'm assuming this issue is the same as outlined on stack overflow

This is the expected behavior when using the build method. See the docs for this method

The first write creates the first log file and associated structure of the table. There is no issue since nothing exists. The second call to build() will construct the DeltaTable struct without reading the underlying storage for the table state hence the writer see it as being uninitialized and attempts to create the table again. Before writing the commit conflict resolution will check for new commits to the table and it notices that a version 0 already exists which conflicts with your new write and returns an error.

When reading an existing table use load() with the builder or use the DeltaOps write as follows

        let table = DeltaOps::try_from_uri("./data/mytable")
            .await?
            .write(mydata)
            .await?

Once you have a table object you don't need to load the table again. You can reuse it like this

        let table = DeltaOps(table)
            .write(vec![batch])
            .await?;