test_repair_on_update broken in main #2109
Labels
binding/rust
Issues for the Rust crate
bug
Something isn't working
storage/aws
AWS S3 storage related
Comments
rtyler
added
bug
rtyler
added a commit
to rtyler/delta-rs
that referenced
this issue
Jan 23, 2024
This unbreaks main, but this does need to get re-enabled as soon as possible See delta-io#2109
|
one way - and from what i see today probably the best option right now - is to move to using the logstore abstraction consistently, at which point repaoi on write will just re-appear. |
rtyler
added a commit
that referenced
this issue
Jan 24, 2024
This unbreaks main, but this does need to get re-enabled as soon as possible See #2109
rtyler
added a commit
that referenced
this issue
Jan 30, 2024
…ad (#2120) # Description Make sure the read path for delta table commit entries passes through the log store, enabling it to ensure the invariants and potentially repair a broken commit in the context of S3 / DynamoDb log store implementation. This also adds another test in the context of S3 log store: repairing a log store on load was not implemented previously. Note that this a stopgap and not a complete solution: it comes with a performance penalty as we're triggering a redundant object store list operation just for the purpose of "triggering" the log store functionality. fixes #2109 --------- Co-authored-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com> Co-authored-by: R. Tyler Croy <rtyler@brokenco.de>
rtyler
added a commit
to dispanser/delta-rs
that referenced
this issue
Jan 31, 2024
…ad (delta-io#2120) # Description Make sure the read path for delta table commit entries passes through the log store, enabling it to ensure the invariants and potentially repair a broken commit in the context of S3 / DynamoDb log store implementation. This also adds another test in the context of S3 log store: repairing a log store on load was not implemented previously. Note that this a stopgap and not a complete solution: it comes with a performance penalty as we're triggering a redundant object store list operation just for the purpose of "triggering" the log store functionality. fixes delta-io#2109 --------- Co-authored-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com> Co-authored-by: R. Tyler Croy <rtyler@brokenco.de>
RobinLin666
pushed a commit
to RobinLin666/delta-rs
that referenced
this issue
Feb 2, 2024
This unbreaks main, but this does need to get re-enabled as soon as possible See delta-io#2109
RobinLin666
pushed a commit
to RobinLin666/delta-rs
that referenced
this issue
Feb 2, 2024
…ad (delta-io#2120) # Description Make sure the read path for delta table commit entries passes through the log store, enabling it to ensure the invariants and potentially repair a broken commit in the context of S3 / DynamoDb log store implementation. This also adds another test in the context of S3 log store: repairing a log store on load was not implemented previously. Note that this a stopgap and not a complete solution: it comes with a performance penalty as we're triggering a redundant object store list operation just for the purpose of "triggering" the log store functionality. fixes delta-io#2109 --------- Co-authored-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com> Co-authored-by: R. Tyler Croy <rtyler@brokenco.de>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The deltalake-aws crate has an integration test failure with the recent refactoring of @roeap , this issue is to track the fixing of it.
The text was updated successfully, but these errors were encountered: