-
Notifications
You must be signed in to change notification settings - Fork 410
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(services/webhdfs): Implement multi write via CONCAT #3939
Conversation
a65577d
to
1ab2c4b
Compare
1ab2c4b
to
086c023
Compare
3fb60a9
to
e646b8e
Compare
ecdf1a2
to
fa17eee
Compare
I run the behavior test locally and I succeed, what are the reasons for failure in CI? |
Write failed for: 2024-01-09T09:03:45.037494Z WARN opendal::services: service=webhdfs operation=Writer::close path=3bfb3b38-eb71-4a3d-8699-774887cd431c written=10485760B -> data close failed: NotFound (permanent) at Writer::close, context: { uri: http://127.0.0.1:9870/webhdfs/v1/07b41151-d577-442f-8505-a6fd7bb6063e/3bfb3b38-eb71-4a3d-8699-774887cd431c?op=CONCAT&sources=/07b41151-d577-442f-8505-a6fd7bb6063e/da42e868-6285-4745-a88d-b2dd2d826eb2%2C/07b41151-d577-442f-8505-a6fd7bb6063e/92daa91e-9513-4165-8f8b-0d91fa97175b, response: Parts { status: 404, version: HTTP/1.1, headers: {"date": "Tue, 09 Jan 2024 09:03:45 GMT", "date": "Tue, 09 Jan 2024 09:03:45 GMT", "cache-control": "no-cache", "expires": "Tue, 09 Jan 2024 09:03:45 GMT", "pragma": "no-cache", "x-frame-options": "SAMEORIGIN", "content-type": "application/json", "transfer-encoding": "chunked"} }, service: webhdfs, path: 3bfb3b38-eb71-4a3d-8699-774887cd431c } => WebHdfsError { exception: "FileNotFoundException", message: "File does not exist: /07b41151-d577-442f-8505-a6fd7bb6063e/92daa91e-9513-4165-8f8b-0d91fa97175b", java_class_name: "java.io.FileNotFoundException" } |
I encountered this error, but it's not necessarily reproducible. Running the test that failed on its own doesn't show any issues. |
Seems the error here is not checked? You can fix block_write related staffs in this PR directly. |
afd4795
to
4a9ac87
Compare
Still have a problem. |
94e19f7
to
cca2f84
Compare
@Xuanwo The issue is caused by automatic redirection, now you can proceed with the merge. |
7e2e064
to
1814dd0
Compare
5216f05
to
cadf550
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot!
dd88098
to
d9259cd
Compare
d9259cd
to
56402b4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you so much for your patience!
fixed #3924