bug: onedrive chunked upload fail with unauthenticated error #5811
Description
Describe the bug
onedrive chunked upload fail with unauthenticated error in onedrive_chunked_upload
The root cause seems to be a confliction between tempauth in the upload url and the Authorization header universally applied for Onedrive requests. Because the request succeed after removing the Authorization header.
Example upload url: https://my.microsoftpersonalcontent.com/personal/<...>/_api/v2.0/drive/items/<...>/uploadSession?guid='<...>'&dc=0&tempauth=<...>
But looks it's not always the case. @erickguan tried and succeeded in #5690 (for me it always fail).
Microsoft official doc explicitly states "SHOULD NOT" include bearer header, so maybe we can safely remove it.
Steps to Reproduce
use std::env;
use anyhow::Ok;
use opendal::{layers::TracingLayer, services::Onedrive, Builder, OperatorBuilder};
#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
tracing_subscriber::fmt::init();
let onedrive = Onedrive::default()
.access_token(env::var("ACCESS_TOKEN")?.as_str());
let operator = OperatorBuilder::new(onedrive.build()?)
.layer(TracingLayer{})
.finish();
let mut data = Vec::with_capacity(5*1024*1024); // 5MB
let dummy: [u8; 16] = [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15];
for _ in 0..5*1024*1024/16 {
data.extend_from_slice(&dummy);
}
operator.write("/test.data", data).await?;
Ok(())
}[package]
name = "opendal_failcase"
version = "0.1.0"
edition = "2024"
[dependencies]
anyhow = "1.0.97"
log = "0.4.26"
opendal = { path = "./path/to/opendal/core", features = ["layers-tracing", "services-onedrive"] }
tokio = { version = "1.44.1", features = ["full"] }
tracing-subscriber = "0.3.19"Expected Behavior
it should work.
Additional Context
The log:
cargo run
Compiling opendal v0.52.0 (/Users/emliunix/Documents/paperfs.rs/opendal/core)
Compiling opendal_failcase v0.1.0 (/Users/emliunix/Documents/opendal_failcase)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 4.54s
Running `target/debug/opendal_failcase`
2025-03-18T13:17:21.672339Z WARN opendal::layers::complete: writer has not been closed or aborted, must be a bug
Error: Unexpected (permanent) at writer::close, context: { uri: https://my.microsoftpersonalcontent.com/personal/<...>/_api/v2.0/drive/items/<...>/uploadSession?guid='<...>'&dc=0&tempauth=<...>, response: Parts { status: 401, version: HTTP/1.1, headers: {"cache-control": "private, max-age=0", "content-length": "64", "content-type": "application/json", "expires": "Mon, 03 Mar 2025 14:17:21 GMT", "last-modified": "Tue, 18 Mar 2025 13:17:21 GMT", "vary": "Origin", "p3p": "CP=\"ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI\"", "date": "Tue, 18 Mar 2025 13:17:20 GMT", "isocdi": "0", "x-networkstatistics": "1,8389440,310,11785,4370905,4204800,4204800,154633", "x-sharepointhealthscore": "0", "x-aspnet-version": "4.0.30319", "x-databoundary": "NONE", "x-1dscollectorurl": "https://mobile.events.data.microsoft.com/OneCollector/1.0/", "x-ariacollectorurl": "https://browser.pipe.aria.microsoft.com/Collector/3.0/", "sprequestguid": "3ab98ba1-b039-8000-2621-a22ae26737db", "request-id": "3ab98ba1-b039-8000-2621-a22ae26737db", "ms-cv": "oYu5OjmwAIAmIaIq4mc32w.0", "alt-svc": "h3=\":443\";ma=86400", "strict-transport-security": "max-age=31536000", "x-frame-options": "SAMEORIGIN", "content-security-policy": "frame-ancestors 'self' teams.microsoft.com *.teams.microsoft.com *.skype.com *.teams.microsoft.us local.teams.office.com teams.cloud.microsoft *.office365.com goals.cloud.microsoft *.powerapps.com *.powerbi.com *.yammer.com engage.cloud.microsoft word.cloud.microsoft excel.cloud.microsoft powerpoint.cloud.microsoft *.officeapps.live.com *.office.com *.microsoft365.com m365.cloud.microsoft *.cloud.microsoft *.stream.azure-test.net *.microsoftstream.com *.dynamics.com *.microsoft.com onedrive.live.com *.onedrive.live.com securebroker.sharepointonline.com;", "x-ms-suspended-features": "features=\"\"", "x-powered-by": "ASP.NET", "microsoftsharepointteamservices": "16.0.0.25905", "x-content-type-options": "nosniff", "x-ms-invokeapp": "1; RequireReadOnly", "x-cache": "CONFIG_NOCACHE", "x-msedge-ref": "Ref A: 8CA4CAAA878644ABBD07B81651294778 Ref B: TYO01EDGE3706 Ref C: 2025-03-18T13:17:20Z"} }, service: onedrive, path: test.data, written: 5242880 } => {"error":{"code":"unauthenticated","message":"Unauthenticated"}}
Are you willing to submit a PR to fix this bug?
- Yes, I would like to submit a PR.