Unexpected 401 when using SharePoint WebDav NTLM authentication

[AliveDevil]

With multiple connections opened for transferring items, there is a concurrency issue with the NTLM authentication state shared in the shared request context.

IMPORTANT: Please note HTTP context implementation, even when thread safe, may not be used concurrently by multiple threads, as the context may contain thread unsafe attributes.

HttpContext (Apache HttpComponents Core HTTP/1.1 5.2.2 API)

1. SardineImpl uses singular cached HttpContext for all requests¹
2. Only one DavClient extends SardineImpl is used for all Dav requests²
3. SardineImpl uses a request local HttpClientContext³
4. But at the same time uses a broad-scoped AuthState-cache ⁴ which is the underlying issue: One AuthState-cache for all request going out. This would need to be configurable or request local scoped, instead of global.

Footnotes

1. https://github.com/iterate-ch/sardine/blob/f519c959582d9d461927100cc444ea996e209232/src/main/java/com/github/sardine/impl/SardineImpl.java#L169 ↩

2. https://github.com/iterate-ch/cyberduck/blob/ebb964cd6761dfb286e44e931bde41072bd9d5a1/webdav/src/main/java/ch/cyberduck/core/dav/DAVSession.java#L107 ↩

3. https://github.com/iterate-ch/sardine/blob/f519c959582d9d461927100cc444ea996e209232/src/main/java/com/github/sardine/impl/SardineImpl.java#L1055 ↩

4. https://github.com/iterate-ch/sardine/blob/f519c959582d9d461927100cc444ea996e209232/src/main/java/com/github/sardine/impl/SardineImpl.java#L258-L262 ↩

[dkocher]

Previously in ¹ with follow up in ²

Footnotes

1. SardineImpl not thread safe - HttpClientContext reused and mutated lookfirst/sardine#263 ↩

2. Persist user token in shared execution context. Fix connection statef… lookfirst/sardine#268 ↩

[dkocher]

The specific issue with NTLM authentication is that connections are stateful indicated by the Persistent-Auth: true response header.

The NTLM authentication scheme is significantly more expensive in terms of computational overhead and performance impact than the standard Basic and Digest schemes. This is likely to be one of the main reasons why Microsoft chose to make NTLM authentication scheme stateful. That is, once authenticated, the user identity is associated with that connection for its entire life span. The stateful nature of NTLM connections makes connection persistence more complex, as for the obvious reason persistent NTLM connections may not be re-used by users with a different user identity. The standard connection managers shipped with HttpClient are fully capable of managing stateful connections

Disabling connection reuse ¹ will immediately show a 401 response.

Footnotes

1. https://github.com/iterate-ch/cyberduck/blob/a4c381b15053215cc07f148eb60f4e0154e12ce7/core/src/main/java/ch/cyberduck/core/http/HttpConnectionPoolBuilder.java#L156-L161 ↩

[dkocher]

Documentation to be updated with iterate-ch/docs#518 until a resolution is found.

[DmitryDemidov]

Is there any expected timeframe for fixing this error?

[dkocher]

The authentication cache in the client context is only populated for the BASIC authentication scheme ¹.

Footnotes

1. https://github.com/mydevotion/httpclient/blob/master/httpclient/src/main/java/org/apache/http/impl/client/AuthenticationStrategyImpl.java#L220C1-L221C1 ↩

[dkocher]

Is there any expected timeframe for fixing this error?

We plan to include a fix in upcoming 8.9.0.

[DmitryDemidov]

Is there any expected timeframe for fixing this error?

We plan to include a fix in upcoming 8.9.0.

Thanks!