NTLM authentication should be supported

[cyberduck]

georgmaass created the issue

To use Cyberduck with Microsoft Sharepoint Services it should support NTLM authentication. To support only BASIC authentication, if the server requires NTLM, is not enough.

Also the error message simply saying "login failed" is not user friendly, because this gives no idea to the user, why it fails. The reason, why it fails is simple and can be detected easily by the software. The authentication method requested by the server using the WWW-Authenticate HTTP header is not in the list of implememnted authentication methods, so the error message in such a case should not simply say "Login failed" but should say the reason why it failed: Could not negotiate authentication method.

I used wireshark to see, why it failed. What I see is: Cyberduck tries BASIC but the server wants NTLM.

[cyberduck]

@dkocher commented

Added NTLM credentials in c4d7db4. For the domain we currently use the default path set in the bookmark. Please try the latest nightly build and let me know if it works for Sharepoint.

[cyberduck]

georgmaass commented

Even with 6a4e6ed it does not work. It now really tries NTLM but it does not succeed even if I abuse the path field to enter the domain. I always get a 401 response.

[cyberduck]

@dkocher commented

Replying to [comment:2 georgmaass]:

Even with 6a4e6ed it does not work. It now really tries NTLM but it does not succeed even if I abuse the path field to enter the domain. I always get a 401 response.

Can you post the transcript from the log drawer, please.

[cyberduck]

anonymous commented

Now I get a 404 response after some 401 respones, which is what I expect, because the domain is an invalid path.

HTTP/1.1 404 [\r][\n]
Server: Microsoft-IIS/6.0[\r][\n]
X-Powered-By: ASP.NET[\r][\n]
MicrosoftSharePointTeamServices: 12.0.0.6219[\r][\n]
Date: Wed, 31 Dec 2008 15:39:22 GMT[\r][\n]
Connection: close[\r][\n]
[\r][\n]

Why must we specify the domain? Isn't it sent in the NTLM challenge? If I use SeaMonkey composer with WebDAV to publish I never specify a domain even if asked in a three text field dialog asking for domain, username and password even there I leave the domain empty. Probably SeaMonkey then takes it from the NTLM challange.

The server response contains the domain within the base64 encoded data. From there you can take it. It is not necessary to abuse the path for this purpose. After decoding the base64 encoded response you find the domain beginning somewhere about byte 20.

[cyberduck]

@dkocher commented

Replying to [comment:4 anonymous]:

I have now changed the implementation in 8808fbd to send an empty string for the domain. It can be overwritten by the property webdav.ntlm.domain using the defaults command:

defaults write ch.sudo.cyberduck webdav.ntlm.domain MYDOMAIN

The default path setting is no longer misused.

I am no expert for NTLM but I assume the client has to send the credentials first before the server ever responding with a domain.

[cyberduck]

anonymous commented

There is now a new problem with e7f84e0:

As result of this request:

PROPFIND /beeprm/documents HTTP/1.1
Depth: 0

I got this response, which I read as "Object /beeprm/documents is a collection (folder) last modified at 2008-12-19T14:59:39 with status 200 OK", but Cyberduck looks at the 404 status below and therefor thinks that this does not exist.

HTTP/1.1 207 MULTI-STATUS
Date: Wed, 31 Dec 2008 17:54:18 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
MicrosoftSharePointTeamServices: 12.0.0.6219
Cache-Control: no-cache
Content-Type: text/xml
Content-Length: 678
Public-Extension: http://schemas.microsoft.com/repl-2
Set-Cookie: WSS_KeepSessionAuthenticated=80; path=/

<D:response><D:href>http://beeprm.de/beeprm/documents/</D:href><D:propstat><D:prop><D:displayname></D:displayname><D:resourcetype><D:collection/></D:resourcetype><D:getlastmodified>2008-12-19T14:59:39Z</D:getlastmodified><D:lockdiscovery/></D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
<D:propstat>
<D:prop>
<D:getcontentlength/><D:getcontenttype/></D:prop>
<D:status>HTTP/1.1 404 Not Found</D:status>
</D:propstat></D:response>
</D:multistatus>

Next Cyberduck appends the path a second time to the request URI

PROPFIND /beeprm/documents/beeprm/documents/ HTTP/1.1
Depth: 0

This is nonsence an now it correctly gets the response

HTTP/1.1 404 NOT FOUND
Date: Wed, 31 Dec 2008 17:54:21 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
MicrosoftSharePointTeamServices: 12.0.0.6219
Exires: Tue, 16 Dec 2008 17:54:21 GMT
Cache-Control: private,max-age=0
Content-Length: 0
Public-Extension: http://schemas.microsoft.com/repl-2
Set-Cookie: WSS_KeepSessionAuthenticated=80; path=/
X-MSDAVEXT_Error: 589831; The%20folder%20that%20would%20hold%20URL%20%27beeprm%2fdocuments%27%20does%20not%20exist%20on%20the%20server%2e

I also see in this response that the Microsoft IIS/6.0 has a bug in the naming of the Expires HTTP header because the "p" is missing. A stupid typo! Shame on Microsoft. ;-)

[cyberduck]

georgmaass commented

Very interesting: when I use an existing subdirectory as path "/beeprm/documents/externaldebug" it works well. So there seems to be a little bit fine tuning necessary. The WebDAV root on this server is "/beeprm/documents" if a path above is specified it is outside the WebDAV enabled application. This is probably the reason for the confusing response on the request

PROPFIND /beeprm/documents HTTP/1.1
Depth: 0
<?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
   <D:prop><D:displayname/><D:getcontentlength/><D:getcontenttype/><D:resourcetype/><D:getlastmodified/><D:lockdiscovery/></D:prop>
</D:propfind>

which contains a 200 status and a 404 status.

[cyberduck]

@dkocher commented

Just a wild guess. Maybe you have to reference the trailing slash like /beeprm/documents/`.

[cyberduck]

georgmaass commented

No, the slash does not make a difference.

The log contains only the HTTP headers, so I can not find the error message text there, even in wireshark I do not find the error message text.

Here the last request / response in the log:

PROPFIND /beeprm/documents/ HTTP/1.1[\r][\n]
Content-Type: text/xml; charset=utf-8[\r][\n]
User-Agent: Cyberduck/3.1 (4435)[\r][\n]
Content-Length: 207[\r][\n]
Authorization: NTLM TlRMTVNTUAADAAAAGAAYAFEAAAAAAAAAaQAAAAAAAABAAAAABgAGAEAAAAALAAsARgAAAAAAAABpAAAABlIAAEdNQUFTU0FFSFJFLkxPQ0FMm3D0Ytwzi04xcerOML83p/mxkuLX+6zj[\r][\n]
Host: beeprm.de[\r][\n]
Cookie: $Version=0; WSS_KeepSessionAuthenticated=80; $Path=/[\r][\n]
Depth: 1[\r][\n]
[\r][\n]

HTTP/1.1 207 MULTI-STATUS[\r][\n]
HTTP/1.1 207 MULTI-STATUS[\r][\n]
Date: Thu, 01 Jan 2009 09:14:25 GMT[\r][\n]
Server: Microsoft-IIS/6.0[\r][\n]
X-Powered-By: ASP.NET[\r][\n]
MicrosoftSharePointTeamServices: 12.0.0.6219[\r][\n]
Cache-Control: no-cache[\r][\n]
Content-Type: text/xml[\r][\n]
Content-Length: 6277[\r][\n]
Public-Extension: http://schemas.microsoft.com/repl-2[\r][\n]
Set-Cookie: WSS_KeepSessionAuthenticated=80; path=/[\r][\n]
[\r][\n]

This is a part of the response body taken from wireshark and partially formatted nice for better readability:

<?xml version="1.0" encoding="utf-8" ?><D:multistatus xmlns:D="DAV:" xmlns:Office="urn:schemas-microsoft-com:office:office" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:Z="urn:schemas-microsoft-com:">
<D:response>
  <D:href>http://beeprm.de/beeprm/documents/</D:href>
  <D:propstat>
    <D:prop>
      <D:displayname></D:displayname>
      <D:resourcetype><D:collection/></D:resourcetype>
      <D:getlastmodified>2008-12-19T14:59:39Z</D:getlastmodified>
      <D:lockdiscovery/>
    </D:prop>
    <D:status>HTTP/1.1 200 OK</D:status>
  </D:propstat>
  <D:propstat>
    <D:prop>
      <D:getcontentlength/>
      <D:getcontenttype/>
    </D:prop>
    <D:status>HTTP/1.1 404 Not Found</D:status>
  </D:propstat>
</D:response>

<D:response><D:href>http://beeprm.de/beeprm/documents/default.aspx</D:href><D:propstat><D:prop><D:displayname>default.aspx</D:displayname><D:getcontentlength>3991</D:getcontentlength><D:resourcetype/><D:getlastmodified>2008-12-19T14:59:40Z</D:getlastmodified><D:lockdiscovery/></D:prop><D:status>HTTP/1.1 200 OK</D:status></D:propstat><D:propstat><D:prop><D:getcontenttype/></D:prop><D:status>HTTP/1.1 404 Not Found</D:status></D:propstat></D:response>
<D:response><D:href>http://beeprm.de/beeprm/documents/15</D:href><D:propstat><D:prop><D:displayname>15</D:displayname><D:resourcetype><D:collection/></D:resourcetype><D:getlastmodified>2008-12-23T17:17:44Z</D:getlastmodified><D:lockdiscovery/></D:prop>
<D:status>HTTP/1.1 200 OK</D:status>
</D:propstat>
<D:propstat>
<D:prop>
<D:getcontentlength/><D:getcontenttype/></D:prop>
<D:status>HTTP/1.1 404 Not Found</D:status>
</D:propstat></D:response>
</D:multistatus>

I do not understand why this response results in this error messages:

The English error message text is not part of the communication between Cyberduck and the Server. So this seems to be an internal error inside Cyberduck.

Is Cyberduck confused by the collection /beeprm/documents having no properties called "DAV::getcontentlength" and "DAV::getcontenttype"?

[cyberduck]

@dkocher commented

Thanks for the detailed transcript. To catch the transcript from Cyberduck in console.log, set the defaults property logging to DEBUG with

defaults write ch.sudo.cyberduck logging DEBUG

The issue seems to be that the href elements are fully qualified URI whereas only a absolute path is expected. To resolve this issue however I would need to have access to an account for testing on a SharePoint server to debug this.

[cyberduck]

georgmaass commented

Would a virtual machine help you to do your own tests with Microsoft Sharepoint Services 3.0 as WebDAV server?

[cyberduck]

@dkocher commented

Replying to [comment:11 georgmaass]:

Would a virtual machine help you to do your own tests with Microsoft Sharepoint Services 3.0 as WebDAV server?

Yes, that would help a lot! Preferably a VMWare or VirtualBox image.

[cyberduck]

anonymous commented

I now download this VM provided by Microsoft and check it, whether I can launch it using Parallels Desktop.

(http://www.microsoft.com/downloads/details.aspx?FamilyID=DD939ED9-87A5-4C13-B212-A922CC02B469&displaylang=en)

If this does not work or does not reproduce a useful test case, I try to setup a VM from the scratch using Parallels Desktop. I don't know whether VMWare Fusion can convert virtual machines created with Parallels Desktop. An other problem might be the Size, if it is too big I can not upload it to my webserver. Also it my require reregistration at Microsoft, when you change the VM engine. (If I convert a Windows VM from VMWare to Parallels or clone a real computer it requires new registration at Microsoft). So if the VM provided by Microsoft does not fit our needs I probably have to open a port in my firewall and run a self made VM locally at times negotiated with you before. At the moment I'm downloading the Microsoft stuff, which will take some hours.

[cyberduck]

@dkocher commented

The remaining compatibility issue with the blank in an URI returned from the server is a duplicate of issue #2223.

[cyberduck]

isloat commented

I have found a problem related with that.
I'm trying to get the property getcontenttype from a file using the webdav of MOSS 2007.
But I have found the following:

For a propfind operation of a directory, using Depth = infinity. It works OK and the property getcontenttype is returned.

<?xml version="1.0" encoding="utf-8" ?>
<D:multistatus xmlns:D="DAV:" xmlns:Office="urn:schemas-microsoft-com:office:office" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:Z="urn:schemas-microsoft-com:">
	<D:response>
		<D:href>http://site/sharepoint/webdav</D:href>
		<D:propstat>
			<D:prop>
				<D:displayname>webdav</D:displayname>
				<D:lockdiscovery/>
				<D:supportedlock/>
				<D:isFolder>t</D:isFolder>
				<D:iscollection>1</D:iscollection>
				<D:ishidden>0</D:ishidden>
				<D:getcontenttype>application/octet-stream</D:getcontenttype>
				<D:getcontentlength>0</D:getcontentlength>
				<D:resourcetype>
					<D:collection/>
				</D:resourcetype>
				<Repl:authoritative-directory>t</Repl:authoritative-directory>
				<D:getlastmodified>2010-05-28T08:31:10Z</D:getlastmodified>
				<D:creationdate>2010-05-19T10:25:27Z</D:creationdate>
				<Repl:repl-uid>rid:{0CEF145C-3565-46AF-8078-4842B0CC7B83}</Repl:repl-uid>
				<Repl:resourcetag>rt:0CEF145C-3565-46AF-8078-4842B0CC7B83@00000000000</Repl:resourcetag>
				<D:getetag>&quot;{0CEF145C-3565-46AF-8078-4842B0CC7B83},0&quot;</D:getetag>
			</D:prop>
			<D:status>HTTP/1.1 200 OK</D:status>
		</D:propstat>
	</D:response>

As you see the property getcontenttype is returned. But in the same response, the information of a file shows like this:

<D:response>
		<D:href>http://site/sharepoint/webdav/file.txt</D:href>
		<D:propstat>
			<D:prop>
				<D:displayname>file.txt</D:displayname>
				<D:lockdiscovery/>
				<D:supportedlock>
					<D:lockentry>
						<D:lockscope>
							<D:exclusive/>
						</D:lockscope>
						<D:locktype>
							<D:write/>
						</D:locktype>
					</D:lockentry>
				</D:supportedlock>
				<D:getlastmodified>2010-05-28T08:55:16Z</D:getlastmodified>
				<D:creationdate>2010-05-28T08:55:16Z</D:creationdate>
				<D:getcontentlength>60</D:getcontentlength>
				<Repl:repl-uid>rid:{AB3A1AE1-3D65-487C-9122-4ECAEC9CA887}</Repl:repl-uid>
				<Repl:resourcetag>rt:AB3A1AE1-3D65-487C-9122-4ECAEC9CA887@00000000001</Repl:resourcetag>
				<D:getetag>&quot;{AB3A1AE1-3D65-487C-9122-4ECAEC9CA887},1&quot;</D:getetag>
				<Office:modifiedby>USERS\user</Office:modifiedby>
			</D:prop>
			<D:status>HTTP/1.1 200 OK</D:status>
		</D:propstat>
	</D:response>
	<D:response>

So, the propfind of a file doesn't return the getcontenttype property. Even if I try to do propfind with Depth = 0 using the URL of the file.

Is that normal? Is there any other way to get the Content Type using SharePoint's webdav?

Thanks!