jetty client cannot work with NTLM authentication

[jmcc0nn3ll]

migrated from Bugzilla #322534
status REOPENED severity enhancement in component client for 7.1.x
Reported in version unspecified on platform PC
Assigned to: Project Inbox

Original attachment names and IDs:

• mycontribution.zip (ID 176746)

On 2010-08-12 10:07:56 -0400, Shooray wrote:

Build Identifier: 2204

It is truth that jetty-client doesn't support NTLM authentication. There is only one NTLM bug report which is on https://bugs.eclipse.org/bugs/show_bug.cgi?id=289669
So I've to do it myself. However, to my disappointed, jetty-client seems to has no ability to work with NTLM authentication. I've researched for a long time. My conclusion is jetty HttpDestination use different HttpConnection to resend with a pool. So IIS web server doesn't verify the challenge of the type3message which is sent by client.

Here are my source codes:

BasicTest.java
// ========================================================================
// This is the test main program.
// ========================================================================
import org.eclipse.jetty.client.ContentExchange;
import org.eclipse.jetty.client.HttpClient;
import org.eclipse.jetty.client.security.Realm;
import org.eclipse.jetty.client.security.RealmResolver;
import org.eclipse.jetty.client.security.SimpleRealmResolver;

public class BasicTest {

public static void main(String[] args) throws Exception {
HttpClient _client = new HttpClient();
_client.setConnectorType(HttpClient.CONNECTOR_SELECT_CHANNEL);
RealmResolver resolver = new SimpleRealmResolver(new Realm() {
public String getId() {
return "realm";
}

      public String getPrincipal() {
          return "test";
      }

      public String getCredentials() {
          return "password";

      }
  });
  _client.setRealmResolver(resolver);
  _client.start();

  ContentExchange exchange = new ContentExchange();

  // if send type2message directly, it may work fine mostly.
  //exchange.setRequestHeader("Authorization", "NTLM TlRMTVNTUAABAAAAATIAAA0ADQAgAAAADAAMAC0AAABXSU4ySzMtU0VSVkVSSkNJRlMwXzE4XzMx");
  exchange.setURL("http://192.168.10.156/ntlm/index.asp");
  exchange.setMethod("GET");
  exchange.setVersion(11);

  _client.send(exchange);
  exchange.waitForDone();

  System.out.println(exchange.getResponseStatus()); // all be 401
  System.out.println(new String(exchange.getResponseContentBytes()));

}
}

org/eclipse/jetty/client/security/SecurityListener.java
// ========================================================================
// Copyright (c) 2008-2009 Mort Bay Consulting Pty. Ltd.
// ------------------------------------------------------------------------
// All rights reserved. This program and the accompanying materials
// are made available under the terms of the Eclipse Public License v1.0
// and Apache License v2.0 which accompanies this distribution.
// The Eclipse Public License is available at
// http://www.eclipse.org/legal/epl-v10.html
// The Apache License v2.0 is available at
// http://www.opensource.org/licenses/apache2.0.php
// You may elect to redistribute this code under either of these licenses.
// ========================================================================

package org.eclipse.jetty.client.security;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.StringTokenizer;

import org.eclipse.jetty.client.HttpDestination;
import org.eclipse.jetty.client.HttpEventListenerWrapper;
import org.eclipse.jetty.client.HttpExchange;
import org.eclipse.jetty.http.HttpHeaders;
import org.eclipse.jetty.http.HttpStatus;
import org.eclipse.jetty.io.Buffer;
import org.eclipse.jetty.util.StringUtil;
import org.eclipse.jetty.util.log.Log;

/**

• SecurityListener

• Allow for insertion of security dialog when performing an

• HttpExchange.
  */
  public class SecurityListener extends HttpEventListenerWrapper
  {
  private HttpDestination _destination;
  private HttpExchange _exchange;
  private boolean _requestComplete;
  private boolean _responseComplete;
  private boolean _needIntercept;

  private int _attempts = 0; // TODO remember to settle on winning solution

  public SecurityListener(HttpDestination destination, HttpExchange ex)
  {
  // Start of sending events through to the wrapped listener
  // Next decision point is the onResponseStatus
  super(ex.getEventListener(),true);
  _destination=destination;
  _exchange=ex;
  }

  /**

  • scrapes an authentication type from the authString

  • @param authString

  • @return the authentication type
    */
    protected String scrapeAuthenticationType( String authString )
    {
    String authType;

    if ( authString.indexOf( " " ) == -1 )
    {
    authType = authString.toString().trim();
    }
    else
    {
    String authResponse = authString.toString();
    authType = authResponse.substring( 0, authResponse.indexOf( " " ) ).trim();
    }
    return authType;
    }

  /**

  • scrapes a set of authentication details from the authString

  • @param authString

  • @return the authentication details
    */
    protected Map<String, String> scrapeAuthenticationDetails( String authString )
    {
    Map<String, String> authenticationDetails = new HashMap<String, String>();
    authString = authString.substring( authString.indexOf( " " ) + 1, authString.length() );
    StringTokenizer strtok = new StringTokenizer( authString, ",");

    while ( strtok.hasMoreTokens() )
    {
    String token = strtok.nextToken();
    String[] pair = token.split( "=" );

    // authentication details ought to come in two parts, if not then just skip
    if ( pair.length == 2 )
    {
        String itemName = pair[0].trim();
        String itemValue = pair[1].trim();
    
        itemValue = StringUtil.unquote( itemValue );
    
        authenticationDetails.put( itemName, itemValue );
    }    
    else
    {
        Log.debug("SecurityListener: missed scraping authentication details - " + token );
    }
    

    }
    return authenticationDetails;
    }

  @OverRide
  public void onResponseStatus( Buffer version, int status, Buffer reason )
  throws IOException
  {
  if (Log.isDebugEnabled())
  Log.debug("SecurityListener:Response Status: " + status );

  if ( status == HttpStatus.UNAUTHORIZED_401 && _attempts<_destination.getHttpClient().maxRetries()) 
  {
      // Let's absorb events until we have done some retries
      setDelegatingResponses(false);
      _needIntercept = true;
  }
  else 
  {
      setDelegatingResponses(true);
      setDelegatingRequests(true);
      _needIntercept = false;
  }
  super.onResponseStatus(version,status,reason);
  

  }

  @OverRide
  public void onResponseHeader( Buffer name, Buffer value )
  throws IOException
  {
  if (Log.isDebugEnabled())
  Log.debug( "SecurityListener:Header: " + name.toString() + " / " + value.toString() );

  if (!isDelegatingResponses())
  {
      int header = HttpHeaders.CACHE.getOrdinal(name);
      switch (header)
      {
          case HttpHeaders.WWW_AUTHENTICATE_ORDINAL:
  
              // TODO don't hard code this bit.
              String authString = value.toString();
              String type = scrapeAuthenticationType( authString );
  
              String type2Message = null;
              if("ntlm".equalsIgnoreCase(type)) {
                if ( authString.indexOf( " " ) != -1 ) {
                    // if header is "WWW-Authenticate=NTLM TlRMTVNTUAACAAA......
                    // type2Message is not null
                    type2Message = authString.substring(authString.indexOf( " " ), authString.length()).trim();
                }
              }
  
              // TODO maybe avoid this map creation
              Map<String,String> details = scrapeAuthenticationDetails( authString );
              String pathSpec="/"; // TODO work out the real path spec
              RealmResolver realmResolver = _destination.getHttpClient().getRealmResolver();
  
              if ( realmResolver == null )
              {
                  break;
              }
  
              Realm realm = realmResolver.getRealm( details.get("realm"), _destination, pathSpec ); // TODO work our realm correctly 
  
              if ( realm == null )
              {
                  Log.warn( "Unknown Security Realm: " + details.get("realm") );
              }
              else if ("digest".equalsIgnoreCase(type))
              {
                  _destination.addAuthorization("/",new DigestAuthentication(realm,details));
  
              }
              else if ("basic".equalsIgnoreCase(type))
              {
                  _destination.addAuthorization(pathSpec,new BasicAuthentication(realm));
              }
              else if ("ntlm".equalsIgnoreCase(type))
              {
                _destination.addAuthorization("/",new NTLMAuthentication(realm, type2Message));
              }
  
              break;
      }
  }
  super.onResponseHeader(name,value);
  

  }

  @OverRide
  public void onRequestComplete() throws IOException
  {
  _requestComplete = true;

  if (_needIntercept)
  {
      if (_requestComplete && _responseComplete)
      {
         if (Log.isDebugEnabled())
             Log.debug("onRequestComplete, Both complete: Resending from onResponseComplete "+_exchange); 
          _responseComplete = false;
          _requestComplete = false;
          setDelegatingRequests(true);
          setDelegatingResponses(true);
          _destination.resend(_exchange);  
      } 
      else
      {
          if (Log.isDebugEnabled())
              Log.debug("onRequestComplete, Response not yet complete onRequestComplete, calling super for "+_exchange);
          super.onRequestComplete(); 
      }
  }
  else
  {
      if (Log.isDebugEnabled())
          Log.debug("onRequestComplete, delegating to super with Request complete="+_requestComplete+", response complete="+_responseComplete+" "+_exchange);
      super.onRequestComplete();
  }
  

  }

  @OverRide
  public void onResponseComplete() throws IOException
  {
  _responseComplete = true;
  if (_needIntercept)
  {
  if (_requestComplete && _responseComplete)
  {
  if (Log.isDebugEnabled())
  Log.debug("onResponseComplete, Both complete: Resending from onResponseComplete"+_exchange);
  _responseComplete = false;
  _requestComplete = false;
  setDelegatingResponses(true);
  setDelegatingRequests(true);
  _destination.resend(_exchange);

      }
      else
      {
         if (Log.isDebugEnabled())
             Log.debug("onResponseComplete, Request not yet complete from onResponseComplete,  calling super "+_exchange);
          super.onResponseComplete(); 
      }
  }
  else
  {
      if (Log.isDebugEnabled())
          Log.debug("OnResponseComplete, delegating to super with Request complete="+_requestComplete+", response complete="+_responseComplete+" "+_exchange);
      super.onResponseComplete();  
  }
  

  }

  @OverRide
  public void onRetry()
  {
  _attempts++;
  setDelegatingRequests(true);
  setDelegatingResponses(true);
  _requestComplete=false;
  _responseComplete=false;
  _needIntercept=false;
  super.onRetry();
  }

}

org/eclipse/jetty/client/security/NTLMAuthentication.java
// ========================================================================
// Author: Lion Shooray
// shooray@gmail.com
// ========================================================================
package org.eclipse.jetty.client.security;

import java.io.IOException;

import jcifs.ntlmssp.Type1Message;
import jcifs.ntlmssp.Type2Message;
import jcifs.ntlmssp.Type3Message;

import org.eclipse.jetty.client.HttpExchange;
import org.eclipse.jetty.http.HttpHeaders;
import org.eclipse.jetty.http.security.B64Code;
import org.eclipse.jetty.io.Buffer;
import org.eclipse.jetty.io.ByteArrayBuffer;
import org.eclipse.jetty.util.log.Log;

public class NTLMAuthentication implements Authentication {
private Buffer _authorization;

public NTLMAuthentication(Realm realm, String strType2Message) throws IOException
{
  String domain = "WIN2K3-SERVER";
  String authenticationString = null;
  if(strType2Message == null) { //type1
      authenticationString = "NTLM " + new String(B64Code.encode( buildType1Message(Type1Message.getDefaultFlags(),domain, null)));
  } else {//type2 -> type3
      byte[] byte2Msg = B64Code.decode(strType2Message);
      Type2Message type2Message = new Type2Message(byte2Msg);
      byte[] byte3Msg = buildType3Message(type2Message, realm.getCredentials(), domain, realm.getPrincipal(), null, Type3Message.getDefaultFlags());
      authenticationString = "NTLM " + new String(B64Code.encode(byte3Msg));
  }
    if (Log.isDebugEnabled())
        Log.debug("NTLMAuthentication(" + (strType2Message == null?"type1":"type3") + "): " + authenticationString );
    _authorization= new ByteArrayBuffer(authenticationString);
}

public void setCredentials(HttpExchange exchange) throws IOException {
exchange.setRequestHeader( HttpHeaders.AUTHORIZATION_BUFFER, _authorization);
}

protected byte[] buildType1Message(int flag, String domain, String workstation) {
Type1Message type1 = new Type1Message(flag, domain, workstation);
return type1.toByteArray();
}

protected byte[] buildType3Message(Type2Message type2, String password,
String domain, String user, String workstation, int flags) {
Type3Message type3 = new Type3Message(type2, password, domain, user,
workstation, flags);
return type3.toByteArray();
}
}

Reproducible: Always

Steps to Reproduce:

1. replace SecurityListener.java with my edition
   org/eclipse/jetty/client/security/SecurityListener.java
2. add new java file:
   org/eclipse/jetty/client/security/NTLMAuthentication.java
3. add jcifs-1.3.14.jar to classpath, you can get it from
   http://jcifs.samba.org/src/
4. run BaseTest.java to test

On 2010-08-16 07:57:24 -0400, Michael Gorovoy wrote:

Greetings,

NTLM authentication is connection-based, but Jetty HTTP Client is not aware of that, so this is likely the reason you are having problems.

It is possible however to implement NTLM authentication in Jetty HTTP Client by detecting it in a custom listener and delegating the transport to the java.net.UrlConnection that supports NTLM authentication in Java 6. However, you are going to loose Jetty HTTP Client's asynchronous request handling as a result.

Thanks,
Michael

*** This bug has been marked as a duplicate of bug 289669 ***

On 2010-08-16 23:16:06 -0400, Shooray wrote:

Created attachment 176746
This is a suggested implementation with jetty-client NTLM Authentication.

This is a suggested implementation with jetty-client NTLM Authentication.

NTLMAuthenticationTest.java is the test main program with JUnit4.

Steps to Reproduce:

1. prepare a NTLM authentication needed web server, in my case, it is http://192.168.10.156/ntlm/index.asp, you should modify it.
2. replace the org.eclipse.....xxx.java files in jetty-client project
3. add jcifs-1.3.14.jar and JUnit4 jars to classpath, you can get jcifs from
   http://jcifs.samba.org/src/
4. compile and run. it will work properly

I wish jetty become better and better. Thanks for your efforts.

Lion Sooray / shooray@gmail.com / 2010-8-17

On 2010-08-16 23:32:07 -0400, Shooray wrote:

Hi Michael:
You said right, I've also found detail about NTLM:
http://davenport.sourceforge.net/ntlm.html#ntlmHttpAuthentication
"3. ......From this point forward, the connection is kept open; closing the connection requires reauthentication of subsequent requests. This implies that the server and client must support persistent connections..."
And it was proved by sniffer tools.

However, it's a pity to hear you don't intend to implement NTLM Authentication in this jetty release. You said "loose Jetty HTTP Client's asynchronous request handling as a

result", I think that is not the truth. I've submited a suggested implementation with jetty-client NTLM Authentication for jetty project. Of cource, I'll use it in a production environment.

By the way, I suggest that you should put Realm(or RealmResolver) into exchange, not in HttpClient. So proxy can handle different request with different Principal/Credentials.

I wish jetty become better and better. Thanks for your efforts.

On 2010-08-16 23:48:04 -0400, Jesse McConnell wrote:

sadly jcifs is lgpl and as such there is no way we could take this patch...even if we were at apache I doubt we could take the patch since LGPL is basically off limits in most scenarios

http://jcifs.samba.org/

pity its not a friendly license, if it had been we would have used it a long time ago

the best bet is still banking on what support the jvm gives which is intermittent and dependent on the jvm and host operating system...we have had jetty client working with ntlm on linux, mac and windos jvm's using the mechanism that michael mentioned

cheers
jesse

On 2010-08-16 23:48:30 -0400, Jesse McConnell wrote:

oh, but I applaud the effort! :)

On 2010-08-17 00:35:17 -0400, Shooray wrote:

Hi jesse:
Thanks for your explanation about license and praise. I'm glad to have ability to contribute my efforts for you.
I don't think jcifs is the problem. Because I only use jcifs to make or parse type1,2,3 message. Moreover, NTLM protocol is open, anybody can implement it. I can rewrite it, or maybe Eclipse Communication Framework Project has finished it.

On 2010-08-23 23:28:29 -0400, Greg Wilkins wrote:

I've reopened this, so the effort does not get wasted.
We can monitor the situation and if a license suitable jcifs replacement becomes available then we can move.

We could also consider doing something in the codehaus release.

On 2010-08-23 23:31:39 -0400, Greg Wilkins wrote:

*** Bug 289669 has been marked as a duplicate of this bug. ***

On 2010-08-24 02:48:09 -0400, Michael Gorovoy wrote:

There are several other NTLM authentication libraries, e.g. Waffle (http://waffle.codeplex.com/), http://www.luigidragone.com/networking/ntlm.html, etc. However all of them are LGPL licensed.

On 2011-09-28 12:56:28 -0400, Jesse McConnell wrote:

nothing has really changed on this, its an enhancement and currently hands are tied regarding this due to IP issues...should that situation change I'll take it and resolve it once and for all

On 2011-10-13 12:46:11 -0400, Shooray wrote:

I used jetty to implement a reverse proxy last year. More bugs had been found and be resolved temporarily by myself, such as url encoding, etc. For work with NTLM, I have to use jetty work in bio not nio. however, in bio, jetty worker thread will auto increment until the thread pool be exhausted. I have to write a shell script to monitor the proxy, if jetty proxy cannot work, restart it automatically. It's a headache.
However, I am not responsible for that product now. If jetty team need my help, I'm still very happy to contribute something.
Good luck jetty!

On 2012-05-04 18:23:20 -0400, Carey Evans wrote:

It looks like Waffle is now using the Eclipse Public License:

https://github.com/dblock/waffle/blob/master/LICENSE

Does this make it useable for client authentication?

On 2012-05-04 21:29:04 -0400, Jesse McConnell wrote:

score! that is good news

first step is to get a CQ opened to use the library and then to get it into orbit

I'll try and get that process started next week

nice catch!

On 2012-07-17 10:09:52 -0400, Jesse McConnell wrote:

update, until waffle is published in maven central I don't think we'll bother with getting the CQ through for this...it had moved to github and is EPL which is nice

On 2013-05-13 02:23:29 -0400, Michael Hawkshaw wrote:

Looks like Waffle is now available in Maven Central: https://oss.sonatype.org/content/repositories/releases/com/github/dblock/waffle/

[sbordet]

Since fixing #408 there is a mechanism to use the connection a request was sent on, so it may be possible to implement NTLM (since it's the only connection-oriented authentication protocol).

[sbordet]

The patch is outdated and not license friendly, and likely the code has changed enough to require an extensive review.

[joakime]

Closing as the jcifs license (GPL) isn't Eclipse friendly, so this is unlikely to be implemented by Eclipse Jetty.
This would be a good external project / library that can easily plug into the existing jetty-client facilities as a custom authentication protocol.