MacOS Monterey authentication failure.

[isabsent]

MacOS Monterey 12.1 for com.hierynomus:smbj versions 0.11.1 and 0.11.3

Error in the method:

public Session establish(AuthenticationContext authContext) {
    try {
        Authenticator authenticator = getAuthenticator(authContext);
        BuilderContext ctx = newContext(authContext, authenticator);

        authenticator.init(config);
        processAuthenticationToken(ctx, connectionContext.getGssNegotiateToken());

        Session session = setupSession(ctx);
        logger.info("Successfully authenticated {} on {}, session is {}", authContext.getUsername(), connection.getRemoteHostname(), session.getSessionId());
        sessionTable.registerSession(session.getSessionId(), session);
        return session;
    } catch (SpnegoException | IOException e) {
        throw new SMBRuntimeException(e);
    }
}

on the string Session session = setupSession(ctx)

    com.hierynomus.mssmb2.SMBApiException: STATUS_LOGON_FAILURE (0xc000006d): Authentication failed for 'lev' using com.hierynomus.smbj.auth.NtlmAuthenticator@5702f0b
    at com.hierynomus.smbj.connection.SMBSessionBuilder.setupSession(SMBSessionBuilder.java:144)
    at com.hierynomus.smbj.connection.SMBSessionBuilder.setupSession(SMBSessionBuilder.java:142)
    at com.hierynomus.smbj.connection.SMBSessionBuilder.establish(SMBSessionBuilder.java:109)
    at com.hierynomus.smbj.connection.Connection.authenticate(Connection.java:202)



ConnectionContext{
  serverGuid=c77ee07e-ce60-0751-b571-0be29c955039,
  serverName='192.168.10.3',
  negotiatedProtocol=NegotiatedProtocol{dialect=SMB_3_0_2, maxTransactSize=4194304, maxReadSize=4194304, maxWriteSize=4194304},
  clientGuid=d2f28808-7cd9-4ecc-94d9-2070b6f2062c,
  clientCapabilities=[SMB2_GLOBAL_CAP_LARGE_MTU],
  serverCapabilities=[SMB2_GLOBAL_CAP_LEASING, SMB2_GLOBAL_CAP_LARGE_MTU, SMB2_GLOBAL_CAP_ENCRYPTION],
  clientSecurityMode=1,
  serverSecurityMode=3,
  server='com.hierynomus.smbj.server.Server@8e988bf'
}

authContext = {
     domain = ""
     password = {char[7]@16689} [l, e, v, 1, 9, 6, 7]
     username = "lev"
}

authenticator = {
   completed = false
   initialized = true
   random = {SecureRandom@16659} 
   securityProvider = {BCSecurityProvider@16660} 
   workStationName = null
}

ctx = {
   authContext = {AuthenticationContext@16573} "AuthenticationContext[lev@]"
   authenticator = {NtlmAuthenticator@16641} 
   digest = null
   request = null
   response = null
   securityContext = {byte[66]@16662} [96, 64, 6, 6, 43, 6, 1, 5, 5, 2, -96, 54, 48, 52, -96, 14, 48, 12, 6, 10, 43, 6, 1, 4, 1, -126, 55, 2, 2, 10, -94, 34, 4, 32, 78, 84, 76, 77, 83, 83, 80, 0, 1, 0, 0, 0, 21, -126, -120, -32, 0, 0, 0, 0, 32, 0, 0, 0, 0, 0, 0, 0, 32, 0, 0, 0]
   sessionId = 0
   sessionKey = null
}

config = {SmbConfig@16572} 
   authenticators = {ArrayList@16665}  size = 1
    clientGSSContextConfig = {GSSContextConfig@16666} 
    clientGuid = {UUID@16614} "d2f28808-7cd9-4ecc-94d9-2070b6f2062c"
    dfsEnabled = false
    dialects = {RegularEnumSet@16667}  size = 5
    encryptData = false
    random = {SecureRandom@16659} 
    readBufferSize = 1048576
    readTimeout = 60000
    securityProvider = {BCSecurityProvider@16660} 
    signingRequired = false
    soTimeout = 0
    socketFactory = {ProxySocketFactory@16668} 
    transactBufferSize = 1048576
    transactTimeout = 60000
    transportLayerFactory = {DirectTcpTransportFactory@16669} 
    useMultiProtocolNegotiate = false
    workStationName = null
    writeBufferSize = 1048576
    writeTimeout = 60000
 }

As I have found the same error was faced on Big Sur year ago #454 (comment), but the provided solution doesn't work now.

The same code works fine for shared folders on Ubuntu 20.04.

P.S.: com.hierynomus:smbj:0.10.0 doesn't have this problem on MacOS Monterey 12.1! I have found the same situation in this message #478 (comment).

[chrisgch]

A user of my free Android app has reported the same problem when accessing a share on MacOS. For him it fails when connecting to MacOS 10.15.7.

Until recently, I was using a much older version of smbj, a snapshot from 28.8.2017. With this version, connecting to MacOS does work! It used config = SmbConfig.builder().withMultiProtocolNegotiate(true).build(); client = new SMBClient(config);

Then I changed to a current snapshot recently for the SMB3 support, and with this version the connection fails with STATUS_LOGON_FAILURE (0xc000006d). The user tried SMB2 only (builder.withDialects(SMB2Dialect.SMB_2_1, SMB2Dialect.SMB_2_0_2)) and also SMB3 (builder.withDialects(SMB2Dialect.SMB_3_1_1, SMB2Dialect.SMB_3_0_2, SMB2Dialect.SMB_3_0)) and a combination of both, it always failed with the same error.

Maybe a code comparison could give you a clue why it works with the older smbj version but not the latest.

[hierynomus]

Between 2017 and now, quite a number of things have changed. Is there any way of obtaining a pcap file of a working connect and a non-working to analyze the difference?

[isabsent]

Between 2017 and now, quite a number of things have changed. Is there any way of obtaining a pcap file of a working connect and a non-working to analyze the difference?

I will try, but I have no idea what is pcap and how to get it.

[hierynomus]

pcap is a packet capture. There are multiple ways to grab one, depending on your OS.

[isabsent]

pcap is a packet capture. There are multiple ways to grab one, depending on your OS.

Ok, I will try to get it.

[chrisgch]

isabsent: Why did you close this issue? Did you find a solution? The user of my app who has this problem would also appreciate a solution.

[isabsent]

isabsent: Why did you close this issue? Did you find a solution? The user of my app who has this problem would also appreciate a solution.

I have closed it accidentally. I have no the solution yet.

[chrisgch]

I have now made a code comparison between the smbj version from 2017 which works, and the current version which doesn't, and found the following differences in the authentication functions which may have an influence. Unfortunately I don't have access to a Mac so it would be nice if you (isabsent) or anyone else with this problem could test whether one of these changes causes the problem.

1. mssmb2\messages\SMB2SessionSetup.java
   new version:
   buffer.putUInt32(clientCapabilities & 0xFF); // Capabilities (4 bytes)
   old version:
   buffer.putUInt32(clientCapabilities & 0x01); // Capabilities (4 bytes) (only last byte can be set)

2. ntlm\messages\NtlmNegotiate.java
   new:
   buffer.putUInt32(0x20); // DomainNameBufferOffset (4 bytes)
   // WorkstationFields (8 bytes)
   buffer.putUInt16(0x0); // WorkstationLen (2 bytes)
   buffer.putUInt16(0x0); // WorkstationMaxLen (2 bytes)
   buffer.putUInt32(0x20); // WorkstationBufferOffset (4 bytes)
   old:
   buffer.putUInt32(0x0); // DomainNameBufferOffset (4 bytes)
   // WorkstationFields (8 bytes)
   buffer.putUInt16(0x0); // WorkstationLen (2 bytes)
   buffer.putUInt16(0x0); // WorkstationMaxLen (2 bytes)
   buffer.putUInt32(0x0); // WorkstationBufferOffset (4 bytes)

3. smbj\auth\NtlmAuthenticator.java
   new:
   Object msAvTimestamp = serverNtlmChallenge.getTargetInfo().getAvPairObject(AvId.MsvAvTimestamp);
   if (msAvTimestamp != null) {
   old:
   Object msvAvFlags = challenge.getAvPairObject(AvId.MsvAvFlags);
   if (msvAvFlags instanceof Long && ((long) msvAvFlags & 0x00000002) > 0) {
   you can't use this old code directly because the object names changed. You will need to use something like this:
   Object msvAvFlags = serverNtlmChallenge.getTargetInfo().getAvPairObject(AvId.MsvAvFlags);
   if (msvAvFlags instanceof Long && ((long) msvAvFlags & 0x00000002) > 0) {

4. a lot was rearranged in smbj\connection\Connection.java, so if the problem is there, it will be difficult to analyze.

Also there are a lot of protocol negotiations if SMB2Dialect.SMB_3_1_1 is set, so try connecting with SMB2 only:
builder.withDialects(SMB2Dialect.SMB_2_1, SMB2Dialect.SMB_2_0_2)

[isabsent]

I will try to check it in this weekend.

[chrisgch]

isabsent: Have you tried my 3 suggested changes in the meantime? They are so small, you can just edit the 3 files by hand and recompile. Unfortunately I don't have access to a Mac, so I can't test it myself. In the meantime, more MacOS users have reported the login failure.

[marosseleng]

@chrisgch I applied first 3 changes you proposed (points 1, 2 and 3) and it started working. The changes, however caused test NegTokenInitSpec."should correctly encode ntlm choice negInitToken" to fail. Afterwards, I tried applying changes one by one with results (in all cases the negotiated protocol dialect is SMB_3_0_2):

1. "1." only:
   a. NOT working;
2. "1." and "2.":
   a. NOT working;
3. "1." and "2." and "3.":
   a. WORKING;
4. "1." and "3.":
   a. WORKING;
5. "3." only:
   a. WORKING;

TL;DR version would be, that it seems that changes in NtlmAuthenticator seem to make it working again on MacOS 12.3.1.

After applying only change "3" (in NtlmAuthenticator), the failing test is now working allowing for build.

[chrisgch]

Thanks a lot for your tests! So apparently there is no msAvTimestamp field in the authentication with the Mac, but the flags field is there.
@hierynomus
Any idea of what this could be? Was there a specific reason for switching to the timestamp check? The comment above the changed check still reads:
// MIC (16 bytes) provided if in AvPairType is key MsvAvFlags with value & 0x00000002 is true

[chrisgch]

@isabsent: Are you still using MacOS 12.3.1? A user with MacOS 12.4 has reported that the bugfix doesn't work for him.

[marosseleng]

@chrisgch I tried running it on my machine (MacOS 12.4) and the fix seems to be working fine (I haven't tried the original library, though).

[L-JINBIN]

3. smbj\auth\NtlmAuthenticator.java
new:
Object msAvTimestamp = serverNtlmChallenge.getTargetInfo().getAvPairObject(AvId.MsvAvTimestamp);
if (msAvTimestamp != null) {
old:
Object msvAvFlags = challenge.getAvPairObject(AvId.MsvAvFlags);
if (msvAvFlags instanceof Long && ((long) msvAvFlags & 0x00000002) > 0) {
you can't use this old code directly because the object names changed. You will need to use something like this:
Object msvAvFlags = serverNtlmChallenge.getTargetInfo().getAvPairObject(AvId.MsvAvFlags);
if (msvAvFlags instanceof Long && ((long) msvAvFlags & 0x00000002) > 0) {

It work for me. macOS Ventura 13.1

[hierynomus]

Does the original code work, or the updated for 13.1?

[L-JINBIN]

Does the original code work, or the updated for 13.1?

The original code doesn't work.

Object msAvTimestamp = serverNtlmChallenge.getTargetInfo().getAvPairObject(AvId.MsvAvTimestamp);
(msAvTimestamp != null) is true

Object msvAvFlags = serverNtlmChallenge.getTargetInfo().getAvPairObject(AvId.MsvAvFlags);
(msvAvFlags instanceof Long && ((long) msvAvFlags & 0x00000002) > 0) is false

When the code takes the false branch, it works fine.

[dkocher]

Looks like this is a duplicate of #730 with a proposed fix in #731 and changeset c81367c.