LDAP Lambda does not support active directory #822

bradleykite · 2020-08-19T22:45:46Z

Hi,

We are trying to authenticate directly to Active Directory with an LDAP lambda.

Please see here for reference:

https://fusionauth.io/community/forum/topic/256/ldap-lambda

To reproduce, under Customizations => Lambdas, create an "LDAP connector reconcile"

// Using the response from an LDAP connector, reconcile the User.
function reconcile(user, userAttributes) {
  
  user.email = userAttributes.userPrincipalName;
  user.firstName = userAttributes.givenName;
  user.lastName  = userAttributes.sn;
  user.active    = true;
  
  var testGUID = "\\374\\240\\270\\317\\260\\260\\213\\104\\206\\233\\357\\330\\240\\225\\130\\207";
  
  console.debug("Test Binary GUID: " + guidToString(testGUID));
  // The above should decode to "3437335c-325c-3034-5c32-37305c333137"

  console.debug("Actual Binary GUID: " + guidToString(userAttributes.objectGUID));

   console.debug("USER: " + JSON.stringify(user));
   console.debug("ATTR: " + JSON.stringify(userAttributes));
}

function guidToString(x)
{
    var ret = "";
  
    // bytes.push((charCode & 0xFF00) >> 8);
    // bytes.push(charCode & 0xFF);
    for (i = 3; i >= 0; i--)
    {
        ret += ('00'+x.charCodeAt(i).toString(16)).substr(-2,2);
    }
    ret += "-";
    for (i = 5; i >= 4; i--)
    {
        //ret = ret + ('00' + (charCode & 0xFF00) >> 8);
        ret += ('00'+x.charCodeAt(i).toString(16)).substr(-2,2);
    }
    ret += "-";
    for (i = 7; i >= 6; i--)
    {
        //ret = ret + ('00' + (charCode & 0xFF00) >> 8);
        ret += ('00'+x.charCodeAt(i).toString(16)).substr(-2,2);
    }
    ret += "-";
    for (i = 8; i <= 9; i++)
    {
        //ret = ret + ('00' + (charCode & 0xFF00) >> 8);
        ret += ('00'+x.charCodeAt(i).toString(16)).substr(-2,2);
    }
    ret += "-";
    for (i = 10; i < 16; i++)
    {
        //ret = ret + ('00' + (charCode & 0xFF00) >> 8);
        ret += ('00'+x.charCodeAt(i).toString(16)).substr(-2,2);
    }
  
    return ret;
}

Then, under Settings => Connectors, create an LDAP connector which pulls out the following AD attributes, and assign the reconcile lambda created above.

In my case, my test user has an objectGUID in AD of "3437335c-325c-3034-5c32-37305c333137" - or in binary: "\374\240\270\317\260\260\213\104\206\233\357\330\240\225\130\207"

Decoding this binary string within the LAMDA works correctly, so the decode function can be proven to work, however decoding the actual object GUID which comes from AD looks like it has already been interpreted as a UTF-16 string, and since its actually binary, it replaces invalid UTF-16 characters with the replacement character 65533

This makes it impossible to reconcile AD users while maintaining their original GUID.

The actual output I get from the LAMBDA is:

Lambda invocation result.

Id: 79e93d02-aeb6-47a4-9e41-1478ec79a4e5
Name: Active Directory

Test Binary GUID: 3437335c-325c-3034-5c32-37305c333137
Actual Binary GUID: f0fdfdfd-fdfd-fd44-fdfd-20fd58fdaNaN
USER: {"active":true,"data":{},"memberships":[],"passwordChangeRequired":false,"preferredLanguages":[],"registrations":[],"twoFactorEnabled":false,"verified":false,"email":"bradley.kite@cybanetix.com","firstName":"Bradley","lastName":"Kite"}
ATTR: {"cn":"Bradley Kite","givenName":"Bradley","objectGUID":"���ϰ��D���ؠ�X�","sn":"Kite","userPrincipalName":"bradley.kite@cybanetix.com"}

The text was updated successfully, but these errors were encountered:

robotdan · 2020-08-21T23:55:15Z

That is a good clue. In order to marshal objects in and out of the lambda engine we essentially serialize an object to JSON using Jackson, and then use JSON.parse to build an actual JavaScript object for use in the Lambda.

We then do the reverse on the way out for any object that can be mutated by the lambda using JSON.stringify to get JSON back in order to deserialize the object.

It looks like JSON.stringify used to suffer from a lone surrogate issue, that was fixed by the "well formed JSON.stringify specification".

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#Well-formed_JSON.stringify

tc39/ecma262#944

However, in theory Nashorn supports ECMAScript-262, and claims 100% support of ECMAScript 5.1.

But perhaps a clue, perhaps we can take values you provided of 3437335c-325c-3034-5c32-37305c333137 and \374\240\270\317\260\260\213\104\206\233\357\330\240\225\130\207 and recreate.

robotdan · 2020-08-22T00:56:02Z

Using this code example for the Java side of things ( https://gist.github.com/davidmc24/0588900f3200eba3ea80 ) I get 3d0ef076-0404-68d5-cee9-654af0e182cf from those bytes instead of 437335c-325c-3034-5c32-37305c333137.

Can you confirm the actual Guid from Microsoft AD? And does the object Guid actuall show up in the userAttributes as "\\374\\240\\270\\317\\260\\260\\213\\104\\206\\233\\357\\330\\240\\225\\130\\207"?

bradleykite · 2020-08-24T08:46:46Z

Apologies, I was trying something new with the LAMBDA code so it wasn't converting correctly.

An actual objectGUID in binary format is as follows:

Octal:
374 240 270 317 260 260 213 104 206 233 357 330 240 225 130 207

Hex:
FC A0 B8 CF B0 B0 8B 44 86 9B EF D8 A0 95 58 87

UUID (Same as hex, but corrected byte order).
cfb8a0fc-b0b0-448b-869b-efd8a0955887

Is it possible to simply encode any binary data into a hex string before it gets serialised into JSON for the javascript engine to process? That way there wont be any UTF-16 encoding issues, and the Javascript LAMBDA can just flip the various bytes around to produce a valid UUID string. This solution will allow any kind of binary data to be processed, not just UUID's.

Regards

Brad.

robotdan · 2020-08-24T15:39:36Z

Do you know how the Guid is represented in the SAML response? Assuming a string form?

bradleykite · 2020-08-24T15:44:05Z

This is from an LDAP lambda.

We want to use FusionAuth as a SAML IDP for our applications, with LDAP being the source of the users.

In other words, it doesnt get as far as issuing the SAML response to our applications.

--
Brad.

robotdan · 2020-08-24T15:59:19Z

Correct, but the Guid is coming from AD, and is passed into the lambda using the userAttributes argument. So when the GUiD comes in on the userAttributes.objectGUID parameter - what form is objectGUID in before you try to convert it? 8-4-4-4-12 or bytes?

bradleykite · 2020-08-24T16:05:15Z

Have a look at the end of this post:

#822 (comment)

Its a messed up (due to "invalid" UTF-16) string:

Lambda invocation result.

Id: 79e93d02-aeb6-47a4-9e41-1478ec79a4e5
Name: Active Directory

Test Binary GUID: 3437335c-325c-3034-5c32-37305c333137
Actual Binary GUID: f0fdfdfd-fdfd-fd44-fdfd-20fd58fdaNaN
USER: {"active":true,"data":{},"memberships":[],"passwordChangeRequired":false,"preferredLanguages":[],"registrations":[],"twoFactorEnabled":false,"verified":false,"email":"bradley.kite@cybanetix.com","firstName":"Bradley","lastName":"Kite"}
ATTR: {"cn":"Bradley Kite","givenName":"Bradley","objectGUID":"���ϰ��D���ؠ�X�","sn":"Kite","userPrincipalName":"bradley.kite@cybanetix.com"}

robotdan · 2020-08-25T14:09:29Z

That is after it has gone through the JSON.stringify - do you have an example of how the value is sent in the XML response? In other words, what form does it arrive in from AD? If it is in a SAML request it has to be in some text representation, encoded, escaped, or something like that.

bradleykite · 2020-08-25T14:17:25Z

It comes directly from an LDAP connector - not from a SAML request.

Under "Settings" => "Connectors" we have created an LDAP connector, and then we are using this connector in the Tenant (under "Tenants" => "Connectors").

robotdan · 2020-08-25T14:35:34Z

Ha, yes, thank you. I have SAML on the brain.

If you enable debug for the connector, there will be a Debug Event Log with the output of all of the attributes returned from LDAP. Can you post that so we can see what the objectGUID looks like there before we run it through the Lambda?

If the Lambda is mangling it somehow, this will give us a before value.

bradleykite · 2020-08-25T16:26:51Z

Its the same:

LDAP Connector Debug Log for [Cybanetix AD] with Id [0d554587-8feb-40c3-85b9-311dd93c3485].

8/25/2020 05:23:25 PM BST Attempting authentication request to application with Id [3c219e58-ed0e-4b18-ad48-f4f92793ae32] from IP address [XXXXXX] for [bradley.kite@cybanetix.com] against the URL [XXXXXX].
8/25/2020 05:23:25 PM BST Attempting bind against DN [XXXXXX].
8/25/2020 05:23:26 PM BST Bind against DN [XXXXXX] was successful.
8/25/2020 05:23:26 PM BST Attempting search with filter [(userPrincipalName=bradley.kite@cybanetix.com)].
8/25/2020 05:23:26 PM BST Attempting bind against DN [CN=Bradley Kite,XXXXXX].
8/25/2020 05:23:26 PM BST Bind against DN [CN=Bradley Kite,XXXXXXX] was successful.
8/25/2020 05:23:26 PM BST Invoking lambda with Id [79e93d02-aeb6-47a4-9e41-1478ec79a4e5] with attributes returned from connector:
{
  "cn" : "Bradley Kite",
  "givenName" : "Bradley",
  "objectGUID" : "���ϰ��D���ؠ�X�",
  "sn" : "Kite",
  "userPrincipalName" : "bradley.kite@cybanetix.com"
}
8/25/2020 05:23:26 PM BST Resolved Connector User:
{
  "active" : true,
  "data" : { },
  "email" : "bradley.kite@cybanetix.com",
  "firstName" : "Bradley",
  "lastName" : "Kite",
  "memberships" : [ ],
  "passwordChangeRequired" : false,
  "preferredLanguages" : [ ],
  "registrations" : [ ],
  "twoFactorEnabled" : false,
  "verified" : false
}
8/25/2020 05:23:26 PM BST 
. WARNING DISCARDING USER because it was missing a unique id in the [user.id] property.

bradleykite · 2020-08-25T16:36:29Z

This might be of use:

https://help.mulesoft.com/s/article/Ldap-ObjectGUID-field-retrieval

bradleykite · 2020-08-25T17:41:35Z

OK I've made some progress.

Based on the table here:

https://docs.oracle.com/javase/jndi/tutorial/ldap/misc/attrs.html#BYTES

I have figured out that I can retrieve the attribute by appending a ";binary" on the end, so the LDAP connector now looks like this:

This then creates a base-64-encoded version of the attribute which looks like this:

USER: {"active":true,"data":{},"memberships":[],"passwordChangeRequired":false,"preferredLanguages":[],"registrations":[],"twoFactorEnabled":false,"verified":false,"email":"bradley.kite@cybanetix.com","firstName":"Bradley","lastName":"Kite"}
ATTR: {"cn":"Bradley Kite","givenName":"Bradley","objectGUID;binary":"/KC4z7Cwi0SGm+/YoJVYhw==","sn":"Kite","userPrincipalName":"bradley.kite@cybanetix.com"}

I can then base-64 decode the "objectGUID;binary" field to get the actual binary version, and then convert that into hex (with the corrections for byte-order).

This gives me the following output:

Lambda invocation result.

Id: 79e93d02-aeb6-47a4-9e41-1478ec79a4e5
Name: Active Directory

Binary String: 18 length: ü ¸Ï°°�D��ïØ �X�ÿ¿
Decoding CHAR: ü - 252
Decoding CHAR:   - 160
Decoding CHAR: ¸ - 184
Decoding CHAR: Ï - 207
Decoding CHAR: ° - 176
Decoding CHAR: ° - 176
Decoding CHAR: � - 139
Decoding CHAR: D - 68
Decoding CHAR: � - 134
Decoding CHAR: � - 155
Decoding CHAR: ï - 239
Decoding CHAR: Ø - 216
Decoding CHAR:   - 160
Decoding CHAR: � - 149
Decoding CHAR: X - 88
Decoding CHAR: � - 135
Decoding CHAR: ÿ - 255
Decoding CHAR: ¿ - 191
Actual Binary GUID: cfb8a0fc-b0b0-448b-869b-efd8a0955887
USER: {"active":true,"data":{},"memberships":[],"passwordChangeRequired":false,"preferredLanguages":[],"registrations":[],"twoFactorEnabled":false,"verified":false,"email":"bradley.kite@cybanetix.com","firstName":"Bradley","lastName":"Kite"}
ATTR: {"cn":"Bradley Kite","givenName":"Bradley","objectGUID;binary":"/KC4z7Cwi0SGm+/YoJVYhw==","sn":"Kite","userPrincipalName":"bradley.kite@cybanetix.com"}

bradleykite · 2020-08-25T17:49:00Z

For reference, this LAMBDA now works as expected, creating FusionAuth users based on the user within Active Directory.

Its messy, and requires many layers of decoding, but at least I now have a working solution.

Hopefully this can help you get to a better, more permanent solution - ideally decoding functions should not be required within the LAMBDA itself.

// Using the response from an LDAP connector, reconcile the User.
function reconcile(user, userAttributes) {
  

  user.email = userAttributes.userPrincipalName;
  user.firstName = userAttributes.givenName;
  user.lastName  = userAttributes.sn;
  user.active    = true;
  user.id = guidToString(userAttributes['objectGUID;binary']);
  
  console.debug("Actual Binary GUID: " + user.id);
  console.debug("USER: " + JSON.stringify(user));
  console.debug("ATTR: " + JSON.stringify(userAttributes));


}

function decodeBase64(string)
{
	var b=0,l=0, r='',
  m='ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/';
  string.split('').forEach(function (v) {
    b=(b<<6)+m.indexOf(v); l+=6;
    if (l>=8) r+=String.fromCharCode((b>>>(l-=8))&0xff);
  });
  return r;
}

function guidToString(b64)
{
    var x = decodeBase64(b64);
  
    console.debug("Binary String: " + x.length + " length: " + x);
  
/*    for (i = 0; i < x.length; i++)
    {
      console.debug("Decoding CHAR: " + x.charAt(i) + " - " + x.charCodeAt(i));
    }
*/  
    var ret = "";
  
    for (i = 3; i >= 0; i--)
    {
        ret += ('00'+x.charCodeAt(i).toString(16)).substr(-2,2);
    }
    ret += "-";
    for (i = 5; i >= 4; i--)
    {
        //ret = ret + ('00' + (charCode & 0xFF00) >> 8);
        ret += ('00'+x.charCodeAt(i).toString(16)).substr(-2,2);
    }
    ret += "-";
    for (i = 7; i >= 6; i--)
    {
        //ret = ret + ('00' + (charCode & 0xFF00) >> 8);
        ret += ('00'+x.charCodeAt(i).toString(16)).substr(-2,2);
    }
    ret += "-";
    for (i = 8; i <= 9; i++)
    {
        //ret = ret + ('00' + (charCode & 0xFF00) >> 8);
        ret += ('00'+x.charCodeAt(i).toString(16)).substr(-2,2);
    }
    ret += "-";
    for (i = 10; i < 16; i++)
    {
        //ret = ret + ('00' + (charCode & 0xFF00) >> 8);
        ret += ('00'+x.charCodeAt(i).toString(16)).substr(-2,2);
    }
  
    return ret;
}

robotdan · 2020-08-25T18:11:47Z

Thanks for the update @bradleykite this is helpful. I suppose it would be possible for us to decode on our end, but we won't know which attribute or what format the attribute is in either I don't think.

Maybe the best way to handle this would be just provide some utility methods available in the lambda to decode or convert common Id types to a UUID.

For example, we could expose guidToString(b64) to the Lambda and then move all of that code back into FusionAuth.

robotdan · 2020-09-22T22:35:52Z

@mooreds @bradleykite

Anyone have an opinion on the syntax for FusionAuth utils exposed in the Lambda? We could namespace them or just dump a bunch of things in the context.

Examples:

// Namespace under 'FusionAuth'
var uuid1 = FusionAuth.guidToString(encodedGuid);

// No name space
var uuid2 = guidToString(encodedGuid);

Or something else?

bradleykite · 2020-09-23T11:53:03Z

If it matters, I guess doing it under the FusionAuth namespace seems better.

Would it be possible to create custom plugins which can extend the functionality of LAMBDA's? Each plugin can then have its own namespace.

mooreds · 2020-09-23T14:55:11Z

That would be a cool feature, @bradleykite , but might be deferred until this issue is resolved: #571

Doesn't make sense to invest a bunch of time in Nashorn unless the work is definitely going to be portable to other runtimes.

But would love if you'd add a feature request fleshing out the lambda plugin functionality more.

robotdan · 2020-09-23T22:22:10Z

FYI, I finally got to the bottom of this issue.

TL;DR

@bradleykite is right, by default this doesn't work with AD.

Longer version

The Sun LDAP provider can only represent attributes as strings, or byte arrays. By default it will use a string.

The objectGUID is a non-string value, and this value will be retrieved in string form instead of a byte array. By the time this attribute is returned and we get a hold of it, it is already too late and we have lost data in the conversion to the string. So you have to indicate the attribute is a non-string value, or binary.

From what I can tell, the only way to make this work is to use @bradleykite 's work around which is to request the attribute in our configuration by appending the suffix ;binary, or for us to know ahead of time which fields we are requesting that are non-string values. These two options are functionally equivalent but it probably makes more sense for us to handle this if possible.

See more here: https://docs.oracle.com/javase/jndi/tutorial/ldap/misc/attrs.html

The way for FusionAuth to handle this is to register non-string values in the bind environment. For example:

env.put("java.naming.ldap.attributes.binary", "objectGUID objectSid");

The above environment value will indicate we know that objectGUID and objectSid are non-string values and they should be returned as a byte array instead of a string.

If we think there is a good possibility that these two values do not represent the exhaustive list of Ids that could be returned by LDAP that are non-string values, we should probably make this configurable. Or, we could just cover the two cases, and document that ;binary can be added to any non-string attribute so that it is retrieved in a readable format.

I haven't quite nailed down how the value in the lambda is a Base64 encoded string, perhaps Nashorn is doing that on our behalf or something.

Re: Utility Helpers

Planning to ship this utility method, heavily name spaced so it is clear this is a Microsoft conversion utility. Example usage:

user.id = FusionAuth.ActiveDirectory.b64GuidToString(userAttributes.objectGUID)

robotdan · 2020-09-24T16:24:44Z

Delivered in 1.19.7. For now just adding FusionAuth.ActiveDirectory.b64GuidToString and documenting the usage of ;binary. https://site-local.fusionauth.io/docs/v1/tech/lambdas/ldap-connector-reconcile

mooreds added the bug Something isn't working label Aug 20, 2020

robotdan added this to Backlog in FusionAuth Issues via automation Sep 22, 2020

robotdan added this to the 1.19.7 milestone Sep 22, 2020

robotdan moved this from Backlog to In progress in FusionAuth Issues Sep 22, 2020

robotdan mentioned this issue Sep 22, 2020

When using LDAP connector to connect to active directory, see encoding exception when can't connect #879

Closed

robotdan moved this from In progress to Code complete in FusionAuth Issues Sep 23, 2020

robotdan moved this from Code complete to Done in FusionAuth Issues Sep 24, 2020

robotdan closed this as completed Sep 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LDAP Lambda does not support active directory #822

LDAP Lambda does not support active directory #822

bradleykite commented Aug 19, 2020 •

edited

Loading

robotdan commented Aug 21, 2020

robotdan commented Aug 22, 2020

bradleykite commented Aug 24, 2020

robotdan commented Aug 24, 2020

bradleykite commented Aug 24, 2020 •

edited

Loading

robotdan commented Aug 24, 2020

bradleykite commented Aug 24, 2020

robotdan commented Aug 25, 2020

bradleykite commented Aug 25, 2020

robotdan commented Aug 25, 2020

bradleykite commented Aug 25, 2020

bradleykite commented Aug 25, 2020

bradleykite commented Aug 25, 2020

bradleykite commented Aug 25, 2020

robotdan commented Aug 25, 2020

robotdan commented Sep 22, 2020 •

edited

Loading

bradleykite commented Sep 23, 2020

mooreds commented Sep 23, 2020

robotdan commented Sep 23, 2020 •

edited

Loading

robotdan commented Sep 24, 2020

LDAP Lambda does not support active directory #822

LDAP Lambda does not support active directory #822

Comments

bradleykite commented Aug 19, 2020 • edited Loading

robotdan commented Aug 21, 2020

robotdan commented Aug 22, 2020

bradleykite commented Aug 24, 2020

Regards

robotdan commented Aug 24, 2020

bradleykite commented Aug 24, 2020 • edited Loading

robotdan commented Aug 24, 2020

bradleykite commented Aug 24, 2020

robotdan commented Aug 25, 2020

bradleykite commented Aug 25, 2020

robotdan commented Aug 25, 2020

bradleykite commented Aug 25, 2020

bradleykite commented Aug 25, 2020

bradleykite commented Aug 25, 2020

bradleykite commented Aug 25, 2020

robotdan commented Aug 25, 2020

robotdan commented Sep 22, 2020 • edited Loading

bradleykite commented Sep 23, 2020

mooreds commented Sep 23, 2020

robotdan commented Sep 23, 2020 • edited Loading

robotdan commented Sep 24, 2020

bradleykite commented Aug 19, 2020 •

edited

Loading

bradleykite commented Aug 24, 2020 •

edited

Loading

robotdan commented Sep 22, 2020 •

edited

Loading

robotdan commented Sep 23, 2020 •

edited

Loading