LDAP pagination support (for very large user databases)

[richardhofman]

Expected Behavior (or desired behavior if a feature request)

LDAP user import imports the ~1700 in my organisation's Active Directory database.

Actual Behavior

Only the first 1000 users are imported (and if the import is re-run, it simply re-imports the same 1000 and flags them as "exists").

---

I've resolved this issue already on my local installation by modifying the postLDAP() function in UsersController.php, but would like to see similar functionality implemented in the official version (happy to work on this).

[snipe]

What exactly did you do to resolve the issue? We do not impose any limits on the number of users to import, and that is typically a restriction on the LDAP server itself.

[snipe]

(Also, if you want to see similar functionality, and have already addressed the issue locally, a pull request is always appreciated.)

[snipe]

Looks like later versions of PHP have a paging option, but it is only supported by LDAP v3. http://php.net/manual/en/function.ldap-control-paged-result.php

I know at least a few users are not able to use v3, so we'd have to make sure we check for those or invoke the critical flag.

[richardhofman]

The LDAP query size limit is server-side, but for LDAP versions >= 3 pagination allows you to make repeated smaller queries that enumerate much larger result sets.

I implemented a do...while structure very similar to the one in example 2 on this page: https://secure.php.net/manual/en/function.ldap-control-paged-result.php

My current solution is somewhat clunky because it iterates over all of the result pages, until it has a massive $results array with every user entry, and then uses the existing code to insert that into the database, which seems to result in very high memory usage (not sure if the array or something DB related). I had to increase the memory limit on my server to avoid exhausting available memory, but obviously something nicer would be preferable before I made a pull request - I'm happy to look into it.

[snipe]

Sure, I'd love to take a look at what you have. Making changes to the core code is always risky, since updates will blow out those changes, and we have a very large update coming soon (big enough that almost every file in the app has been touched.)