PHP LDAP Paging
UPDATE : The information on this page applies to Moodle 1.9 and PHP 5.3 only! As of PHP 5.4, LDAP paging is now part of the official distribution (see PHP bug #42060), and Moodle 2.4 uses paging by default when available (see MDL-36119).
Although it’s not a requirement, it’s better when users are synchronised into Moodle. When student accounts exist in the system, the cegep module will automatically add the student account as Participant in the course. That way the teacher doesn’t have to wait until his students log in to Moodle to create groups, etc.
In a common cegep setting, student login information exists in a LDAP database. Therefore we need to use auth/ldap/auth_ldap_sync_users.php to create the student accounts in Moodle. However, while there are often many thousand of student accounts to synchronise PHP and Moodle don’t currently support LDAP paging, or LDAP paged results,to allow the server to transfer very large result sets to a client. Without LDAP paged results, only a subset of users are created in Moodle.
The first step is to patch the PHP LDAP module to support extended operations.
Begin by downloading the source package and its dependencies with apt-src install php5
This page offers a patch suited to the php5 5.2.6 package in Debian lenny : http://www.eteo.mondragon.edu/descargas/php-ldap/
This page offers a patch suited to the php5 5.3.3 package in Debian squeeze : PHP LDAP paging - Debian squeeze PHP 5.3.x patch
Download the patch into “debian/patches” inside the php5 source directory and rename it to 0041-ldap_paging.patch. Then edit the file “debian/patches/series” and add the filename just after 004-ldap_fix.patch.
Now go inside the source directory and launch dpkg-buildpackage. This will compile PHP and create the php5 binary packages. If you’re in a hurry, you can skip the rather lengthy tests with export DEB_BUILD_OPTIONS=nocheck.
After the operation is completed, install the new php5-ldap package. If you want to prevent accidental upgrading, you can the the package on hold as follows : echo "php5-ldap hold" | dpkg --set-selections
Because of changed function names, this patch is only suitable for use with the Debian squeeze PHP patch. See the page history for an older version.
diff --git a/auth/ldap/auth.php b/auth/ldap/auth.php
index 15ec0bc..291f85f 100644
--- a/auth/ldap/auth.php
+++ b/auth/ldap/auth.php
@@ -613,6 +613,17 @@ class auth_plugin_ldap extends auth_plugin_base {
array_push($contexts, $this->config->create_context);
}
+ //verify if PHP supports LDAP paged results
+ if (function_exists('ldap_control_paged_results') && function_exists('ldap_control_paged_results_response')) {
+ $ldap_pagedresults = true;
+ $ldap_pagesize = 100;
+ $ldap_cookie = '';
+ print "Using LDAP paged results with page size of $ldap_pagesize\n";
+ } else {
+ $ldap_pagedresults = false;
+ print "LDAP paged results not supported in PHP\n";
+ }
+
$fresult = array();
foreach ($contexts as $context) {
$context = trim($context);
@@ -620,39 +631,56 @@ class auth_plugin_ldap extends auth_plugin_base {
continue;
}
begin_sql();
- if ($this->config->search_sub) {
- //use ldap_search to find first user from subtree
- $ldap_result = ldap_search($ldapconnection, $context,
- $filter,
- array($this->config->user_attribute));
- } else {
- //search only in this context
- $ldap_result = ldap_list($ldapconnection, $context,
- $filter,
- array($this->config->user_attribute));
- }
- if ($entry = ldap_first_entry($ldapconnection, $ldap_result)) {
- do {
- $value = ldap_get_values_len($ldapconnection, $entry, $this->config->user_attribute);
- $value = $textlib->convert($value[0], $this->config->ldapencoding, 'utf-8');
- // usernames are __always__ lowercase.
- array_push($fresult, moodle_strtolower($value));
- if (count($fresult) >= $bulk_insert_records) {
- $this->ldap_bulk_insert($fresult, $temptable);
- $fresult = array();
- }
- } while ($entry = ldap_next_entry($ldapconnection, $entry));
- }
- unset($ldap_result); // free mem
+ do {
+
+ if ($ldap_pagedresults) {
+ ldap_control_paged_results($ldapconnection, $ldap_pagesize, true, &$ldap_cookie);
+ }
+
+ if ($this->config->search_sub) {
+ //use ldap_search to find first user from subtree
+ $ldap_result = ldap_search($ldapconnection, $context,
+ $filter,
+ array($this->config->user_attribute));
+ } else {
+ //search only in this context
+ $ldap_result = ldap_list($ldapconnection, $context,
+ $filter,
+ array($this->config->user_attribute));
+ }
- // insert any remaining users and release mem
- if (count($fresult)) {
- $this->ldap_bulk_insert($fresult, $temptable);
- $fresult = array();
- }
+ if ($ldap_pagedresults)
+ ldap_control_paged_results_response($ldapconnection, $ldap_result, &$ldap_cookie);
+
+ if ($entry = ldap_first_entry($ldapconnection, $ldap_result)) {
+ do {
+ $value = ldap_get_values_len($ldapconnection, $entry, $this->config->user_attribute);
+ $value = $textlib->convert($value[0], $this->config->ldapencoding, 'utf-8');
+ // usernames are __always__ lowercase.
+ array_push($fresult, moodle_strtolower($value));
+ if (count($fresult) >= $bulk_insert_records) {
+ $this->ldap_bulk_insert($fresult, $temptable);
+ $fresult = array();
+ }
+ } while ($entry = ldap_next_entry($ldapconnection, $entry));
+ }
+ unset($ldap_result); // free mem
+
+ // insert any remaining users and release mem
+ if (count($fresult)) {
+ $this->ldap_bulk_insert($fresult, $temptable);
+ $fresult = array();
+ }
+
+ } while ($ldap_pagedresults && !empty($ldap_cookie));
+
commit_sql();
}
+
+ // if LDAP paged results was used, the current connection
+ // must be closed and a new one created
+ if ($ldap_pagedresults)
+ $this->ldap_close();
/// preserve our user database
/// if the temp table is empty, it probably means that something went wrong, exit
As soon as possible, also merge these fixes that improve auth_ldap_sync_users.php :