LDAP: Domain Users are not being loaded with User Filter

[sbelov1]

Environment:
OC v7.0.4 on apache
CentOS 6.5
php 5.4.32
MySQL
AD LDAP
Description: Customer is attempting to select just a Domain Users group from the User Filter, however only 2 (as oppose to 200) are shown. As a workaround we have to either use a different group or allow all the groups to login with OC.

Here's the log excerpt:

{"app":"PHP","message":"ldap_search(): Partial search results returned: Sizelimit exceeded at \/var\/www\/html\/owncloud\/apps\/user_ldap\/lib\/ldap.php#252","level":3,"time":"2015-01-20T17:24:20+00:00"}
{"app":"PHP","message":"ldap_search(): Partial search results returned: Sizelimit exceeded at \/var\/www\/html\/owncloud\/apps\/user_ldap\/lib\/ldap.php#252","level":3,"time":"2015-01-20T17:24:20+00:00"}
{"app":"PHP","message":"ldap_search(): Partial search results returned: Sizelimit exceeded at \/var\/www\/html\/owncloud\/apps\/user_ldap\/lib\/ldap.php#252","level":3,"time":"2015-01-20T17:24:21+00:00"}
{"app":"PHP","message":"ldap_search(): Partial search results returned: Sizelimit exceeded at \/var\/www\/html\/owncloud\/apps\/user_ldap\/lib\/ldap.php#252","level":3,"time":"2015-01-20T17:24:31+00:00"}
{"app":"PHP","message":"ldap_search(): Partial search results returned: Sizelimit exceeded at \/var\/www\/html\/owncloud\/apps\/user_ldap\/lib\/ldap.php#252","level":3,"time":"2015-01-20T17:24:42+00:00"}
{"app":"PHP","message":"ldap_search(): Partial search results returned: Sizelimit exceeded at \/var\/www\/html\/owncloud\/apps\/user_ldap\/lib\/ldap.php#252","level":3,"time":"2015-01-20T17:24:59+00:00"}
{"app":"PHP","message":"ldap_search(): Partial search results returned: Sizelimit exceeded at \/var\/www\/html\/owncloud\/apps\/user_ldap\/lib\/ldap.php#252","level":3,"time":"2015-01-20T17:25:03+00:00"}
{"app":"PHP","message":"ldap_search(): Partial search results returned: Sizelimit exceeded at \/var\/www\/html\/owncloud\/apps\/user_ldap\/lib\/ldap.php#252","level":3,"time":"2015-01-20T17:25:04+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/apps\/user_ldap\/group_ldap.php#477","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/lib\/private\/user\/manager.php#100","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/lib\/private\/user\/database.php#209","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Statement.php#138","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/lib\/private\/user\/database.php#250","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/apps\/user_ldap\/user_ldap.php#136","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/apps\/user_ldap\/lib\/user\/manager.php#140","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/apps\/user_ldap\/lib\/user\/manager.php#142","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"ldap_explode_dn() expects parameter 1 to be string, array given at \/var\/www\/html\/owncloud\/apps\/user_ldap\/lib\/ldap.php#252","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Statement.php#138","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Statement.php#138","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/apps\/user_ldap\/user_ldap.php#142","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/apps\/user_ldap\/user_ldap.php#144","level":3,"time":"2015-01-20T17:31:15+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/lib\/private\/user\/manager.php#100","level":3,"time":"2015-01-20T17:31:17+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/lib\/private\/user\/database.php#209","level":3,"time":"2015-01-20T17:31:17+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Statement.php#138","level":3,"time":"2015-01-20T17:31:17+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/lib\/private\/user\/database.php#250","level":3,"time":"2015-01-20T17:31:17+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/apps\/user_ldap\/user_ldap.php#136","level":3,"time":"2015-01-20T17:31:17+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/apps\/user_ldap\/user_ldap.php#137","level":3,"time":"2015-01-20T17:31:17+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/lib\/private\/user\/manager.php#100","level":3,"time":"2015-01-20T17:31:21+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/lib\/private\/user\/database.php#209","level":3,"time":"2015-01-20T17:31:21+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Statement.php#138","level":3,"time":"2015-01-20T17:31:21+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/lib\/private\/user\/database.php#250","level":3,"time":"2015-01-20T17:31:21+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/apps\/user_ldap\/user_ldap.php#136","level":3,"time":"2015-01-20T17:31:21+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/apps\/user_ldap\/user_ldap.php#137","level":3,"time":"2015-01-20T17:31:21+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/lib\/private\/user\/manager.php#100","level":3,"time":"2015-01-20T17:31:22+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/lib\/private\/user\/database.php#209","level":3,"time":"2015-01-20T17:31:22+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Statement.php#138","level":3,"time":"2015-01-20T17:31:22+00:00"}
{"app":"PHP","message":"Illegal offset type in isset or empty at \/var\/www\/html\/owncloud\/lib\/private\/user\/database.php#250","level":3,"time":"2015-01-20T17:31:22+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/apps\/user_ldap\/user_ldap.php#136","level":3,"time":"2015-01-20T17:31:22+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/apps\/user_ldap\/user_ldap.php#137","level":3,"time":"2015-01-20T17:31:22+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/apps\/user_ldap\/group_ldap.php#561","level":3,"time":"2015-01-20T17:32:11+00:00"}
{"app":"PHP","message":"Array to string conversion at \/var\/www\/html\/owncloud\/apps\/user_ldap\/group_ldap.php#561","level":3,"time":"2015-01-20T17:32:12+00:00"}

@gig13

[karlitschek]

@blizzz

[sbelov1]

Updating the post with an exact php version:

PHP 5.4.34 (cli) (built: Oct 16 2014 10:19:38)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2014 Zend Technologies
with Zend Guard Loader v3.3, Copyright (c) 1998-2013, by Zend Technologies

[blizzz]

@sbelov1 what are the reproduction steps? What is the LDAP config?

Hint: https://raw.githubusercontent.com/owncloud/core/master/issue_template.md

[sbelov1]

@blizzz I believe its the same user we were working with here: #12118

Reproduction steps are as follows:

1. Select Domain Users group only from the Group Filter
2. Select Domain Users group from the User Filter
   Result: 2 users are shown
   Expected Result: ~200 users

[sbelov1]

@blizzz oh yes, and we have requested for LDAP config. I'll update once we get it back from customer.

[blizzz]

ok, in the meantime, the missing users have "Domain Users" assigned as their primary group?

[sbelov1]

@blizzz I believe so, yes. Looks like something we've already resolved, didn't we?

[blizzz]

i am not aware, just that rework of primary group handling is still on my list (but for other reasons). And unfortunately everything works with my test AD.

[sbelov1]

@blizzz would it be easier jumping on a quick call with customer? They've been having this issue for awhile now. It could be something specific to their environment as I'm not having this type of issue with anyone else.

[blizzz]

Not really, at least not that alone. I would provide a patch to gather some debug output instead.

[sbelov1]

@blizzz sounds good. Should I expect the patch now or do we have to wait for the LDAP config output from customer first?

[blizzz]

@sbelov1 I'd like to LDAP have a look on the config first.

[sbelov1]

@blizzz please check your email for the output.

[blizzz]

@sbelov1: now i remember… the issue here is that the user filter restricts login to a "standard" group membership, but does not take primary groups into account. We sought to fix this with #12233

[blizzz]

I follow up there, stay tuned.

[blizzz]

Oh, and for a manual workaround: currently there is no group limitation in user or login filter. To make this work with primary groups do as follow:

Change User Filter from

(&(|(objectclass=organizationalPerson)(objectclass=person)(objectclass=top)(objectclass=user)))

To

(&(|(objectclass=organizationalPerson)(objectclass=person)(objectclass=top)(objectclass=user))(|(memberof=CN=Domain Users,…)(primaryGroupID=513)))

and Login filter from

(&(&(|(objectclass=organizationalPerson)(objectclass=person)(objectclass=top)(objectclass=user)))(|(sAMAccountName=%uid))) 

To

(&((&(|(objectclass=organizationalPerson)(objectclass=person)(objectclass=top)(objectclass=user))(|(memberof=CN=Domain Users,…)(primaryGroupID=513))))(sAMAccountName=%uid))

In both cases you must insert the whole DN of the domain users group in the memberof= pair.

The primaryGroupID should be 513 for Domain Users. Otherwise you can get details of a user's domain group using this script: https://gist.github.com/blizzz/101e17cdc1d399031b50#file-primgrtest-php

Example usage and output:

$ sudo -u www-data ./primGrTest.php "cn=tara gilbert,ou=vampires,dc=madder,dc=owncloud,dc=bzoc" s01
User Primary Group ID is: 513
The DN of the domain is: dc=madder,dc=owncloud,dc=bzoc
The SID of the domain is: S-1-5-21-249921958-728525901-1594176202
The object SID should be: S-1-5-21-249921958-728525901-1594176202-513
VERY GOOD: The group DN is: cn=domain users,cn=users,dc=madder,dc=owncloud,dc=bzoc

It should not be necessary in this case though.

[blizzz]

OK, i think i have a fix here: #13740
(mind: it is against master)

[sbelov1]

@blizzz sounds good. Please let me know the instructions for applying it once its all set.

[blizzz]

@sbelov1 here is the corresponding PR for stable7: #13742

You can get a diff here https://github.com/owncloud/core/pull/13742.diff, save it and apply it using patch against 7.0.4 (worked for me):

$ patch -p1 < 13742.diff

[sbelov1]

@blizzz this patch wouldn't affect any changes to the database or anything, right? Just a php code? Just wanted to make sure its safe to run as customer is on a production environment.

[blizzz]

@sbelov1 exactly. If you took it from before, please save it again, i did a fix there. About code quality in general: yet it has neither not been tested by someone else and from review side got only few comments on coding style and inline documentation.

[gig13]

@sbelov1 @blizzz
This should not be tested in a production environment. Please test internally first with the lab. Also provide a back out process.

[sbelov1]

@blizzz I'll let Allen take it from here. Please let me know if there's I can do..

[blizzz]

@gig13 Well, testers are necessery. Maybe a test setup can be created there. I tried it with our lab, but I need two more people who give their go in #13742

[gig13]

@craigpg @bboule @blizzz
I tested oc705 and saw the same issue as the customer -- Domain Users were not being found correctly in the Wizard or in the Users page.
After the latest patch was applied, the Domain Users are correctly found (47 in our test instance) and the Users page displays them correctly when only filtering on that group. I tried with IE and Firefox and both browsers displayed the Users page fine.

I will coordinate a time to install with the customer.

[sbelov1]

@blizzz I have another prospect experiencing same/similar issue. A user selects Domain Users group -> only 2 users are shown under User Filter (should be much more) -> 2 users are shown under Users Page -> Only those two users can login. Does it sound like the same issue? If it does, can we please update the patch to 8.0.2 because that's the version we're using. Unless we ported it into 8.0.3 in which case, I'll just update their build.

[sbelov1]

@blizzz ?

[bboule]

My understanding is that a fix given to @gig13 solved this issue for a customer running 7.0.5 can we confirm the same fix will apply in this case @sbelov1 I think you referred to 8.0.3. Have we applied to an 8.0.3 instance internally to test?

[jnfrmarks]

@bboule

I can get you an 8.0.3 for testing 13533 if that helps

[MorrisJobke]

My understanding is that a fix given to @gig13 solved this issue for a customer running 7.0.5 can we confirm the same fix will apply in this case @sbelov1 I think you referred to 8.0.3. Have we applied to an 8.0.3 instance internally to test?

Can we stop apply random patches to production instances. If the patch is verified to fix an issue, we should get it into the next patch release and provide that for a production environment.

[karlitschek]

I agree with @MorrisJobke We have to handle such problems this more coordinated and professionally

[gig13]

@bboule @MorrisJobke @sbelov1 @karlitschek
The patch for OC7.0.5 has been successfully applied with the customer environment and is closed from my side.
The customer had several failed patches attempted (and reverted) over a 4 month period and this was critical for them to properly administer and thus made it very difficult to open up to additional users.

[sbelov1]

@bboule We've tested it against 8.0.2 and it did not work. That is why I wanted to confirm with @blizzz on whether this was ported into 8.0.3 before upgrading them.

[craigpg]

@blizzz, has this patch made it into a release yet?

[blizzz]

If you referring to #13533 (comment), this still waits for a test result from @gig13.

If you refer to the patch mentioned in #13533 (comment) it probably refers to #13742. Which is a backport of #13740 which made it to 8.0.2. However, the patch in #13533 (comment) would need to be applied to OC 8 as well. I will do a PR out of it.

[gig13]

@blizzz
The patch worked for OC7.0.5 internally -- and was applied at the customer as stated previously.

I just tested OC8.0.3 (no patching) and it does not work. @sbelov1 is testing against OC8, I was testing against OC7.

[sbelov1]

Which is a backport of #13740 which made it to 8.0.2

This is odd. The prospect having this issue was running 8.0.2. However, I've just applied this patch in my internal lab and it did fix the issue. Are we sure it made it to 8.0.2?

From the notes on #13740 it looks like its in 8.1 milestone. Please clarify.

[blizzz]

This is odd. The prospect having this issue was running 8.0.2. However, I've just applied this patch in my internal lab and it did fix the issue. Are we sure it made it to 8.0.2?

From the notes on #13740 it looks like its in 8.1 milestone. Please clarify.

Indeed, it was not backported to stable8.

[blizzz]

Patch from #13533 (comment) is added to #15606 (eventually needs to be backported to OC 8, too).

For OC 7 it is added to #13742 which incorporates the backport of #15606 already, since the issues are connected and it was open for so long.

[MorrisJobke]

Stable8 backport is in #16456

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.