Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement join between entries for AD queries #273

Merged
merged 3 commits into from
May 3, 2019
Merged

Implement join between entries for AD queries #273

merged 3 commits into from
May 3, 2019

Conversation

hlecleme
Copy link
Contributor

@hlecleme hlecleme commented May 2, 2019

Allow joining LDAP entries together based on a DN attribute. The syntax is similar to the one used to give the CN of a DN attribute (e.g. {member.cn}) and allow to give entry attributes through an additional LDAP query (e.g. {member.userPrincipalName}).

The ldap configuration has been extended with one additional attribute joins to permit the user to specify the filter to be used for the query (e.g. to filter out entries not having the referenced attribute).

Thanks in advance for the review.

Add support of joining entries based on the DN of attributes to allow
expressions like `{member.userPrincipalName}` for role membership
definition when roles are defined from the `userPrincipalName`
attribute.
@codecov-io
Copy link

codecov-io commented May 2, 2019

Codecov Report

Merging #273 into master will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@          Coverage Diff          @@
##           master   #273   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files          12     12           
  Lines        1700   1758   +58     
=====================================
+ Hits         1700   1758   +58
Impacted Files Coverage Δ
ldap2pg/validators.py 100% <100%> (ø) ⬆️
ldap2pg/manager.py 100% <100%> (ø) ⬆️
ldap2pg/ldap.py 100% <100%> (ø) ⬆️
ldap2pg/utils.py 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d8380ad...aa00b99. Read the comment docs.

@bersace
Copy link
Member

bersace commented May 2, 2019

Wow, very interesting ! Did you found somethings hard to understand in the code base ? Some remark :

  • Can you write a func test or extend the func test fixtures to validate the features ?
  • Can you add a Changelog entry ?
  • Would you mind add a full example of join in documentation (for exemple in cookbook) ?

Did you know my naïve mock up of join feature : #128 (comment) ?

Thanks for this huge contribution :-)

@hlecleme
Copy link
Contributor Author

hlecleme commented May 3, 2019

Thanks for the review. The ldap2pg code base is very well structured which allowed to make this change quite small.

  • Added a func test which is a variation on the default ldap2pg.yml configuration. As the AD schema is not loaded in the LDAP directory, I used the mail attribute instead of sAMAccountName. The default fixture of the LDAP database has been modified for this.
  • Changelog added
  • Added an example of configuration in the cookbook. I'm not 100% satisfied about it as it is not something that could be tried on the LDAP database configured by the default fixture, but in my opinion it is closer to a real usage.

Yes, I've seen the mock up of the join feature when investigating how to link objects together. I took some latitude as I struggled to find a link between the configuration of the mock up and the configuration found in recent versions of ldap2pg.yml. The advantages of specifying the attribute (instead of join) is to allow multiple joins from the same entry (although I do not think of a usage scenario for that) and implicit configuration of the search base. The disadvantage is that it adds one more level in the configuration of the join.

Would you prefer the implementation to follow more closely the mock up of issue #128?

ORDER BY 1;


privileges:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can drop privileges management for this test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with that, it does not impact what needs to be tested.

@bersace
Copy link
Member

bersace commented May 3, 2019

@hlecleme That's very good. For the format, I prefer list over dict. List item are more embeddable with all information. What do you think of a using extension like this:

- ldap:
    base: 
    filter: 
    join:
      using: member
      filter: 
  role:
  - name: '{sAMAccountName}'
    member: '{member.sAMAccountName}'
  - name: '{member.sAMAccountName}'

join can be a list. And joins is accepted as an allias as usual. You did it right.

except KeyError:
raise RDNError("Unknown RDN %s" % (path[0],), raw_dn)

if path[0] in DN_COMPONENTS:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand properly, the trick is that member.dn does not triggers a subquery while member.unknownField will. Right ? I find this elegant. 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's right.

values_with_attrs.append((value, sub_attrs))
sub_attrs_cache[(attr, value)] = sub_attrs
except UserError as e:
logger.warn('Ignoring %s: %s' % (value, e))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A detail this line should be:

logger.warning('Ignoring %s: %s', value, e)

logger.warn is deprecated by logging. Formatting should be done by logging.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, sorry.

@bersace
Copy link
Member

bersace commented May 3, 2019

Note that it's just a matter of yaml formatting. using: member is just a way to pass the key in value. This limits depth of YAML but loose the implicit unicity of each join. Still, you can map this to a dict in internal representation and thus, not changing the code).

Maybe you could accept both:

- ldap:
    join:
      using: member
- ldap:
    joins:
    - using: member
- ldap:
    joins:
      member:
        filter: 
...

I tend to accept different formats in YAML. The inner representation must always be stable (in this case, a dict). The idea is that internal representation handle all cases. But user can write only what he need to define : a single join rather than a list of one join.

Actually, lighter YAML syntax can be added later. The join dict is a good starting point !

@bersace
Copy link
Member

bersace commented May 3, 2019

@hlecleme I'm ok to merge. We could improve this later (more YAML syntax, some refactoring, etc.). Can you confirm you're ready for merging ?

@hlecleme
Copy link
Contributor Author

hlecleme commented May 3, 2019

@bersace Thanks again for the review. Yes, I'm ready.

@bersace bersace merged commit 1851605 into dalibo:master May 3, 2019
@bersace
Copy link
Member

bersace commented May 3, 2019

Bravo \o/

@hlecleme hlecleme deleted the join-ldap-entries branch March 21, 2020 17:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants