Add ldap entry creation tool

[Kyoshiro-san]

Hi,

I'm currently using your module but also https://github.com/datacentred/datacentred-ldap to have a ldap_entry method to ensure the database contains some entries.

Although, it's not perfect since it requires authentication, which implies some entries are already present. I'd have liked a ldap_entry using -Y EXTERNAL like yours do for cn=config entries.

Do you think you could adapt "openldap" resource to be able to enforce entries presence ?
Cheers

[bodgit]

Someone else created #32 that added something like this however my hesitation to merge that was then Puppet ends up slurping the whole directory into memory, which is fine for a small directory (that's what cn=config is) but would be a major performance hit with a bigger directory. Looking at the other LDAP module you linked to, it doesn't appear they support purging unmanaged entries so you can only remove entries that you explicitly define with ensure => absent which I think this new type would also have to do. The current openldap type does support purging to be able to remove unmanaged configuration so it wouldn't be suitable as-is.

I'm split as I really don't think Puppet is the right tool to manage the whole directory but I can maybe see value in being able to create/ensure some of the basic directory structure such as ou=People,dc=example,dc=com.

But for example managing user objects with a password, what happens when the user changes their password? Does Puppet then change it back again? If you want to use salted password hashes it means on every Puppet run a different password would be generated.

I think it would require a slightly different type that defines three types of attribute:

• Those that must exist with the desired values
• Those that must exist and use the provided value if they're created but ignore any other value (i.e. password hashes)
• Those that may exist but should neither be created or modified, (i.e. values that are created/updated by an overlay configured on the database)

Also, using -Y EXTERNAL still uses authentication and needs ACL's to be set up on the database and it can only create/read/update/delete attributes that the ACL's permit it access to, it's just by default that my module grants root full access via the SASL EXTERNAL method, you could in theory use the root DN over LDAP/LDAPS.

[Kyoshiro-san]

I totally agree that puppet should not manage the whole LDAP directory and thus not load it in cache. Although, as you correctly stated, I was more going for a basic structure and systems accounts enforcement.

For example I'd like to have group / users / domains / mail aliases OU defined and enforced by puppet, along with dovecot, postfix, and these other system accounts.

As for salted passwords, right now I'm using already hashed ones (by slappasswd tool). Since the purpose of such ldap_entry method would be to enforce some infrequent changing entries, using hashed passwords should not be of great inconvenience.

If it can't use -Y EXTERNAL without jeopardizing whole directory's security, maybe you could add an ACL similar to the one on cn=config database, but restricted to entries like cn=*,dc=domain,dc=tld ? Or something that the puppet admin could configure ?

[WetHippie]

Just adding my support for this request so that we can build basic structures and system accounts, as well as other infrastructure (puppet manages IoT devices for example).

[WetHippie]

I've been playing around with the openldap type that you have included. However, because it uses slapcat, it always errors after the first run when trying to set up data. Since I really would like to use this module, would you prefer that I modify the existing openldap type, or create a parallel ldap_entry to mirror the datacentred style entry? I'd keep it really simple like your existing type (thus only usable on the server machine) but change the internal logic to use ldapsearch/add/modify/delete.

[bodgit]

I think a separate type is probably best as then you can also tweak the autorequires for the new type to depend on all of the server configuration is done otherwise they may get applied and depend on overlays, etc. that haven't yet been added.

I'm curious why slapcat errors though?

[WetHippie]

The main problem that I'm running into is that slapcat returns index-qualified dn's and when we're looking to update or modify something, it doesn't pick up that the entry already exists, and then fails with the ldapmodify with a "duplicate entry" or similar error. This applies for one-off configuration and more dynamic options. I'll use the SSH key as an example:

I have a hiera definition of alll my "standard" LDAP structures:

ldap:
  config:
    "cn=openssh-openldap,cn=schema,cn=config":
      attributes:
        cn: "openssh-openldap"
        objectClass: olcSchemaConfig
        olcAttributeTypes: "( 1.3.6.1.4.1.24552.500.1.1.1.13 NAME 'sshPublicKey' DESC 'MANDATORY: OpenSSH Public key' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )"
        olcObjectClasses: "( 1.3.6.1.4.1.24552.500.1.1.2.0 NAME 'ldapPublicKey' SUP top AUXILIARY DESC 'MANDATORY: OpenSSH LPK objectclass' MAY ( sshPublicKey $ uid ) )"
    "cn=uidnext,cn=schema,cn=config":
      attributes:
        cn: "uidnext"
        objectClass: olcSchemaConfig
        olcObjectClasses: "objectClass ( 1.3.6.1.4.1.4203.666.599 NAME 'uidNext' SUP top STRUCTURAL MUST ( cn $ uidNumber ) )"

Then I have my class definition that manages my LDAP server:

  $ldap_config_defaults = {
    ensure => present,
    purge  => true,
  }

  $config_entries = hiera_hash('ldap.config')
  create_resources('openldap', $config_entries, $ldap_config_defaults)

When this runs the first time on a clean server, everything gets populated correctly. The second time, and every time thereafter:

Notice: /Stage[main]/Ldap_server/Openldap[cn=openssh-openldap,cn=schema,cn=config]/ensure: created
Debug: dn: cn=openssh-openldap,cn=schema,cn=config
changetype: add
cn: openssh-openldap
objectClass: olcSchemaConfig
olcAttributeTypes: ( 1.3.6.1.4.1.24552.500.1.1.1.13 NAME 'sshPublicKey' DESC 'MANDATORY: OpenSSH Public key' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
olcObjectClasses: ( 1.3.6.1.4.1.24552.500.1.1.2.0 NAME 'ldapPublicKey' SUP top AUXILIARY DESC 'MANDATORY: OpenSSH LPK objectclass' MAY ( sshPublicKey $ uid ) )

Debug: Executing: '/bin/ldapmodify -Y EXTERNAL -H ldapi:/// -f /tmp/openldap.cn=openssh-openldap,cn=schema,cn=config20170320-19263-1sexkxl'
Error: /Stage[main]/Ldap_server/Openldap[cn=openssh-openldap,cn=schema,cn=config]: Could not evaluate: Execution of '/bin/ldapmodify -Y EXTERNAL -H ldapi:/// -f /tmp/openldap.cn=openssh-openldap,cn=schema,cn=config20170320-19263-1sexkxl' returned 80: SASL/EXTERNAL authentication started
SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0
ldap_add: Other (e.g., implementation specific) error (80)
	additional info: olcAttributeTypes: Duplicate attributeType: "1.3.6.1.4.1.24552.500.1.1.1.13"
adding new entry "cn=openssh-openldap,cn=schema,cn=config"

The reason that it doesn't detect the existance, is that the type $name is set to cn=openssh-openldap... but slapcat returns dn: cn={1}openssh-openldap,cn=schema,cn=config

As you can see, with the basic string hash lookup used in the openldap type, this doesn't match, and we end up with it attempting to add in new entry, followed by a failure and then the whole ldap data population blows up for even the new entries. (Also note that if I transferred this to an LDIF file, the same result happens)

On a similar vein, the problem is that it just looks at the entry in the entirety and either adds or deletes it. That's not very handy if I want to modify one of the attributes - say I'm managing my LDAP groups and I have a typo in the description string. I don't want to drop the whole thing, nor add it again, I just want to modify an existing attribute of an existing dn. The current openldap type isn't subtle enough to do that :(

eg

Error: /Stage[main]/Ldap_server/Openldap[dc=example,dc=com]: Could not evaluate: Execution of '/bin/ldapmodify -Y EXTERNAL -H ldapi:/// -f /tmp/openldap.dc=example,dc=com20170320-19263-lw0wgk' returned 68: SASL/EXTERNAL authentication started
SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0
ldap_add: Already exists (68)
adding new entry "dc=example,dc=com"```

Hopefully that makes sense and what we're trying to do in dynamically maintaining our LDAP structures with puppet

(FWIW, the SSH key config is so common in LDAP implementations, might be worth making it a standard option in the openldap::server class - I can do that for you if you'd like)

[WetHippie]

Bah... Despite using the code formatter, it trashed it all :(

[WetHippie]

I should note that I have also tried using the openldap::server::schema to do the same population, with the same results. eg

openldap::server::schema { 'openssh-openldap':
    position => 4,
    ldif     => 'puppet:///ldapserver/schema/ssh_key.ldif'
}

results in:

Notice: /Stage[main]/Ldap_server/Openldap[cn=openssh-openldap,cn=schema,cn=config]/ensure: created
Error: /Stage[main]/Ldap_server/Openldap[cn=openssh-openldap,cn=schema,cn=config]: Could not evaluate: Execution of '/bin/ldapmodify -Y EXTERNAL -H ldapi:/// -f /tmp/openldap.cn=openssh-openldap,cn=schema,cn=config20170320-11254-1dh2hw9' returned 80: SASL/EXTERNAL authentication started
SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0
ldap_add: Other (e.g., implementation specific) error (80)
   additional info: olcAttributeTypes: Duplicate attributeType: "1.3.6.1.4.1.24552.500.1.1.1.13"
adding new entry "cn=openssh-openldap,cn=schema,cn=config"

[bodgit]

If using the openldap::server::schema define also errors then that's probably because you have the wrong value for position; it exists entirely to handle that {x} index notation and the acceptance tests check that multiple Puppet runs don't fail.

The problem is OpenLDAP cares about the order of schemas and to be able to enforce the configuration and order it ends up having to manage all of the schemas. Schemas are weird, but belong in the server configuration hence why that define exists. If you try and manage both schemas and entries that have attributes from that schema, how does the type know which order to do things in? The openldap type is clever enough that it will try and manage dc=example,dc=com before ou=people,dc=example,dc=com and cn={1}foo,... before cn={2}bar,..., etc. but it can't know that it needs to do cn={x}someschema,cn=schema,cn=config before uid=myuser,ou=people,dc=example,dc=com without making some very broad and messy assumptions. Ditto for things like overlays.

On a similar vein, the problem is that it just looks at the entry in the entirety and either adds or deletes it. That's not very handy if I want to modify one of the attributes - say I'm managing my LDAP groups and I have a typo in the description string. I don't want to drop the whole thing, nor add it again, I just want to modify an existing attribute of an existing dn. The current openldap type isn't subtle enough to do that :(

Yes, it is. See all of the set theory here: https://github.com/bodgit/puppet-openldap/blob/master/lib/puppet/provider/openldap/olc.rb#L173-L231 it generates an LDIF file with all of the changes needed to modify an entry and will try and do it as efficiently as it can.

[WetHippie]

Yup, using the positional stuff. Just a simple config of the following:

class { "::openldap::server":
    suffix   => $root_dn,
    root_dn   => $admin_dn,
    root_password   => $admin_password,
    ssl_ca   => $cacert_file,
    ssl_cert => "$ldap_cert_dir/$ldap_cert_name.pem",
    ssl_key  => "$ldap_cert_dir/$ldap_cert_name.key",
    memberof => true,
    ldap_interfaces => [ $infranet_ip ],
    ldaps_interfaces => [ $infranet_ip ],
    require =>  Networks::App_cert[ $ldap_cert_name ]
  }

  openldap::server::schema { "cosine":
    position => 1,
  }
  openldap::server::schema { "inetorgperson":
    position => 2,
  }
  openldap::server::schema { "nis":
    position => 3,
  }

causes the above errors to show up on the second run of puppet - almost always on the cosine schema processing.

Notice: /Stage[main]/Ldap_server/Openldap::Server::Schema[cosine]/Openldap[cn={1}cosine,cn=schema,cn=config]/ensure: created
Error: /Stage[main]/Ldap_server/Openldap::Server::Schema[cosine]/Openldap[cn={1}cosine,cn=schema,cn=config]: Could not evaluate: Execution of '/bin/ldapmodify -Y EXTERNAL -H ldapi:/// -a -f /etc/openldap/schema/cosine.ldif' returned 80: SASL/EXTERNAL authentication started
SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0
ldap_add: Other (e.g., implementation specific) error (80)
	additional info: olcAttributeTypes: Duplicate attributeType: "0.9.2342.19200300.100.1.2"
adding new entry "cn=cosine,cn=schema,cn=config"

[bodgit]

The only thing I can think of is that you have an existing OpenLDAP installation with the schemas in a different order. What does a slapcat filtering just the distinguished names show? The acceptance tests are here which starts with a clean install and runs Puppet twice to check that it doesn't have any flapping resources.

I'm going to embark on a major update that targets Puppet 4.4.0 onwards so it will use the Puppet 4 language types and I will look and see if it's also worth adding a dedicated openldap_schema type that works similarly to the current openldap type but hides the positional complexity as I don't particularly like that you have to know the order. I'll then make the original type ignore any schema objects. At the same time I can investigate this issue too.