Binary attributes are cut on \x00

[maethor]

Hi,

I have a problem when I set a binary attribute with a value containing "\x00" (Null Character).

>>> user.jpegPhoto = Avatar.generate(8, user.mail)
>>> user.jpegPhoto
b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x00\x00\x01\x00\x01\x00\x00\xff\xdb\x00C\x00\x08\x06\x06\x07\x06\x05\x08\x07\x07\x07\t\t\x08\n\x0c\x14\r\x0c\x0b\x0b\x0c\x19\x12\x13\x0f\x14\x1d\x1a\x1f\x1e\x1d\x1a\x1c\x1c $.\' ",#\x1c\x1c(7),01444\x1f\'9=82<.342\xff\xdb\x00C\x01\t\t\t\x0c\x0b\x0c\x18\r\r\x182!\x1c!22222222222222222222222222222222222222222222222222\xff\xc0\x00\x11\x08\x00\x08\x00\x08\x03\x01"\x00\x02\x11\x01\x03\x11\x01\xff\xc4\x00\x15\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x06\xff\xc4\x00!\x10\x00\x01\x02\x05\x05\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x02\x03\x04\x11\x121\x05\x13!BQa\xff\xc4\x00\x15\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x06\x07\xff\xc4\x00\x17\x11\x00\x03\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x021\xff\xda\x00\x0c\x03\x01\x00\x02\x11\x03\x11\x00?\x00\xa5\x898\x1f\xa7B\x95\xdb \xb1\xd5\xbe\xf3C\x9e\xbe\xf3\x9f\x81\x11\x14\xd2\xad\xd6\x85\x1boO\xff\xd9'
>>> user.save()
>>> user.fetch()
>>> user.jpegPhoto
b'\xff\xd8\xff\xe0'

In connection.py, line 391, after doing strval = ffi.new("char[]", value):

>>> strval[4]
b'\x00'

This must cause a problem in CFFI, because \x00 means "end of string" in C.

What do you think ? Can I escape theses bytes ?

Thanks.

[leonhandreke]

Because we don't really use binary attributes internally, it hadn't been implemented yet. The struct berval type (which is basically a char * with a length field) has to be used for both saving and retrieving attributes. Today I finally sat down and did it, the result is in the branch feature/binary-values.

Please try it out and let me know if it works for you. If it does, I'll merge it and do a release sometime soon.

[maethor]

(following #30)

Hum, I don't know what happen. Sorry about that.

So, fetching an entry works fine, I get the right value in entry.jpegPhoto.

When I set a new value and save the entry. I get an error:

>>> entry = LDAPEntry(ldap, "cn=guillaume,dc=example,dc=fr")
>>> entry.jpegPhoto =  Avatar.generate(8, "guillaume@example.fr")
>>> entry.save()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/maethor/.envs/test2/lib/python3.3/site-packages/ldapom/entry.py", line 138, in save
    return self._connection.save(self, *args, **kwargs)
  File "/home/maethor/.envs/test2/lib/python3.3/site-packages/ldapom/connection.py", line 52, in func_wrapper
    return func(lc, *args, **kwargs)
  File "/home/maethor/.envs/test2/lib/python3.3/site-packages/ldapom/connection.py", line 434, in save
    handle_ldap_error(err)
  File "/home/maethor/.envs/test2/lib/python3.3/site-packages/ldapom/connection.py", line 44, in handle_ldap_error
    raise error.LDAPError(error_string)
ldapom.error.LDAPError: Invalid syntax

I have try to fetch the entry before setting the attribute, same error. But if I set something else, it will work:

>>> entry = LDAPEntry(ldap, "cn=guillaume,ou=users,dc=example,dc=fr")
>>> entry.sn = "Foo"
>>> entry.jpegPhoto = Avatar.generate(8, "guillaume@example.fr")
>>> entry.save()
>>> entry.fetch()
>>> entry.jpegPhoto
{b'\xff\xd8\xff\xe0\x00n………xaf\xbc\xb3\xf0"">\xc86im?\xff\xd9'}

[leonhandreke]

Hm, I don't really have a good idea to fix this. test_set_binary_attribute in tests.py basically does exactly this.

Would it be possible for you to get some logs and figure out what exactly the LDAP server is complaining about? Debug level 256 (-d on the command line) should give something along the lines of 5370a393 conn=1163 op=5 RESULT tag=103 err=17 text=..., with text being a detailed explanation of the error.

[maethor]

Ok, I understand. In wireshark, I see the error message : unique_modify() got null op.orm_modlist. That is because I try to set the same photo that the one it is already on the server… my mistake. If I generate a different picture in my test, no problem.

So thank you, this works perfectly. I really look forward to see this branch merged in master and published on pypi.

[leonhandreke]

Good to hear that it's working for you. The error when saving without modifications doesn't sound familiar to me. Just to clarify, do you also get an error when you do this?

entry = LDAPEntry(ldap, "cn=guillaume,ou=users,dc=example,dc=fr")
entry.save()

[maethor]

In fact, there is 2 different errors, when I doesn't set any attribute, and when I set an attribute with the same value it already have. The first is raised by LDAPom, so it is ok. The second is quite annoying because the error is not very clear.

>>> entry = LDAPEntry(ldap, "cn=guillaume,ou=users,dc=example,dc=fr")
>>> entry.save()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/maethor/.envs/test2/lib/python3.3/site-packages/ldapom/entry.py", line 138, in save
    return self._connection.save(self, *args, **kwargs)
  File "/home/maethor/.envs/test2/lib/python3.3/site-packages/ldapom/connection.py", line 52, in func_wrapper
    return func(lc, *args, **kwargs)
  File "/home/maethor/.envs/test2/lib/python3.3/site-packages/ldapom/connection.py", line 376, in save
    raise error.LDAPomError("Cannot save without attributes "
ldapom.error.LDAPomError: Cannot save without attributes previously fetched or set.
>>> entry.sn
{'Subiran'}
>>> entry.sn = "Subiran"
>>> entry.sn
{'Subiran'}
>>> entry.save()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/maethor/.envs/test2/lib/python3.3/site-packages/ldapom/entry.py", line 138, in save
    return self._connection.save(self, *args, **kwargs)
  File "/home/maethor/.envs/test2/lib/python3.3/site-packages/ldapom/connection.py", line 52, in func_wrapper
    return func(lc, *args, **kwargs)
  File "/home/maethor/.envs/test2/lib/python3.3/site-packages/ldapom/connection.py", line 434, in save
    handle_ldap_error(err)
  File "/home/maethor/.envs/test2/lib/python3.3/site-packages/ldapom/connection.py", line 44, in handle_ldap_error
    raise error.LDAPError(error_string)
ldapom.error.LDAPError: Invalid syntax
>>> entry.sn = "Subiron"
>>> entry.save()

[leonhandreke]

Ha, the second error is interesting, because the following order of operations works perfectly for me:

entry = LDAPEntry(ldap, "cn=guillaume,ou=users,dc=example,dc=fr")
entry.fetch()
entry.save()

Hopefully, we can trace this down. I've opened #33 to deal with this.

[leonhandreke]

I've merged this feature into master. Thanks for bringing this up and for your help testing this!

[maethor]

Thank you very much.