LDIFReader and mixed content from AD ldifde

[idsecurity]

Hello,
I performed an LDIF export from Active Directory using built-in ldifde tool on Windows 2012R2. The export included a group which has more than 1500 members and ldifde wrote the first 1500 members into an LDIF add record and the rest was written as an LDIF modify record.

When I try to read that LDIF using LDIFReader like this:

    while (true) {
                Entry entry = reader_leftLdif.readEntry();

                if (entry == null) {
                    break;
                }

              doSomething();
            }

I get this error when it comes to the modify change record:

LDIFException(lineNumber=12, mayContinueReading=true, message='The record starti
ng at or near line number 12 contains a line that does not begin with an attribu
te name followed by a colon.', dataLines='dn: CN=TestGroup,OU=MyGroups,DC=ad,DC=
acme,DC=se{end-of-line}changetype: modify{end-of-line}add: member{end-of-line}me
mber: CN=Resource Admin,OU=Corp,DC=ad,DC=acme,DC=se{end-of-line}-{end-of-line}ad
d: member{end-of-line}member: CN=Another User,OU=Corp,DC=ad,DC=acme,DC=se{end-of
-line}-{end-of-line}add: member{end-of-line}member: CN=Service User,OU=Corp,DC=a
d,DC=acme,DC=se{end-of-line}-{end-of-line}')
        at com.unboundid.ldif.LDIFReader.parseAttributes(LDIFReader.java:2744)
        at com.unboundid.ldif.LDIFReader.decodeEntry(LDIFReader.java:2070)
        at com.unboundid.ldif.LDIFReader.readEntryInternal(LDIFReader.java:1448)

        at com.unboundid.ldif.LDIFReader.readEntry(LDIFReader.java:1058)

Here is a small LDIF that reproduces the error without all the 1500 members.

version: 1

dn: CN=TestGroup,OU=MyGroups,DC=ad,DC=acme,DC=se
changetype: add
cn: TestGroup
member:: Q049QWxlZW4gQWxsYW1hbixPVT1Db3JwLERDPWFkLERDPWFjbWUsREM9c2UNCg==
member:: Q049QWxlamFuZHJhIEFsbGFuLE9VPUNvcnAsREM9YWQsREM9YWNtZSxEQz1zZQ==
member:: 
 Q049QWxlamFuZHJpbmEgQWxsYXJkLE9VPUNvcnAsREM9YWQsREM9YWNtZSxEQz1zZQ==
member:: Q049QWxlbmEgQWxsYXdheSxPVT1Db3JwLERDPWFkLERDPWFjbWUsREM9c2U=

dn: CN=TestGroup,OU=MyGroups,DC=ad,DC=acme,DC=se
changetype: modify
add: member
member: CN=Resource Admin,OU=Corp,DC=ad,DC=acme,DC=se
-
add: member
member: CN=Another User,OU=Corp,DC=ad,DC=acme,DC=se
-
add: member
member: CN=Service User,OU=Corp,DC=ad,DC=acme,DC=se
-

[dirmgr]

For LDIF files that may contain a mix of entries and change records, you should use the LDIFReader.readLDIFRecord() method rather than LDIFReader.readEntry(). You can then use something like:

     while (true)
     {
       LDIFRecord r = ldifReader.readLDIFRecord();
       if (r == null)
       {
         // End of the LDIF file.
         break;
       }
       else if (r instanceof Entry)
       {
         // Treat it as an Entry
       }
       else
       {
         // Treat it as an LDIFChangeRecord
       }
     }

Another option would be to use the LDIFReader.readChangeRecord(boolean) method with a value of true for the defaultAdd argument. This will cause the LDIF reader to always read LDIF records as if they were change records, and will assume that any record without a changetype element is an add change record. For example, if you're reading an LDIF file with a mix of entries and change records and want to send all of those to an LDAP server, you could use something like:

     while (true)
     {
       LDIFChangeRecord r = ldifReader.readChangeRecord(true);
       if (r == null)
       {
         // End of the LDIF file.
         break;
       }

       r.processChange(ldapConnection);
     }

[idsecurity]

Thanks, one more question.
Just to make sure, I want to "transform" the entries while reading them using different implementations of the LDIFReaderEntryTranslator interface depending on what I want to do.

For example I do this:

1. Create new LDIFReader with a LDIFReaderEntryTranslator
2. Create new LDIFWriter
3. Read using LDIFReader and writing directly to a new LDIF using LDIFWriter

This means if I use readChangeRecord/readLDIFRecord it won't be processed by the LDIFReaderEntryTranslator since those methods don't produce Entry objects?

This is actually not a problem for me, I can apply the same logic that I use in the LDIFReaderEntryTranslator implementation to the LDIFChangeRecord.

[dirmgr]

Although it's not yet in a release, there is code checked into the repository that includes an equivalent of an LDIFReaderEntryTranslator for LDIF change records. So you could check out and build the LDAP SDK for yourself if you want to try that.

But the translator interface is really only beneficial if you're using multiple threads to read data and decode data from the LDIF file concurrently, since in that case the translator also is invoked concurrently. In most cases, a single-threaded approach is good enough, and you can just apply whatever transformation you want in your code after reading the change record and before writing it out to the new file.

[idsecurity]

I've found the translator interface to be beneficial since it's so easy to implement it in JavaScript for quick transformation/processing/prototyping using jjs and Nashorn in Java 8. I can just specify a different JavaScript file each time and I don't need to compile anything.