Intermediate responses

[hlavaatch]

Hi,
I'm trying to make synchronization work using the sync request control of RFC4533 (1.3.6.1.4.1.4203.1.9.1.1).

So far I have managed to create the custom control and pass it to streaming_search().
As expected, initial results are returned and search does not end because SearchResultDone message is not sent, but I'm missing a way to receive the intermediate Sync Info Message that should be sent at this point, and I don't receive any changed entries upon changes in LDAP data. No error is returned. The stream seems to be stuck.

Am I missing something?

(I'm willing to help with this, and possibly with making more controls/ASN.1 stuff)

[inejge]

Heh, I knew that someday someone was going to try using the crate for persistent search, and that I'd be writing a "dog ate my homework" reply 🙂

You're not missing anything, I simply never got around to writing support for persistent searches. While it's true that a persistent search looks like a streaming Search if you squint hard enough, the latter wasn't written with the former in mind, so details like controls attached to SearchResultEntry PDUs and empty SREs for cookie/phase updates aren't handled at all.

I'd be interested in seeing the network packet trace of a stuck search. I suspect that the server is sending replies, but the driver is mishandling them in some way and not sending anything back to the requester.

Minimal Sync support, returning initial results and added/updated entries within a streaming Search, should be possible without much trouble, or so I hope. Better support would require either a custom persistent request, or returning raw entries from a streaming Search.

[hlavaatch]

The communication is over TLS, is there any way to capture the dump at the protocol level?

EDIT: Oh wait, I have the server's key :) I'll try capturing some traces for you with wireshark

[hlavaatch]

I managed to get a nice trace, but its a bit sensitive so I redacted it a bit.

What happens is after I start search with LdapConn::streaming_search(), passing itSyncRequestControl specifying mode of refresh and persist, no cookie and default reload hint of false:

Frame 22: 400 bytes on wire (3200 bits), 400 bytes captured (3200 bits) on interface 0
Ethernet II, Src: ##CLIENT-MAC## (##CLIENT-MAC##), Dst: ##SERVER-MAC## (##SERVER-MAC##)
Internet Protocol Version 4, Src: ##CLIENT-IP##, Dst: ##SERVER-IP##
Transmission Control Protocol, Src Port: 49380, Dst Port: 636, Seq: 678, Ack: 3215, Len: 346
Secure Sockets Layer
Lightweight Directory Access Protocol
    LDAPMessage searchRequest(3) "##ROOT-OF-LDAP##" wholeSubtree
        messageID: 3
        protocolOp: searchRequest (3)
            searchRequest
        [Response In: 23]
        controls: 1 item
            Control
                controlType: 1.3.6.1.4.1.4203.1.9.1.1 (syncRequestOID)
                criticality: True
                SyncRequestValue
                    mode: refreshAndPersist (3)

I get lots of search result entries with Sync State Control as expected and then an intermediate response of 1.3.6.1.4.1.4203.1.9.1.4 (marking end of the initial sync phase and start of update notifications) upon which client just drops the TCP connection and hangs. My code is still blocking inside the EntryStream::next() and has no idea something happened.

I think all that short term solution for now is for the client to not die on the intermediate response - it may even swallow it and continue sending entries on changes - as long as we can get a way to somehow get at the controls of the returned entries to get the type of change (present/add/modify/delete) from Sync State Control...

So changes needed are:

• Not die on intermediate response.
• Find a way to pass intermediate responses, maybe by instrumenting ResultEntry to represent both intermediate responses and actual search result entries
• Access to result entry controls

Lightweight Directory Access Protocol
    LDAPMessage searchResEntry(3) "uid=vavrojoz,ou=users,##ROOT-OF-LDAP##" [1192 results]
        messageID: 3
        protocolOp: searchResEntry (4)
            searchResEntry
                objectName: uid=vavrojoz,ou=users,##ROOT-OF-LDAP##
                attributes: 5 items
        [Response To: 22]
        [Time: 1.090603000 seconds]
        controls: 1 item
            Control
                controlType: 1.3.6.1.4.1.4203.1.9.1.2 (syncStateOID)
                SyncStateValue
                    state: add (1)
                    entryUUID: 135eea98c12110379d5271984042a3a3

No.     Time           Source                Destination           Protocol Length Info
    719 1.123246       ##SERVER-IP##          ##CLIENT-IP##         LDAP     160    intermediateResponse(3) syncInfoOID 

Frame 719: 160 bytes on wire (1280 bits), 160 bytes captured (1280 bits) on interface 0
Ethernet II, Src: ##SERVER-MAC## (##SERVER-MAC##), Dst: ##CLIENT-MAC## (##CLIENT-MAC##)
Internet Protocol Version 4, Src: ##SERVER-IP##, Dst: ##CLIENT-IP##
Transmission Control Protocol, Src Port: 636, Dst Port: 49380, Seq: 617323, Ack: 1024, Len: 106
[2 Reassembled TCP Segments (215 bytes): #718(109), #719(106)]
Secure Sockets Layer
Lightweight Directory Access Protocol
    IntermediateResponse 1.3.6.1.4.1.4203.1.9.1.4 (syncInfoOID)
        messageID: 3
        protocolOp: intermediateResponse (25)
            intermediateResponse
                responseName: 1.3.6.1.4.1.4203.1.9.1.4 (syncInfoOID)
                SyncInfoValue: refreshDelete (1)
                    refreshDelete
        [Response To: 22]
        [Time: 1.090606000 seconds]

No.     Time           Source                Destination           Protocol Length Info
    720 1.123250       ##CLIENT-IP##         ##SERVER-IP##          TCP      54     49380 → 636 [ACK] Seq=1024 Ack=617429 Win=14556 Len=0

Frame 720: 54 bytes on wire (432 bits), 54 bytes captured (432 bits) on interface 0
Ethernet II, Src: ##CLIENT-MAC## (##CLIENT-MAC##), Dst: ##SERVER-MAC## (##SERVER-MAC##)
Internet Protocol Version 4, Src: ##CLIENT-IP##, Dst: ##SERVER-IP##
Transmission Control Protocol, Src Port: 49380, Dst Port: 636, Seq: 1024, Ack: 617429, Len: 0

No.     Time           Source                Destination           Protocol Length Info
    721 1.729674       ##CLIENT-IP##         ##SERVER-IP##          TCP      54     [TCP Window Update] 49380 → 636 [ACK] Seq=1024 Ack=617429 Win=159260 Len=0

Frame 721: 54 bytes on wire (432 bits), 54 bytes captured (432 bits) on interface 0
Ethernet II, Src: ##CLIENT-MAC## (##CLIENT-MAC##), Dst: ##SERVER-MAC## (##SERVER-MAC##)
Internet Protocol Version 4, Src: ##CLIENT-IP##, Dst: ##SERVER-IP##
Transmission Control Protocol, Src Port: 49380, Dst Port: 636, Seq: 1024, Ack: 617429, Len: 0

No.     Time           Source                Destination           Protocol Length Info
    722 2.411564       ##CLIENT-IP##         ##SERVER-IP##          TCP      54     [TCP Window Update] 49380 → 636 [ACK] Seq=1024 Ack=617429 Win=247520 Len=0

Frame 722: 54 bytes on wire (432 bits), 54 bytes captured (432 bits) on interface 0
Ethernet II, Src: ##CLIENT-MAC## (##CLIENT-MAC##), Dst: ##SERVER-MAC## (##SERVER-MAC##)
Internet Protocol Version 4, Src: ##CLIENT-IP##, Dst: ##SERVER-IP##
Transmission Control Protocol, Src Port: 49380, Dst Port: 636, Seq: 1024, Ack: 617429, Len: 0

No.     Time           Source                Destination           Protocol Length Info
    723 2.922460       ##CLIENT-IP##         ##SERVER-IP##          TCP      54     49380 → 636 [FIN, ACK] Seq=1024 Ack=617429 Win=247520 Len=0

Frame 723: 54 bytes on wire (432 bits), 54 bytes captured (432 bits) on interface 0
Ethernet II, Src: ##CLIENT-MAC## (##CLIENT-MAC##), Dst: ##SERVER-MAC## (##SERVER-MAC##)
Internet Protocol Version 4, Src: ##CLIENT-IP##, Dst: ##SERVER-IP##
Transmission Control Protocol, Src Port: 49380, Dst Port: 636, Seq: 1024, Ack: 617429, Len: 0

No.     Time           Source                Destination           Protocol Length Info
    724 2.923305       ##SERVER-IP##          ##CLIENT-IP##         TLSv1.2  85     Alert (Level: Warning, Description: Close Notify)

Frame 724: 85 bytes on wire (680 bits), 85 bytes captured (680 bits) on interface 0
Ethernet II, Src: ##SERVER-MAC## (##SERVER-MAC##), Dst: ##CLIENT-MAC## (##CLIENT-MAC##)
Internet Protocol Version 4, Src: ##SERVER-IP##, Dst: ##CLIENT-IP##
Transmission Control Protocol, Src Port: 636, Dst Port: 49380, Seq: 617429, Ack: 1025, Len: 31
Secure Sockets Layer

No.     Time           Source                Destination           Protocol Length Info
    725 2.923346       ##CLIENT-IP##         ##SERVER-IP##          TCP      54     49380 → 636 [RST, ACK] Seq=1025 Ack=617460 Win=0 Len=0

Frame 725: 54 bytes on wire (432 bits), 54 bytes captured (432 bits) on interface 0
Ethernet II, Src: ##CLIENT-MAC## (##CLIENT-MAC##), Dst: ##SERVER-MAC## (##SERVER-MAC##)
Internet Protocol Version 4, Src: ##CLIENT-IP##, Dst: ##SERVER-IP##
Transmission Control Protocol, Src Port: 49380, Dst Port: 636, Seq: 1025, Ack: 617460, Len: 0

No.     Time           Source                Destination           Protocol Length Info
    726 2.923541       ##SERVER-IP##          ##CLIENT-IP##         TCP      60     636 → 49380 [FIN, ACK] Seq=617460 Ack=1025 Win=32512 Len=0

Frame 726: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) on interface 0
Ethernet II, Src: ##SERVER-MAC## (##SERVER-MAC##), Dst: ##CLIENT-MAC## (##CLIENT-MAC##)
Internet Protocol Version 4, Src: ##SERVER-IP##, Dst: ##CLIENT-IP##
Transmission Control Protocol, Src Port: 636, Dst Port: 49380, Seq: 617460, Ack: 1025, Len: 0

No.     Time           Source                Destination           Protocol Length Info
    727 2.923550       ##CLIENT-IP##         ##SERVER-IP##          TCP      54     49380 → 636 [RST] Seq=1025 Win=0 Len=0

Frame 727: 54 bytes on wire (432 bits), 54 bytes captured (432 bits) on interface 0
Ethernet II, Src: ##CLIENT-MAC## (##CLIENT-MAC##), Dst: ##SERVER-MAC## (##SERVER-MAC##)
Internet Protocol Version 4, Src: ##CLIENT-IP##, Dst: ##SERVER-IP##
Transmission Control Protocol, Src Port: 49380, Dst Port: 636, Seq: 1025, Len: 0

[inejge]

Thanks, that was informative and it confirmed my suspicion that the driver is choking on Intermediate responses. Try the search after applying this patch:

--- a/src/protocol.rs
+++ b/src/protocol.rs
@@ -194,7 +194,7 @@ impl Decoder for LdapCodec {
             },
         };
         match protoop.id {
-            op_id @ 4 | op_id @ 5 | op_id @ 19 => {
+            op_id @ 4 | op_id @ 5 | op_id @ 19 | op_id @ 25 => {
                 let null_tag = Tag::Null(Null { ..Default::default() });
                 let id_tag = Tag::Integer(Integer {
                     inner: id as i64,
@@ -205,13 +205,15 @@ impl Decoder for LdapCodec {
                     Some(h) => h,
                     None => return Err(io::Error::new(io::ErrorKind::Other, format!("id mismatch: {}", id))),
                 };
-                helper.send_item(match op_id {
-                    4 => SearchItem::Entry(protoop),
-                    5 => SearchItem::Done(id, Tag::StructureTag(protoop).into(), controls),
-                    19 => SearchItem::Referral(protoop),
-                    _ => panic!("impossible op_id"),
-                })?;
-                if helper.seen {
+                if op_id != 25 {
+                    helper.send_item(match op_id {
+                        4 => SearchItem::Entry(protoop),
+                        5 => SearchItem::Done(id, Tag::StructureTag(protoop).into(), controls),
+                        19 => SearchItem::Referral(protoop),
+                        _ => panic!("impossible op_id"),
+                    })?;
+                }
+                if helper.seen || op_id == 25 {
                     Ok(Some((u64::MAX, (null_tag, vec![]))))
                 } else {
                     helper.seen = true;

It's a gross hack which will discard Intermediate responses, but also continue receiving search results while the operation is active. Sync State controls and Sync Info messages are still unavailable, although addition, modification and even deletion of individual entries can be recognized.

I'll start working on a better solution once I publish the long overdue v0.6 of the crate.

[hlavaatch]

That works, alright. I am receiving updates and it does not die 👍
It seems I'm able to guess which updates are deletions by absence of any attributes even without access to the controls, so as long as I don't care about additions vs modifications it's fine...

Up with the 0.6!!!!

[hlavaatch]

Ugly hack or not, its still better than a crash and it works!
Why didn't you put this hotfix into 0.6?
Can you please make 0.6.1 with it so I can go on developing with unpatched dependencies?

[inejge]

(Sorry for the delay, I've been rather busy.) Please keep using the patch for a while, I really don't want to have nasty hacks, however minimal, in the published versions of the crate. A proper fix is coming along.

[hlavaatch]

Any progress? Would love to use official crate instead of hacked one...

[hlavaatch]

Alright, since you clearly have no time/motivation for this, would you let me help you with it?

[inejge]

It's more time than motivation (as you can see, my response time is atrocious atm), but it's true that I've been waiting for the async design to settle, probably at the detriment of useful work. So, go for it. My only guideline would be that I'd prefer the existing API to remain unchanged, which also means keeping things like io::Error as the error struct and not trying to introduce comprehensive error handling.

[hlavaatch]

Having looked at the code in depth, I can see this crate is in need of serious rewrite and now understand why you don't want to spend time updating it to run with recent tokio with async/await requiring another rewrite around the corner. Unfortunate.

[hlavaatch]

Okay. Now we have Futures 0.3, async/await, and tokio 0.2.
Time to start thinking about the (inevitable) rewrite? :)

[inejge]

... and it's done, mirabile dictu. All sync-related messages and controls can now be captured by user code.

[hlavaatch]

Amazing! :) Will try...

[StarlessNights]

An example of persistent search would be really nice. Right now this issue and the SyncRequest::RefreshAndPersist enum variant seem to be the only things suggesting that this is possible to begin with.

[inejge]

All examples distributed with the library are meant to work with OpenLDAP initialized from the data subdirectory. OpenLDAP doesn't support persistent search. I might accept a persistent search example which clearly documented the alternative server setup and its own calling options, but I don't think it's something I'd work on myself.

[inejge]

@StarlessNights Addendum: persistent search is this. From your mention of RefreshAndPersist, I realized that you may be thinking of LDAP Content Synchronization, RFC 4533, which OpenLDAP does support. Still, it's a highly specialized and very niche corner of the protocol, and I believe that it can't be reduced to general-purpose examples.

[StarlessNights]

@inejge Ahh right you are. Thanks for the clarification.