Incoming "updates" messages stop after first outbound API call

[leonerd]

I have some proof-of-concept client code which is able to send or receive messages from Telegram. When I start it up, it performs the updates.getState call which enables the receipt of "updates" messages, and my client processes those to handle incoming messages. But the first time I make an outbound API call, perhaps to send a message, the incoming updates stop working.

Tested with both the current stable build + my one-line "emit" patch (cf17f2b) and with the upstream "development" branch as installed by npm install telegram-mtproto@3.0.5-unstable.

[leonerd]

I suspect what is happening (though at present I have no firm evidence) is that each new API call outbound is somehow establishing a new session. Using the stable branch there's no terminal output, but on the unstable one, it prints AUTH a few times around every outbound API hit, so I wonder if that's setting up a new session.

In any case, this appears to collide with Telegram's documentation on how to handle updates, as per https://core.telegram.org/api/updates:

Therefore, you should not start several sessions for regular API queries, but if you must, all messages with updates must be processed in each of them.

[leonerd]

This theory does seem to be correct, in that if I schedule a call to updates.getState again every 5 seconds, then that seems to fix the incoming updates after an outbound call. Admittedly it has lost any message that would have been received in that time, but I should be able to apply the getDifferences mechanism for the seq and pts parameters to fix those up... Quite some workaround though.. ;)

[leonerd]

Hm. Well, in theory that technique might have legs, but in practice I don't think I can make it work.

I've been able to find little real concrete documentation about how the seq and pts sequence numbers work, so the following is all entirely a guess based on some casual observations I have made:

Only non-"short" updates have a seq sequence number to them; most of the updates I receive are "short" ones, so I'm unable to use a seq number gap to detect missing ones.

All updates, even "short" ones, do seem to bump the pts value but from what I can observe, each target (chat or user dialog, or channel) has its own pts sequence numbering (leading to my guess that PTS stands for "Per-Target Sequence"). This means that any gaps in "short" update delivery can be detected on a per-target basis once a later message is received, but not before it.

The reason here is that the updates.getState call returns the overall seq number and from what I can tell some random guess of a recent pts value. I don't see any way to find out the pts values of all the user's subscribed targets, so as to detect ahead of time if there's a gap in the message flow.

So, uhm... I basically have no idea :) I'm also finding it impossible to dig out any better documentation, either from Telegram themselves, or any third party, on how the whole seq + pts mechanism is supposed to be used.

[leonerd]

Oh, and in addition: if I did detect a hole in the flow of updates by comparing sequence numbers, I'd have to call updates.getDifference() to fetch it. This being an outbound API call, would move the updates session sticky pointer and thus break the flow of updates anyway, further compounding the very problem it attempts to solve.

[leonerd]

OK after much further experimentation on actual observed values I come to a more accurate observation.

Every Channel-type peer maintains its own pts sequence numbering, but the pts numbering of all the Chats and User dialogs all share the same counter, and it is that counter which the toplevel updates.getState returns. There isn't, as far as I can see, an API to query the current pts values of all my current channels.

I could poll the updates.getChannelDifferences API for every known channel, but at that point that's a lot of API calls to be making. I feel this isn't a viable workaround strategy.

[wfjsw]

Have you tried messages.getDialogs which seems related?

[leonerd]

Have you tried messages.getDialogs which seems related?

It's entirely unrelated. What that does is lists all of the current "dialogs" the user has - in particular that's all the users and chats, not the channels. So in fact it's the opposite of what I want.

There is a call channels.getChannels which returns a vector of Chat instances; I imagine in that case they'd use the channel constructor, among whose fields there does not appear to be a pts. So that won't work either.

[zerobias]

@leonerd
I will implement this feature as a kind of middleware embedded in the incoming message stream, which will encase every type of related objects separately and provides pts/seq handling.

But first I must fix message handling which was broken here
Because I suspect that it will not be possible to adjust the updates without a stable messages sending

But it's great that you noticed the pattern - that updates disappear after the first outgoing event. I have been long looking for ways to resolve this issue without the previous recursive function leading to the large message flood

[leonerd]

It surely just requires the client not to establish a new session for absolutely every outbound API call. The old code I had, based on telegram.link, didn't do that. I'm currently midway through reading your code to see where it creates new sessions, to see if I can make it not do so

[Flohack74]

Just to note, PTS means persistent time stamp ;)

[zii]

Then what is qts mean?

[Flohack74]

@zii its the same like pts for secret chats (in this case it means queue)...

[munrocket]

Telethon in python and opentl in c# supporting this feature. It's a client librarys with update event lisener.