Unable to synchronize message history with Gajim

[fijam]

Environment

• ejabberd version: 21.01
• Erlang version: Erlang (SMP,ASYNC_THREADS) (BEAM) emulator version 10.2.4
• OS: Linux (Debian 10.10)
• Installed from: distro package

Errors from error.log/crash.log

No errors

Bug description

Attempting to manually synchronize the message archive (Accounts -> Advanced -> Synchronize History) between Gajim 1.3.2 and ejabberd fails.

Gajim reports:
(W) nbxmpp.m.mam (heap.ovh) Received malformed stanza from fijam@heap.ovh: missing first or last element
and does not progress.

Gajim's XML console shows:

<!-- Outgoing 19.07.2021 09:43:30 (fijam@heap.ovh) -->
<iq xmlns="jabber:client" to="fijam@heap.ovh" type="set" id="bb41cf70-db88-4328-a60c-cb9f167c4d7c">
  <query xmlns="urn:xmpp:mam:2">
    <x xmlns="jabber:x:data" type="submit">
      <field type="hidden" var="FORM_TYPE">
        <value>urn:xmpp:mam:2</value>
      </field>
      <field type="text-single" var="start">
        <value>2021-06-19T07:43:28Z</value>
      </field>
      <field type="text-single" var="end">
        <value>2021-07-19T07:43:28Z</value>
      </field>
    </x>
    <set xmlns="http://jabber.org/protocol/rsm">
      <max>0</max>
    </set>
  </query>
</iq>


<!-- Outgoing 19.07.2021 09:43:30 (fijam@heap.ovh) -->
<r xmlns="urn:xmpp:sm:3" />


<!-- Incoming 19.07.2021 09:43:30 (fijam@heap.ovh) -->
<iq xmlns="jabber:client" xml:lang="en" to="fijam@heap.ovh/gajim.KVN0TMA2AF74N3DH" from="fijam@heap.ovh" type="result" id="bb41cf70-db88-4328-a60c-cb9f167c4d7c">
  <fin complete="false" xmlns="urn:xmpp:mam:2">
    <set xmlns="http://jabber.org/protocol/rsm">
      <count>2885</count>
</set>
</fin>
</iq>


<!-- Incoming 19.07.2021 09:43:30 (fijam@heap.ovh) -->
<r xmlns="urn:xmpp:sm:3" />


<!-- Outgoing 19.07.2021 09:43:30 (fijam@heap.ovh) -->
<a xmlns="urn:xmpp:sm:3" h="35" />


<!-- Incoming 19.07.2021 09:43:30 (fijam@heap.ovh) -->
<a h="28" xmlns="urn:xmpp:sm:3" />

Nothing in ejabberd logs.

I initially thought this is a problem with the client and reported the issue at https://dev.gajim.org/gajim/gajim/-/issues/10596 (especially since an old version 1.1.3 works) but the Gajim developers claim this to be an issue with the server.

I am willing to provide additional information as needed.

[licaon-kter]

Can you post a gist of the config ?

[fijam]

Sure, here it is https://gist.github.com/fijam/d1d539d81c01f20b62344b2590145761

[licaon-kter]

This is MAM from 1:1 or external MUCs?

[fijam]

The whole MAM archive is ~70 000 messages so there may be some MUC there but the requested synchronization range (last month) is all 1:1

[licaon-kter]

Other clients like say Conversations work?

[fijam]

I actually use Conversations as well, but I don't know if it's possible to synchronize all chat history. If it is, I can't seem to find it.

I'll try Dino.

[licaon-kter]

In Conversations... scroll up... :)

Dino might pull it all and you need to wait... a lot.

Anyway, if no MAM is pulled try to get some logs from these two too.

[fijam]

Okay, so after a bit of testing I think I need to clarify this a bit.

Getting recent messages from MAM with Dino or Conversations works fine. In fact, it works in Gajim too. E.g. this query gets answered correctly

<iq xmlns="jabber:client" to="fijam@heap.ovh" type="set" id="f2df6752-30be-4aca-a40a-5fa4423adc9d">
  <query xmlns="urn:xmpp:mam:2" queryid="19e907c3-7960-4aa2-b817-f8bdf045cd56">
    <x xmlns="jabber:x:data" type="submit">
      <field type="hidden" var="FORM_TYPE">
        <value>urn:xmpp:mam:2</value>
      </field>
      <field type="text-single" var="start">
        <value>2021-07-12T08:48:30Z</value>
      </field>
    </x>
    <set xmlns="http://jabber.org/protocol/rsm">
      <max>70</max>
      <after>1626616288627891</after>
    </set>
  </query>
</iq>

What does not work is requesting a synchronization of entire chat history (see query in top post). It seems that the query and response are different in that case, and I don't know of any other client than Gajim which manually syncs all of chat history on request so I could test it.

[lovetox]

Hi, sorry i looked into it and its a bug in Gajim, you can close this.

[fijam]

Cheers!

[Neustradamus]

@badlop, @prefiks: Can you reopen ?
@lovetox has noted that it is a server problem in the Gajim ticket.

Can you look before 21.07 release?

[lovetox]

as you can see from my post 3 days ago, i said its not a server problem, so no need to reopen

[Neustradamus]

@lovetox: Sorry:
https://dev.gajim.org/gajim/gajim/-/issues/10596

@lovetox · 2 days ago
In both cases the server sends us invalid content please open an issue with the server software

[lovetox]

Please read the whole ticket not just some message in the middle