OMEMO support

[jubalh]

http://conversations.im/omemo/

Would be nice

[PMaynard]

I think this would be quiet nice, it would also attract potential new users.
The algorithm appears to be documented quite well https://github.com/trevp/axolotl/wiki and the extra XMPP specific appears to be quiet small http://xmpp.org/extensions/xep-0163.html

[quite]

Just read about OMEMO, good vibes!

[philipflohr]

A c library is available at
https://github.com/WhisperSystems/libaxolotl-c

[PMaynard]

👍

How functional is that library? Is there any relation to the Google summer of code person who was implementing OMEMO with conversations? It is the sister C library which they used in the conversation's GSoC development blog here

[philipflohr]

This Blog is only about the integration into Conversations. The library was developped by OpenWhisperSystems - the company behind TextSecure / Signals

There were no updates to the library - but there are also no reported issues.

[ReneVolution]

Would be possible to leverage the Python Plugins API to use e.g. https://python-omemo.readthedocs.org/en/latest/ ?

[boothj5]

@ReneVolution I'm still working on the python support (it works but need to tidy up object referencing and threading).

Releasing an ONEMO plugin along with the 0.5.0 release would be a really great thing to have, however I probably don't have time to do both the plugins framework and the plugin!

If anyone would be interested in attempting this ONEMO plugin, let me know, there are no doubt additional functions/hooks that will be needed in the API but I can implement those alongside any plugin work.

[ReneVolution]

Hey @boothj5,

i will try to work on that during a PythonCamp (https://barcamptools.eu/pycamp201604/sessions) and may need a bit help in advance.

From what i understand i need to at least have control over the actual message sending/receiving and manipulate the raw message data there. Is that correct so far? Can you give me some hints on that?

[boothj5]

Nice one.

Happy to help out as much as I can before the event.

As you say, we'll probably need to access the raw XMPP stanza's on send and receive, and the ability to send stanza's from a plugin, which is not currently in the API.

In principle this should not be too hard to add, the underlying XMPP library has various functions to get the stanza raw text, and send raw stanzas (although the latter is not in the libraries public API, again should be easy to add).

Another thing which might be useful that I'm planning to add, is to allow plugins to store and retrieve custom settings, similar to how they currently can use their own themes.

I'll keep this issue up to date as I make the changes, and feel free to post any ideas/questions etc.

[ReneVolution]

Awesome.

Yes, storing and retrieving plugin settings would be great as well. I am also working on another plugin which ON | OFF states would be great to be saved.

Also the state has to be saved per connection in the same way as with e.g. /otr start from a plugin side.

Thanks for helping with this.

[boothj5]

I've added prof.send_stanza() to the API, the python-test.py plugin has an example, sending a ping.

Also created a new issue which I'll update as the functions are added #772

[boothj5]

As per #772 the send and receive stanza hooks have been added.

I'm guessing the OMEMO plugin would return False from the prof_on_message_stanza_receive hook so it can process the data as needed, this will also need some kind of way to display the message once decrypted etc. I'll look into adding an API call prof_incoming_message() which will do all the window creation, unread message count, notifications, encryption type, message display etc as if it were a regular message.

I did originally think the receive hooks could return a modified stanza (or the original) for Profanity to continue processing similar to the send hooks. But then Profanity could easily be broken by a plugin returning a stanza it doesn't know how to process (which is not the case with sending). So it seemed better to let the plugin choose to take ownership of processing the stanza, as long as the required UI functions are available in the API.

[ReneVolution]

Wow man, that is really great news. I am really looking forward for the weekend now :).
Your decision is spot on when i think about it. Really great progress. Thanks a lot for the effort.

[ReneVolution]

Hey,

just wanted to check if there is any progress on the prof_incoming_message() hook? I have just started working on the plugin. :)

[boothj5]

@ReneVolution sorry I missed you in chat, was out all day.

How did things go? I have added a prof_incoming_message hook, which opens chat window etc.

[ReneVolution]

@boothj5 no worries.

Things did go well so far. Anyway i did not manage to create a prototype yet. The Barcamps seems to be more about talks than about coding :), Will try to work on it tomorrow.

You can follow the progress here: https://github.com/ReneVolution/profanity-omemo-plugin

Thanks for adding the missing hook. Talk to you soon.

[ReneVolution]

Hey @boothj5,

i am actually stuck with a part of the spec i can't figure out and hope you can help me out here.

"In order to determine whether a given contact has devices that support OMEMO, the devicelist node in PEP is consulted. Devices MUST subscribe to 'urn:xmpp:omemo:0:devicelist' via PEP, so that they are informed whenever their contacts add a new device. They MUST cache the most up-to-date version of the devicelist."

So, from what i understand is that i have to subscribe to the devicelist event, but i actually can't figure our how. Do i need to add this to the disco in some way?

Any help help here is desperately appreciated :)

[iNPUTmice]

So, from what i understand is that i have to subscribe to the devicelist event, but i actually can't figure our how. Do i need to add this to the disco in some way?

add urn:xmpp:omemo:0:devicelist+notify to your disco.

Since the XEP doesn't have an official number yet both gajim and Conversations use a private namespace. So you might want to make the name space configurable in some central place and use the Conversations namespace for the time being.

Have a look here: https://github.com/siacs/Conversations/blob/master/src/main/java/eu/siacs/conversations/crypto/axolotl/AxolotlService.java#L54

[ReneVolution]

Thanks for helping out here. I'm happy i was on the right track :)
Unfortunately the hook for that seems to be missing. Will open another Issue for that then.

[cryptono]

@ReneVolution Any news? Are you still working on it?

[ReneVolution]

I am indeed. But still can't confirm any ETA for it.

[shibumi]

Any News about this issue? CC: @ReneVolution

[ReneVolution]

Ooohh /o\ ... this one fell so off my radar. Will try to catch up next weekend.
Sorry for the delay.

[ReneVolution]

Hey everyone,

sorry for being not very responsive on this project. I've recently become a dad and so the focus shifted drastically 😸. Also, I think @reedts approach (providing a native c plugin) which may be included upstream is the better approach.

I am looking forward to see that in action. That being said, I am no longer maintaining the Python plugin. In case anyone still wants to pick it up as-is - please feel free too.

Thanks. It was a great journey in a great community.
Cheers.

[reedts]

@divansantana Unfortunately still WIP.

[jubalh]

We try to have OMEMO support in profanity 0.7.0

[moridius]

That would be great! OMEMO is the only reason why I'm not using profanity anymore.

[philipflohr]

@jubalh is there any effort to implement omemo within profanity? Currently there is not much progress on the c plug in.

[jubalh]

@philipflohr yes someone started two days ago to work on implementing it directly in profanity. makes good progress. do you want to join? I saw that you cloned the c plugin recently.

[philipflohr]

I'm not sure if I'm able to do much work as I'm currently writing my master thesis but I'm interested in helping.

[paulfariello]

@jubalh @philipflohr draft PR is #1039. To do list has still some items but we can start to send receive encrypted messages.

[jubalh]

Great work @paulfariello !

[paulfariello]

#1039 PR should have enough progress so it can be tested. Feel free to play with it and to comment the PR if you find any issue.

[jubalh]

When possible I always look at each commit done to the PR. It would be nice to have @pasis opinion and review at one point too. Maybe even @iNPUTmice could also take a look at OMEMO implementation.

[NiklausHofer]

Hey Guys,

I have been building new versions of profanity from the 'feature/omemo' branch for a couple of days now. However, lately profanity would get stuck during connecting to the server. I have now taken the time to find the exact commit that breaks for me.

• last working: 7f27df9
• first broken: 34ad7a5

What happens is that profanity starts normally, but when I type /connect it freezes after printing these lines to the UI:

16:16 - Using default account vimja.
16:16 - Connecting with account vimja as vimja@xmpp.honet.ch/octarine

At this point it does no longer accept any input or print any output. Also my account never appears 'online' so I guess profanity is not sending out any further messages either. Debug output stops as well. I kann still kill the process with signal 15.

If I remove the omemo information (rm -rf .local/share/profanity/omemo/) and start profanity again, everything works as expected. However, if I re-generate new omemo keys (/omemo gen) and then restart profanity, the same error occurs again.

Here is some information for the broken version:

profanity --version

Profanity, version 0.6.0dev.HEAD.34ad7a50
Copyright (C) 2012 - 2018 James Booth <boothj5web@gmail.com>.
License GPLv3+: GNU GPL version 3 or later <https://www.gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Build information:
XMPP library: libmesode
Desktop notification support: Enabled
OTR support: Enabled (libotr 4.1.1)
PGP support: Disabled
C plugins: Disabled
Python plugins: Disabled
GTK icons: Disabled

ldd:

	linux-vdso.so.1 (0x00007fffd91f5000)
	libcurl.so.4 => //usr/lib64/libcurl.so.4 (0x00007ff43879b000)
	libnotify.so.4 => //usr/lib64/libnotify.so.4 (0x00007ff438592000)
	libgio-2.0.so.0 => //usr/lib64/libgio-2.0.so.0 (0x00007ff4381ee000)
	libgobject-2.0.so.0 => //usr/lib64/libgobject-2.0.so.0 (0x00007ff437f9a000)
	libglib-2.0.so.0 => //usr/lib64/libglib-2.0.so.0 (0x00007ff437c7d000)
	libotr.so.5 => //usr/lib64/libotr.so.5 (0x00007ff437a63000)
	libgcrypt.so.20 => //usr/lib64/libgcrypt.so.20 (0x00007ff43773a000)
	libreadline.so.7 => /lib64/libreadline.so.7 (0x00007ff4374e7000)
	libncursesw.so.6 => /lib64/libncursesw.so.6 (0x00007ff43726b000)
	libmesode.so.0 => //usr/lib64/libmesode.so.0 (0x00007ff437051000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff436e31000)
	libc.so.6 => /lib64/libc.so.6 (0x00007ff436aa5000)
	libssl.so.1.1 => /usr/lib64/libssl.so.1.1 (0x00007ff436832000)
	libcrypto.so.1.1 => /usr/lib64/libcrypto.so.1.1 (0x00007ff436393000)
	libz.so.1 => /lib64/libz.so.1 (0x00007ff43617b000)
	libgdk_pixbuf-2.0.so.0 => /usr/lib64/libgdk_pixbuf-2.0.so.0 (0x00007ff435f53000)
	libgmodule-2.0.so.0 => /usr/lib64/libgmodule-2.0.so.0 (0x00007ff435d4f000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x00007ff435b38000)
	libmount.so.1 => /lib64/libmount.so.1 (0x00007ff4358d6000)
	libffi.so.6 => /usr/lib64/libffi.so.6 (0x00007ff4356cd000)
	libpcre.so.1 => /lib64/libpcre.so.1 (0x00007ff435488000)
	libgpg-error.so.0 => /usr/lib64/libgpg-error.so.0 (0x00007ff435264000)
	libncurses.so.6 => /lib64/libncurses.so.6 (0x00007ff434ffd000)
	libexpat.so.1 => /usr/lib64/libexpat.so.1 (0x00007ff434dc7000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ff438d69000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007ff434bc3000)
	libm.so.6 => /lib64/libm.so.6 (0x00007ff4348e3000)
	libblkid.so.1 => /lib64/libblkid.so.1 (0x00007ff43468c000)
	librt.so.1 => /lib64/librt.so.1 (0x00007ff434484000)
	libuuid.so.1 => /lib64/libuuid.so.1 (0x00007ff43427c000)

last lines of DEBUG output:

07/03/2019 16:16:49: xmpp: DBG: Bind successful.
07/03/2019 16:16:49: conn: DBG: SENT: <iq id="_xmpp_session1" type="set"><session xmlns="urn:ietf:params:xml:ns:xmpp-session"/></iq>
07/03/2019 16:16:49: xmpp: DBG: RECV: <iq id="_xmpp_session1" to="vimja@xmpp.honet.ch/octarine" type="result"/>
07/03/2019 16:16:49: xmpp: DBG: Session establishment successful.
07/03/2019 16:16:49: prof: DBG: Connection handler: XMPP_CONN_CONNECT
07/03/2019 16:16:49: prof: DBG: Connection handler: logged in with account name: vimja
07/03/2019 16:16:49: prof: INF: Loading OTR key for vimja@xmpp.honet.ch
07/03/2019 16:16:49: prof: INF: No OTR private key file found /home/sst-nho/.local/share/profanity/otr/vimja_at_xmpp.honet.ch/keys.txt
07/03/2019 16:16:49: prof: INF: No OTR fingerprints file found /home/sst-nho/.local/share/profanity/otr/vimja_at_xmpp.honet.ch/fingerprints.txt

[paulfariello]

Hi @NiklausHofer

Thanks for giving a try on this PR.
Thanks also for the detailed bug report.

Right now I have no idea why 34ad7a5 would break connection. But let's dig into it anyway.

Could you gdb attach to the stuck profanity and copy/paste me a backtrace?

[NiklausHofer]

@paulfariello Thank you for the quick reply. Here we go:

#0  0x00007f2502d4b24b in select () from /lib64/libc.so.6
#1  0x00007f25039c751c in ?? () from //usr/lib64/libgcrypt.so.20
#2  0x00007f25039c0f20 in ?? () from //usr/lib64/libgcrypt.so.20
#3  0x00007f25039c211a in ?? () from //usr/lib64/libgcrypt.so.20
#4  0x0000564da4409f2e in omemo_random_func ()
#5  0x0000564da440d165 in curve_generate_private_key ()
#6  0x0000564da440d25d in curve_generate_key_pair ()
#7  0x0000564da4416b60 in signal_protocol_key_helper_generate_pre_keys ()
#8  0x0000564da4409037 in omemo_on_connect ()
#9  0x0000564da43c4fb9 in sv_ev_login_account_success ()
#10 0x0000564da43b526f in session_login_success ()
#11 0x00007f250320c1c6 in ?? () from //usr/lib64/libmesode.so.0
#12 0x00007f250321121f in ?? () from //usr/lib64/libmesode.so.0
#13 0x00007f250320de8a in ?? () from //usr/lib64/libmesode.so.0
#14 0x00007f250321a6cf in ?? () from //usr/lib64/libmesode.so.0
#15 0x00007f2500f8820f in ?? () from /usr/lib64/libexpat.so.1
#16 0x00007f2500f8931c in ?? () from /usr/lib64/libexpat.so.1
#17 0x00007f2500f8c12b in XML_ParseBuffer () from /usr/lib64/libexpat.so.1
#18 0x00007f2503210cac in xmpp_run_once () from //usr/lib64/libmesode.so.0
#19 0x0000564da43b06d2 in prof_run ()
#20 0x0000564da43ad0df in main ()

[paulfariello]

Does it stay stuck really long?
Seems to be waiting for random data to be available.
Can you strace profanity?

[NiklausHofer]

@paulfariello You are right, it was waiting for random data. I've had it wait for several minutes before killing it. Starting rngd solved the problem, it now starts as expeted. Thank you so much for your help!

[paulfariello]

Good to know.
I'm still curious how is it possible to run out of entropy on a modern system.
Could you tell me with a strace where gcrypt is trying to gets its random?

And good to know that libsodium didn't produce such an issue 😋 .

[NiklausHofer]

@paulfariello sure. I built a new version from HEAD and disabled rngd.

strace -p 5104

strace: Process 5104 attached
select(1024, [0], NULL, NULL, {tv_sec=0, tv_usec=471145}) = 0 (Timeout)
poll([{fd=0, events=POLLIN}], 1, 0)     = 0 (Timeout)
poll([{fd=0, events=POLLIN}], 1, 0)     = 0 (Timeout)
read(4, 0x5638b82778d0, 4000)           = -1 EAGAIN (Resource temporarily unavailable)
select(1024, [0], NULL, NULL, {tv_sec=1, tv_usec=0}) = 0 (Timeout)
poll([{fd=0, events=POLLIN}], 1, 0)     = 0 (Timeout)
poll([{fd=0, events=POLLIN}], 1, 0)     = 0 (Timeout)
read(4, 0x5638b82778d0, 4000)           = -1 EAGAIN (Resource temporarily unavailable)
select(1024, [0], NULL, NULL, {tv_sec=1, tv_usec=0}) = 0 (Timeout)
poll([{fd=0, events=POLLIN}], 1, 0)     = 0 (Timeout)
poll([{fd=0, events=POLLIN}], 1, 0)     = 0 (Timeout)
read(4, 0x5638b82778d0, 4000)           = -1 EAGAIN (Resource temporarily unavailable)
select(1024, [0], NULL, NULL, {tv_sec=1, tv_usec=0}) = 0 (Timeout)
poll([{fd=0, events=POLLIN}], 1, 0)     = 0 (Timeout)
poll([{fd=0, events=POLLIN}], 1, 0)     = 0 (Timeout)
read(4, 0x5638b82778d0, 4000)           = -1 EAGAIN (Resource temporarily unavailable)
select(1024, [0], NULL, NULL, {tv_sec=1, tv_usec=0}) = 0 (Timeout)

The output does not stop there, it keeps repeating over and over.

[NiklausHofer]

With Profanity now starting I had a chance to actually test the omemo functionality. Unfortunately, when I run /omemo start JID, profanity segfaults. Again I have searched the commit this first happens:

• last working: 8f9c68b
• first broken: b12bf63

Here is what gdb prints when I start profanity through gdb:

Program received signal SIGSEGV, Segmentation fault.
                                                    0x00005555555e3f1b in omemo_sessions_keyfile_save ()

I built from the current HEAD (a38d10b) for testing, here is the version information:

Profanity, version 0.6.0dev.HEAD.a38d10b8
Copyright (C) 2012 - 2018 James Booth <boothj5web@gmail.com>.
License GPLv3+: GNU GPL version 3 or later <https://www.gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Build information:
XMPP library: libmesode
Desktop notification support: Enabled
OTR support: Enabled (libotr 4.1.1)
PGP support: Disabled
C plugins: Disabled
Python plugins: Disabled
GTK icons: Disabled

[paulfariello]

@NiklausHofer can you provide a full backtrace from gdb and use a debug build of profanity?

[NiklausHofer]

Here is the backtrace. It will take me a moment before I can do a debug build of the software:

#0  0x00005555555e3f1b in omemo_sessions_keyfile_save ()
#1  0x00005555555e52b1 in store_session ()
#2  0x00005555555e6fbc in signal_protocol_session_store_session ()
#3  0x00005555555efe14 in session_builder_process_pre_key_bundle ()
#4  0x00005555555e40ff in omemo_start_device_session ()
#5  0x00005555555e5c46 in omemo_start_device_session_handle_bundle ()
#6  0x0000555555593a91 in ?? ()
#7  0x00007ffff6418375 in ?? () from //usr/lib64/libmesode.so.0
#8  0x00007ffff6414e8a in ?? () from //usr/lib64/libmesode.so.0
#9  0x00007ffff64216cf in ?? () from //usr/lib64/libmesode.so.0
#10 0x00007ffff418f575 in ?? () from /usr/lib64/libexpat.so.1
#11 0x00007ffff419031c in ?? () from /usr/lib64/libexpat.so.1
#12 0x00007ffff419312b in XML_ParseBuffer () from /usr/lib64/libexpat.so.1
#13 0x00007ffff6417cac in xmpp_run_once () from //usr/lib64/libmesode.so.0
#14 0x000055555558a972 in prof_run ()
#15 0x000055555558737f in main ()

[NiklausHofer]

OK, when I compile a debug version (-O0 -g3) it does not segfault. It segfaults with -O1 though.

[jubalh]

Would be super cool if comments about the current state of the OMEMO feature branch and testing it could be done at #1039

I think it belongs there. Thanks guys!

[jubalh]

@paulfariello did a great job with implementing OMEMO.
Several people helped with testing the feature branch #1039, I think @kaffeekanne was especially active. Thanks to everybody involved!

I hope a code review can happen soon and then I will merge into master.

Please everybody who has some time to spare help with reviewing the code in the feature branch or help by testing the changes.
If you need help to build profanity from that branch you can find us in profanity@rooms.dismail.de where we will try to help you.

If you test the branch and everything works fine, please still comment on this issue so we know more people tested and had no issues.

Thanks guys!

[jubalh]

After 4 years this issue can finally be closed.

Thanks a lot to @paulfariello for all his work implementing this, and to @kaffeekanne for testing!
Paul, don't forget the bounty ;)