Skip to content
This repository has been archived by the owner. It is now read-only.

Let's get the cellular modem working! #598

Closed
ollieparanoid opened this issue Sep 19, 2017 · 11 comments
Closed

Let's get the cellular modem working! #598

ollieparanoid opened this issue Sep 19, 2017 · 11 comments

Comments

@ollieparanoid
Copy link
Member

@ollieparanoid ollieparanoid commented Sep 19, 2017

Here is research by @pavelmachek, @craftyguy and @tssk together with some of my own on how to continue with cellular modems in postmarketOS.

Possible stacks

  1. Usual Android stack (for reference):

modem -- proprietary-vendor-blob -- rild -- android-userland

  1. Samsung Androids with rild:

modem -- libsamsung-ipc -- Samsung-RIL -- rild -- ofono -- dbus -- networkd

  1. Samsung Androids without rild (dead end):

modem -- libsamsung-ipc -- ofono -- dbus -- networkd

Requires this ofono patchset, which only builds with the outdated libsamsung-ipc 1.x (we're at 3.x now).
(It needs a struct ipc_message_info from a header file, that does not exist anymore, for starters. We would probably need to rewrite and upstream/maintain the whole patch!)

  1. N900 (already working!):

modem -- ofono (with patches) -- dbus -- networkd

Components

  • modem: the hardware cellular modem (runs proprietary firmware, we can't change that now)
  • oFono: almost every Linux phone project (Ubuntu touch, SailfishOS, ...) uses this program to talk to the modem. It translates the low-level stuff to a D-Bus API.
    It can talk to some modems directly, or indirectly via rild for example.
  • rild: Android's radio interface layer implementation. It always works together with a vendor implementation, that accesses the hardware. The code seems to be pretty isolated from the rest of Android, and it is written in C/C++, so it should be possible to package it for pmOS. It even has tagged versions (for Android releases, but it's good enough).
  • networkd: Something like NetworkManager or connman. NM is used by plasma-mobile, Ubuntu Touch/ubports and has great support for using a random MAC address. According to @craftyguy's research it isn't really documented how to use it with ofono though, while he got it working pretty well with connman instead.
  • libsamsung-ipc, Samsung-RIL: both maintained by the Replicant project, basically the reverse engineered counterpart of the vendor RIL implementation.

Recommendations for next steps

  • Package rild together with a custom OpenRC service file
  • Package Samsung-RIL, also with an OpenRC service file
  • Make sure that ofono is working (there are ofono test scripts)
  • Make sure, that a networkd is working
  • Document the actual implementation in the wiki
  • Close this ticket

What about other devices?

  • For Androids, the tricky part is to have a FLOSS-pendant to the vendor implementation.
  • When you have that, you can basically do the 2nd stack.
  • Alternatively, find a patch that makes ofono directly talk to your modem.
  • If none of these exist, you'll need to write the modem driver yourself.
@craftyguy
Copy link
Member

@craftyguy craftyguy commented Sep 19, 2017

Just for clarification, the N900 'working' stack is:

isimodem driver (in ofono) -- ofono (upstream master branch) -- connman-1.35 (not in Alpine)

I omitted the kernel driver(s) necessary for this, and the custom udev rule for the N900 (already in device-nokia-rx51 package)

The other bit we need to consider is that front-end frameworks, like connman (and most likely networkmanager/modemmanager) require configuration of the APN in ofono before they'll 'detect' the modem and use it. This is simply a mattery of creating/editing the relevant gprs file in /var/lib/ofono/<SIM #>/gprs to add the APN. We may want to create a frontend for this, perhaps using pmbootstrap init?

@zhuowei
Copy link
Collaborator

@zhuowei zhuowei commented Sep 20, 2017

@ollieparanoid so the rild used here will be compiled for Alpine, and wouldn't work at all for proprietary RIL implementations, correct?

I was looking at Qualcomm devices: Ofono can talk to Qualcomm modems via the QMI protocol, but it only supports GobiNet and cdc_wdm, not the shared memory communications used in Snapdragon devices. On Android, a daemon named qmuxd talks to the modem via shared memory; more info about that daemon can be found at https://osmocom.org/projects/quectel-modems/wiki/Qualcomm_Linux_SMD#packet-ports, https://github.com/scintill/qmiserial2qmuxd

To boot a Snapdragon modem: on Nexus 6P Android boots the modem by mounting the modem partition to /firmware and then opening /dev/subsys-modem, but doing the same on pmOS gives a kernel panic. Not sure what step I'm missing.

@ollieparanoid
Copy link
Member Author

@ollieparanoid ollieparanoid commented Sep 20, 2017

so the rild used here will be compiled for Alpine

That's my recommendation, yes.

and wouldn't work at all for proprietary RIL implementations, correct?

As I understand it, it might work in theory if someone got libhybris compiled for Alpine, and manages to execute the proprietary vendor RIL blob with that libhybris. Or simply by copying the libraries it links against (check with ldd). However, then we would have proprietary code running which I would rather avoid, and we would depend on the vendor again to provide updates for that piece of code. So if we can do it with open source somehow, I would prefer that personally.

Great research on the Qualcomm devices. About the kernel panic: maybe the kernel tries to load an userspace program, that is not available in postmarketOS, or that does not work for other reasons (linking against libraries we don't have, such as bionic libc)? Maybe you could add some debugging prints to the driver code in the kernel and find out where it crashes exactly.

If there is no open alternative to the proprietary qmuxd, you could try to run it with libhybris or by copying the shared libraries it links against (bionic libc!) to the device and testing if it runs. If that works, we could at least sandbox it (with firejail or bubblewrap for example), and maybe reverse engineer it in the long run (similar to libsamsung-ipc).

(I need to read more into how libhybris works, please correct me when I got something wrong.)

So if I understood you correctly, a possible snapdragon modem stack would be:

modem -- qmuxd (proprietary!) -- qmiserial2qmuxd -- ofono -- dbus -- networkd

@zhuowei
Copy link
Collaborator

@zhuowei zhuowei commented Sep 20, 2017

@ollieparanoid My preferred stack would be
modem->some sort of reimplemented qmuxd that translates from shared memory directly to cdc-wcm->ofono->whatever, so I'm planning to strace qmuxd later to see what it does. (That's OK, right?)
The kernel panic seems to be caused by the modem not initializing correctly; I was planning to enable debug statements in the Android kernel to see what was going on, but I was busy recently.

@ollieparanoid
Copy link
Member Author

@ollieparanoid ollieparanoid commented Oct 5, 2017

To drive this issue forward, I have looked into packaging Android's rild for postmarketOS. TBH it is not as easy as I thought, but I've made some progress already (see the android-ril branch). Feel free to continue from that, whoever reads this.

If we had rild available, we wouldn't need to wire up ofono with the current version of libsamsung-ipc. Doing that would be better in the long run of course, but we might get a proof of concept faster with rild.

@ollieparanoid
Copy link
Member Author

@ollieparanoid ollieparanoid commented Oct 20, 2017

@morphis: You wrote a patchset, that wires up ofono with libsamsung-ipc 1.x. We would like the latest greatest libsamsung-ipc 3.x, which is now maintained by Replicant. Do you have any interest in "refactoring" (more like rewriting it from scratch?) your patchset so it works with the new libsamsung-ipc, or could you help us with that task in case we decide to do it?

If you're wondering where the hell you are now, this is postmarketOS, a real Linux distribution for smartphones. I've seen on your GitHub profile, that you're part of the libhybris, webOS-ports and Anbox teams, which is all somewhat related to what we do.

@lawl
Copy link
Contributor

@lawl lawl commented Dec 17, 2017

i looked a bit more into @zhuowei's research. I have no idea why he kernel panics when he brings up the modem from pmos, by accessing /dev/subsys_modem. It works fine on my phone here, so maybe a kernel problem? Anyways I ripped out qmuxd out of the lineageOS install, including all libraries and the linker. And am able it to strace it from pmos.

openat(AT_FDCWD, "/system/etc/data/qmi_config.xml", O_RDONLY) = -1 ENOENT (No such file or directory)
pipe2([4, 5], O_NONBLOCK)               = 0
socket(AF_UNIX, SOCK_STREAM, 0)         = 6
unlinkat(AT_FDCWD, "/dev/socket/qmux_radio/qmux_connect_socket", 0) = -1 ENOENT (No such file or directory)
bind(6, {sa_family=AF_UNIX, sun_path="/dev/socket/qmux_radio/qmux_connect_socket"}, 110) = -1 ENOENT (No such file or directory)
close(6)                                = 0
socket(AF_UNIX, SOCK_STREAM, 0)         = 6
unlinkat(AT_FDCWD, "/dev/socket/qmux_audio/qmux_connect_socket", 0) = -1 ENOENT (No such file or directory)
bind(6, {sa_family=AF_UNIX, sun_path="/dev/socket/qmux_audio/qmux_connect_socket"}, 110) = -1 ENOENT (No such file or directory)
close(6)                                = 0
socket(AF_UNIX, SOCK_STREAM, 0)         = 6
unlinkat(AT_FDCWD, "/dev/socket/qmux_bluetooth/qmux_connect_socket", 0) = -1 ENOENT (No such file or directory)
bind(6, {sa_family=AF_UNIX, sun_path="/dev/socket/qmux_bluetooth/qmux_connect_socket"}, 110) = -1 ENOENT (No such file or directory)
close(6)                                = 0
socket(AF_UNIX, SOCK_STREAM, 0)         = 6
unlinkat(AT_FDCWD, "/dev/socket/qmux_gps/qmux_connect_socket", 0) = -1 ENOENT (No such file or directory)
bind(6, {sa_family=AF_UNIX, sun_path="/dev/socket/qmux_gps/qmux_connect_socket"}, 110) = -1 ENOENT (No such file or directory)
close(6)                                = 0
socket(AF_UNIX, SOCK_STREAM, 0)         = 6
unlinkat(AT_FDCWD, "/dev/socket/qmux_nfc/qmux_connect_socket", 0) = -1 ENOENT (No such file or directory)
bind(6, {sa_family=AF_UNIX, sun_path="/dev/socket/qmux_nfc/qmux_connect_socket"}, 110) = -1 ENOENT (No such file or directory)
close(6)                                = 0
pselect6(6, [4], NULL, NULL, NULL, NULL

Since it's hanging on pselect i'm fairly sure it's not crashing. it's just not working either (yet) because it can't find it's devices.

All libraries + qmuxd are 4.5MB. Considering all of /firmware is 130+ MB on this phone here, I'm not too worried about 4.5MB of sandboxed code to get cellular up and running, on potentially A LOT of devices. Quallcom still has quite a bit of market share I think, and this could potentially allow us to work on every single quallcom modem, at the cost of running a bit more proprietary code (for now).

Edit: More research/documentation/reversing on the qmuxd: https://www.contextis.com/blog/targeting-android-ota-exploitation

@zhuowei
Copy link
Collaborator

@zhuowei zhuowei commented Dec 17, 2017

@lawl you need additional daemons - I believe per_mgr/per_proxy boots the modem (they open /dev/subsys_modem on Android, according to the selinux policy); when I tried running them alone in Android's recovery they didn't try booting the modem, so we may need rild as well.

Also, https://osmocom.org/projects/quectel-modems/wiki/Wiki may help (it's a reverse engineering attempt for Qualcomm's blobs on a related modem)

@lawl
Copy link
Contributor

@lawl lawl commented Dec 29, 2017

Qualcomm discussion/documentation/progress tracking split out to here.

@mirh
Copy link

@mirh mirh commented Jan 22, 2018

So.. Not sure which thread to post, but afaik there are free implementations of RIL on both Qualcomm and ST-Ericsson

@ollieparanoid
Copy link
Member Author

@ollieparanoid ollieparanoid commented Mar 13, 2018

This is a broad topic. Created a wiki page for now, let's create smaller issues when we're working on something specific. Thanks for all the input everyone, we've progressed greatly since the issue was started!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.