Modem vanishes randomly

[justinmbrock]

I am running the Phosh build on a 3GB PinePhone 1.2a modded to 1.2b.
Several other maintainers are troubleshooting issues with the modem disappearing (ex: https://invent.kde.org/teams/plasma-mobile/issues/-/issues/3). When the phone sleeps, the modem may or may not work when the phone wakes up, and restarting eg25-manager or resetting the usb port usually fixed it for most, but that's just a bad hack.

I've had this problem intermittently for months, but the last few weeks the phone doesn't even need to deep sleep for the modem to randomly disappear, plugged in or not. Restarting eg25-manager does not "fix" anything, and to get anything through the modem again I have to reboot the phone.

Here is something else I noticed. I just rebooted. Here is the output of ls /dev/ttyUSB* and lsusb:

[alarm@pinephone][~]% ls /dev/ttyUSB*
/dev/ttyUSB0  /dev/ttyUSB1  /dev/ttyUSB2  /dev/ttyUSB3

[alarm@pinephone][~]% lsusb
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
**Bus 002 Device 002**: ID 2c7c:0125 Quectel Wireless Solutions Co., Ltd. EC25 LTE modem
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

And about 2 minutes later after the modem vanished:

[alarm@pinephone][~]% ls /dev/ttyUSB*
/dev/ttyUSB0

[alarm@pinephone][~]% lsusb
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
**Bus 002 Device 004**: ID 2c7c:0125 Quectel Wireless Solutions Co., Ltd. EC25 LTE modem
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

The modem is a different device number now, in addition to the serial ports missing. If you want anything else let me know what you want and how to get it. If this is better addressed upstream, please point me in the right direction, and I'll go there. Thanks!

[Danct12]

Got dmesg? It's possible that the OS inside the modem crashed, which firmware do you use?

[Danct12]

Tomas Oqvist reported the same problem on Telegram, the user claimed that most of the modem vanishing caused while the phone is charging.

There seems to be no sign of USB EMI when the modem is lost. Once the modem is lost, eg25-manager daemon tries to reboot the modem but failed to do so.

jun 28 16:28:55 danctnix eg25manager[3334]: Lost modem, resetting...
jun 28 16:28:55 danctnix eg25manager[3334]: Couldn't unbind modem: wrote -1/3 bytes
jun 28 16:28:55 danctnix eg25manager[3334]: Couldn't bind modem: wrote -1/3 bytes

A few seconds later, a mysterious USB device appears on the bus which ModemManager doesn't know how to use it.

jun 28 16:29:02 danctnix kernel: usb 2-1: new high-speed USB device number 3 using ehci-platform
jun 28 16:29:02 danctnix kernel: option 2-1:1.0: GSM modem (1-port) converter detected
jun 28 16:29:02 danctnix kernel: usb 2-1: GSM modem (1-port) converter now attached to ttyUSB0
jun 28 16:29:02 danctnix systemd-udevd[5082]: 2-1: /usr/lib/udev/rules.d/80-modem-eg25.rules:3 Failed to write ATTR{/sys/devices/platform/soc/1c1b000.usb/usb2/2-1/power/wakeup}, ignoring: No such file or directory
jun 28 16:29:06 danctnix systemd[1]: NetworkManager-dispatcher.service: Deactivated successfully.
jun 28 16:29:06 danctnix audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj==unconfined msg='unit=NetworkManager-dispatcher comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
jun 28 16:29:06 danctnix kernel: audit: type=1131 audit(1624890546.501:83): pid=1 uid=0 auid=4294967295 ses=4294967295 subj==unconfined msg='unit=NetworkManager-dispatcher comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
jun 28 16:29:10 danctnix ModemManager[3495]: <info>  [device /sys/devices/platform/soc/1c1b000.usb/usb2/2-1] creating modem with plugin 'quectel' and '1' ports
jun 28 16:29:10 danctnix ModemManager[3495]: <warn>  [plugin/quectel] could not grab port ttyUSB0: Cannot add port 'tty/ttyUSB0', unhandled port type
jun 28 16:29:10 danctnix ModemManager[3495]: <warn>  [base-manager] couldn't create modem for device '/sys/devices/platform/soc/1c1b000.usb/usb2/2-1': Failed to find primary AT port

I personally think that there is something wrong with the software running on the modem as I have never seen this strange behavior before and I have no way to reproduce the issue on my end.

Modem software version: EG25GGBR07A08M2G_01.002.01.002

[lazmol]

Hello,
have the same problem but I am on Biktorgj's 0.2.1 modem. Just wanted to try going back to the vendor modem firmware but it seems the problem occurs there too...
I see it more often when the phone is hot (e.g. when charging, like mentioned above).
Attached the output of journalctl -b | grep -i "eg25\|modem"
journalctlb_modemfail_grepeg25modem.log

I might update to Biktorgj's latest firmware version and check.
Pls tell me if I can help somehow debugging this problem.

[justinmbrock]

Attached is a dmseg log from a reboot that ended with the modem vanishing after the screen timed out once. I'm running EG25GGBR07A08M2G_01.002.07 per the wiki instructions here.

dmesg.log

[Michasze]

Had the same problem. Downgrading eg25-manager to 0.3.0+2+g64145ac fixed the issue for me.

[returnofthelambda]

I had the same issue on a fresh install of the 6/28 build. Cell modem would disappear every couple of hours. Disabling the wifi/bluetooth via the physical kill-switch helped delay the issue, but did not fix it. I enabled the wifi/bluetooth toggle and then I tried downgrading eg25-manager as suggested by @Michasze. It also helped keep the modem connected longer, but the modem would still disappear after awhile. Running pacman -Syu last night pulled a new kernel (5.12.14-1) and I haven't had any issues since. I am still using eg25-manager version 0.3.0+2+g64145ac from the 6/13 build.

[kescherCode]

Jul 07 15:20:45 pinephone systemd[1]: Started Quectel EG25 modem.
Jul 07 15:20:45 pinephone eg25manager[3588]: Opening config file: /usr/share/eg25-manager/pine64,pinephone-1.2.toml
...
Jul 07 15:21:06 pinephone eg25manager[3588]: ModemManager object `/org/freedesktop/ModemManager1/Modem/0' added
...
Jul 07 17:14:33 pinephone eg25manager[3588]: ModemManager object `/org/freedesktop/ModemManager1/Modem/0' removed
Jul 07 17:14:33 pinephone eg25manager[3588]: Lost modem, resetting...
Jul 07 17:14:33 pinephone eg25manager[3588]: Couldn't unbind modem: wrote -1/3 bytes
Jul 07 17:14:33 pinephone eg25manager[3588]: Couldn't bind modem: wrote -1/3 bytes
Jul 07 17:14:39 pinephone eg25manager[3588]: ModemManager object `/org/freedesktop/ModemManager1/Modem/1' added
...
Jul 07 19:55:06 pinephone eg25manager[3588]: Lost modem, resetting...
Jul 07 19:55:06 pinephone eg25manager[3588]: Couldn't unbind modem: wrote -1/3 bytes
Jul 07 19:55:06 pinephone eg25manager[3588]: Couldn't bind modem: wrote -1/3 bytes
Jul 07 19:55:06 pinephone eg25manager[3588]: ModemManager object `/org/freedesktop/ModemManager1/Modem/1' removed
Jul 07 19:55:12 pinephone eg25manager[3588]: ModemManager object `/org/freedesktop/ModemManager1/Modem/2' added

Looks like ModemManager knows how to handle the reconnecting modem after all... But it will not continue resetting the modem on every failure. At some point, it will not add another modem.

Jul 08 05:24:44 pinephone eg25manager[3588]: Lost modem, resetting...
Jul 08 05:24:44 pinephone eg25manager[3588]: Couldn't unbind modem: wrote -1/3 bytes
Jul 08 05:24:44 pinephone eg25manager[3588]: Couldn't bind modem: wrote -1/3 bytes
Jul 08 05:24:44 pinephone eg25manager[3588]: ModemManager object `/org/freedesktop/ModemManager1/Modem/6' removed

No further output (into the journal) happens. And this is when we see the modem never come back from vanishing (and wwan, telephony being completely unavailable). So, the modem is disappearing more often than we think it is, but ModemManager and eg25-manager seem to be recovering a lot.

However, when we restart, we see the following output:

Jul 08 10:13:34 pinephone eg25manager[3588]: Request to quit...
Jul 08 10:13:34 pinephone systemd[1]: Stopping Quectel EG25 modem...
Jul 08 10:13:34 pinephone eg25manager[3588]: dropping systemd sleep delay inhibitor
Jul 08 10:13:34 pinephone eg25manager[3588]: Powering down the modem...
Jul 08 10:13:35 pinephone eg25manager[3588]: Executed power-on/off sequence
Jul 08 10:13:35 pinephone eg25manager[3588]: Executed power-off sequence
Jul 08 10:13:37 pinephone eg25manager[3588]: Modem down, quitting...
Jul 08 10:13:38 pinephone systemd[1]: eg25-manager.service: Deactivated successfully.
Jul 08 10:13:38 pinephone systemd[1]: Stopped Quectel EG25 modem.
...
Jul 08 10:13:38 pinephone systemd[1]: Started Quectel EG25 modem.
...
Jul 08 10:14:00 pinephone eg25manager[27082]: ModemManager object `/org/freedesktop/ModemManager1/Modem/7' added

At that point, the modem comes back and becomes usable again, as indicated by the cellular icon in the (in my case) Phosh statusbar.
This means that we are actually encountering two issues here:

• The modem vanishing in the first place
• It not being properly reset after having vanished a number of times

[1001111github]

I have had the same problem with the modem disappearing for the past month. I also had phosh login screen control issues. I religiously pacman -Syu daily. As of two days ago the problems stopped occurring. Other than the pacman updates the only thing I changed was my power options in Settings.

Previous settings were 10 minutes display, 15 minute suspend on battery. No suspend on power.
New settings were 10 minutes display, 15 minute suspend on battery. 1 hour suspend on power.

For two days the phone has been 100% reliable. No modem drops, no login screen issues
YMMV
Dave (aka lf)

[kescherCode]

I flashed the 01.002.07 modem firmware (which still shows up as 01.002.01.002, go figure). Apparently, that breaks GPS, but my modem has never gone away since. Same as Dave before me: YMMV.

[Danct12]

According to biktor, the only single /dev/ttyUSB device showing up after losing cellular is the modem crashed to EDL/Sahara mode.

Sometimes, the modem successfully rebooted back to the OS and work again, then crashed sometime later.

There's also reports on Pine64 forum as well: https://forum.pine64.org/showthread.php?tid=13885

Mobian: https://gitlab.com/mobian1/issues/-/issues/251

[kescherCode]

I would like to add that the permanent vanishing seems to occur with modem firmwares 01.002.01.002 and 01.003.01.003, but I've flashed 01.002.07 (which still reports as 01.002.01.002 for some reason), and ever since, the modem still vanishes occasionally ("Lost modem, resetting..."), but always comes back within 10 seconds by itself, and never disappears permanently. Also, the vanishing occurs much less frequently.

(I don't suggest flashing that modem firmware though, it seems to be broken in a number of other ways)

[Jab2870]

@kescherCode could you elaborate on some of the other ways the 01.002.07 version of the modem is broken?

[kescherCode]

@Jab2870 well, AT+QGPS=1 returning +CME ERROR=504, for instance. And its version string is still returning 01.002.01.002.

Allegedly, there are more things that are broken in comparison to 01.002.01.002, but I don't really notice as the PinePhone isn't my daily driver.