Lenovo x230t : BackSpace keycode (c:160) is mapped to XF86ScreenSaver, not BackSpace

[tlaurion]

Qubes OS version:

R3.2, R4.0rc1, R4.0rc2

Affected TemplateVMs:

dom0 fedora-23, dom0 fedora-25

---

Steps to reproduce the behavior:

Install Qubes 3.2. At login prompt, backspace doesn't do its backspace function anymore.

Expected behavior:

Under x230t (not true for x230!?) doing sudo xev key shows that it's mapped to BackSpace

Actual behavior:

sudo xev and typing the backspace key shows that it's linked to XF86ScreenSaver, not BackSpace

General notes:

Corrective action is to add the following line in the file /etc/X11/Xmodmap:
keycode 160 = BackSpace NoSymbol BackSpace

---

Related issues:

[tlaurion]

Doesn't resolve the issue after reboot. Any hint?

[marmarek]

I guess /etc/X11/Xmodmap isn't loaded by lightdm...

[marmarek]

Does it work correctly on plain Fedora 25 (or newer)?

[tlaurion]

Overwriting /etc/X11/Xmodmap with the following file makes the change persistent over 4.0 rc3, effective from dom0 and launched qubes. Didn't verify for 3.2.

xmodmap_x230t.txt

[tlaurion]

My x230t is dead. Cannot test this but for anyone else with the same kind of problem:

forums.puri.sm/t/keyboard-layout-unable-to-recognize-pipe/2022

Reference

Anyone with a x230t that could give the content of /sys/class/dmi/id/modalias ? I suspect it should differenciate x230t of x230. If it is, it would be possible to deploy a permanent fix.

[tlaurion]

X230t resuscitated.

modalias content:
dmi:bvncoreboot:bvrCBET4000heads:bd01/01/1970:svnLENOVO:pn343727U:pvrThinkPadX230Tablet:rvnLENOVO:rn343727U:rvrThinkPadX230Tablet:cvnLENOVO:ct9:cvr:

[andrewdavidwong]

This issue is being closed because:

• This issue has the "Release 3.2 updates" milestone.
• Qubes OS 3.2 has reached end-of-life (EOL).
• This issue has not been updated in over a year.

If anyone believes that this issue should be reopened, please let us know in a comment here.

[matthewspangler]

I can confirm I am having this issue on the following setup:

Thinkpad X230T
Qubes OS 4.1.2
Heads-x230-hotp-maximized-v0.2.0-1577-g91f65be

• https://app.circleci.com/pipelines/github/osresearch/heads/612/workflows/87b50d50-6afd-46bc-9130-7475bb9fc1ee/jobs/10337/artifacts

Both the 'Backspace' and | \ keys are unresponsive.

I have tried the following fix detailed in these two links:

• keyboard backspace bug for x230-hotp-maximized (x230t) linuxboot/heads#958 (comment)
• systemd/systemd@3bbdb32#diff-df937b870825d805803c76ae8258342dcd6850770f4404721e044af93ac3d231R963

Here are the steps for this fix:

1. Edit /usr/lib/udev/hwdb.d/60-keyboard.hwdb in dom0
2. Find the line containing: Thinkpad*X2*Tablet* (for me it was around line 963)
3. Add :rvn* to the end of the line, like so: evdev:atkbd:dmi:bvn*:bvr*:bd*:svnLENOVO*:pn*:pvrThinkPad*X2*T*:rvn*

Afterwards both keys are working, but occasionally during daily usage I'll find that the issue pops up again. I'm still trying to replicate the conditions where the fix regresses.

EDIT 11/16/23: I doubt the above fix actually worked, it may have just been because I was warm booting and it appeared to resolve.

[matthewspangler]

I've figured out why this bug sometimes appears for me, and sometimes doesn't.

Bug appears when:

1. Do a cold boot. Unplug battery, unplug power cable, hold power button to drain power -- then boot.
2. Type LUKS password -- at this point, backspace and "| " work in the LUKS text box
3. User login -- at this point, backspace and "| " have stopped working, and continue to not work after login.

Bug doesn't appear when:

1. Do warm boot by rebooting from Qubes OS.
2. Keys work fine @ LUKS prompt + user login + desktop.

This is talked about in this Heads issue: linuxboot/heads#1525 (comment)

The issue seems to be related to i8042.c in the linux kernel per linuxboot/heads#958 (comment)

[toothlesslizard]

Same on Qubes 4.2 and x230 from xytech (keyboard from x220).
I have Ubuntu 23.10 on second disk drive (nvme mod + ssd) and there is no issues.

Qubes 4.2.0-rc4
xen 4.17.2
Kernel 6.5.8-1
dom0: fedora-37
FW_VER - Heads-v0.2.0-1796-g35f3b22-dirty
Kernel: Linux 5.10.5 - heads
X230-maximized-eDP

[tlaurion]

Somebody reported that using q4.2 changed the behavior. Went extensively with the user and this happens only on cold boot, where reboot detect the keyboard correctly. Curated the logs and opened a PR under heads at linuxboot/heads#1525 which contains his logs. The bug is that on cold boot, i8042 probes the keyboard and detects it as Raw translated 2, where on reboot it's detected correctly as AT translated 2. Unfortunately I do not understand what is happening there. But the logs have i8042 in debug mode with kernel option to not timeout which is the best I could come up to gather logs from the user. No clue what is happening there and cannot replicate on my x230. This is not a x230t here. Keyboard sku still unknown.

…

On Thu, Nov 16, 2023, 3:28 PM toothlesslizard ***@***.***> wrote: Same on Qubes 4.2 and x230 from xytech (keyboard from x220). backspace, /, | this keys work intermittently I have Ubuntu on second disk drive and there is no troubles. Qubes 4.2 FW_VER - Heads-v0.2.0-1796-g35f3b22-dirty Kernel: Linux 5.10.5 - heads X230-maximized-eDP — Reply to this email directly, view it on GitHub <#3306 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGKBMURLFFFERSHWQB5IBDYEZZQRAVCNFSM4EDQQP4KU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRGUZDMNBWGE3Q> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[toothlesslizard]

@tlaurion do you need dmesg +sku? this bug not only 230t related.
Try x230 + 45n2141 (for example) (maybe ec-firmware? but other distros works flawlessly)

[matthewspangler]

I'm ignorant as to a lot of this, but I'm going to take a wild guess--what about kernel option "i8042.probe_defer" in kernels >=6.7? It's related to integrated PS/2 keyboards not being ready at cold boot because they're busy performing a self-test.

torvalds/linux@9222ba6
https://bugs.launchpad.net/ubuntu/+source/linux-source-2.6.17/+bug/76881

[tlaurion]

Analysis of debug logs at linuxboot/heads#1525 (comment)

[tlaurion]

EDIT nevermind, past comment misleading.

[tlaurion]

I'm ignorant as to a lot of this, but I'm going to take a wild guess--what about kernel option "i8042.probe_defer" in kernels >=6.7? It's related to integrated PS/2 keyboards not being ready at cold boot because they're busy performing a self-test.

torvalds/linux@9222ba6
https://bugs.launchpad.net/ubuntu/+source/linux-source-2.6.17/+bug/76881

I think this would happen too late. My reasoning to push for a detection fix or hardcoding on coreboot side is that Linux should do the right thing if coreboot result was good where Heads Linux just uses the keyboard as detected without udev fixes. If detection was stable there, workarounds would not be needed later on. At least that's my reasoning which could be faulty.

After that, os apply or not their udev fix, as seen in this thread. If os applies udev fix whatever raw/at then behavior might still be unexpected unless i8042 initiates a reset on controller.

[tlaurion]

I think I fixed it with ugly workaround, disabling exits on pc80 code of coreboot. Needs testing to raise attention from coreboot developers.

Should work for x220t/x230t/x230.

See linuxboot/heads#1525 (comment)

Edit: it "worked".
From Heads recovery shell with that rom tested, coldboot:

But coreboot says that pc80 driver for ps2 is not used and that it adds delay and that the payload should handle with ps2 init and keyboard proper init.

Meaning we have to find the proper i8042 kernel parameters to fixate At translated 2 when coreboot boots Heads first kernel.

That means disabling ps2 support from coreboot and having coreboot config pass to the kernel the right i8042 drivers parameters to permanently fix the issue.

Someone having misbehaving hardware can play around and propose a PR based on the commits of testing PR?

[tlaurion]

Looking at i8042 source, it seems that passing from coreboot config to heads kernel : CONFIG_LINUX_COMMAND_LINE="i8042.force=1:kbd" should actually be enough

(or force this in grub config for easier qubesos testing ).

Anyone can test and report back?

[toothlesslizard]

strange, problem persists.
added these lines and rebuild rom. lightdm greeter won't respond to the backspace key.
or iam doing something wrong?

user@heads:~/heads/config$ cat coreboot-x230-maximized-fhd_edp.config | grep 8042
CONFIG_LINUX_COMMAND_LINE="intel_iommu=igfx_off i8042.force=1:kbd quiet"

Qubes grub no effect too.

[tlaurion]

strange, problem persists.
added these lines and rebuild rom. lightdm greeter won't respond to the backspace key.
or iam doing something wrong?

user@heads:~/heads/config$ cat coreboot-x230-maximized-fhd_edp.config | grep 8042
CONFIG_LINUX_COMMAND_LINE="intel_iommu=igfx_off i8042.force=1:kbd quiet"

Qubes grub no effect too.

Well. That doesn't seem enough.
Can you confirm from dmesg output of
'sudo dmesg | grep -i keyb` under heads recovery shell and final os and
'cbmem -1 | grep -I keyb' under heads recovery shell please.

[tlaurion]

@toothlesslizard also if you could deactivate CONFIG_DRIVERS_PS2_KEYBOARD from coreboot config that would make a consistent test without coreboot ps2 init being in the way whatsoever.

behavior should be different between cold boot (power off, holding power 10 seconds) and warm boot (reset, reboot) as for everybody else.

[toothlesslizard]

@tlaurion I'm getting different outputs from the shell recovery (on/off laptop) :

[          0.828917] input : AT Translated Set 2 keyaboard as /devices/platform/i8042/serio0/input/input3

[DEBUG] Keyboard init ...
[ INFO ] Keyboard controller output buffer result timeout
[DEBUG] PS/2 Keyabord initialized on primary channel 

or this

[     0.824340] input : AT RAW Set 2 keyboard as /devices/platform/i8042/serio0/input/input3
[     1.692471] input : AT RAW Set 2 keyboard as /devices/platform/i8042/serio0/input/input6

[DEBUG] Keyboard init ...
[ INFO ] Keyboard controller output buffer result timeout
[ INFO ] Keyboard controller output buffer result timeout
[ INFO ] Keyboard controller output buffer result timeout
[ERROR] Keyboard reset failed ASK: 0xaa

rebuilding the rom from scratch for now again.

# CONFIG_DRIVERS_PS2_KEYBOARD is not set

[toothlesslizard]

full output from Qubes OS. cant filtering cause keys not working.
full-dmesg.txt

new rom from scratch
CBET4000 Heads-v0.2.0-1914-g1f39d16-dirty
stock bios from here : https://github.com/xy-tech/x330-bios/tree/main/stock/original

user@heads:~/heads/config$ cat coreboot-x230-maximized-fhd_edp.config | grep -i "8042"
CONFIG_LINUX_COMMAND_LINE="intel_iommu=igfx_off i8042.force=1:kbd quiet loglevel=2"
user@heads:~/heads/config$ cat coreboot-x230-maximized-fhd_edp.config | grep -i "KEYBO"
# CONFIG_DRIVERS_PS2_KEYBOARD is not set

cbmem | grep keyb or Keyboa - no outut now.
other things about AT Translated / AT RAW same. See prev post.

If anyone interesting i have different sku keyboards. But swapping not help.

Boot from second drive and output from Ubuntu (keys works good)

toothless@ubuntu:~$ sudo dmesg | grep -i Keyb
[sudo] password for toothless: 
[    0.387853] input: AT Raw Set 2 keyboard as /devices/platform/i8042/serio0/input/input3
[   21.529352] systemd[1]: Starting keyboard-setup.service - Set the console keyboard layout...

[tlaurion]

new rom from scratch\nCBET4000 Heads-v0.2.0-1914-g1f39d16-dirty\nstock bios from here : https://github.com/xy-tech/x330-bios/tree/main/stock/original\n\nuser@heads:~/heads/config$ cat coreboot-x230-maximized-fhd_edp.config | grep -i "8042"\nCONFIG_LINUX_COMMAND_LINE="intel_iommu=igfx_off i8042.force=1:kbd quiet loglevel=2"\nuser@heads:~/heads/config$ cat coreboot-x230-maximized-fhd_edp.config | grep -i "KEYBO"\n# CONFIG_DRIVERS_PS2_KEYBOARD is not set\ncbmem | grep keyb or Keyboa - no outut now.\nother things about AT Translated / AT RAW same. See prev post.\n\nIf anyone interesting i have different sku keyboards. But swapping not help.

I'm sorry @toothlesslizard i didn't get that part.

Are you saying that on stock bios (x330 referenced) you are getting the same coldboot=raw, reboot/warm reset=at?

[toothlesslizard]

No, I just added details about the firmware build, nothing more.
Its about blobs in xx30 folder.
The point is, the keys continue to malfunction.
Only cold boot. Recovey shell and LUKS - good, LightDM Greeter - keys fall off.

[tlaurion]

Boot from second drive and output from Ubuntu (keys works good)

The Raw Translated 2 being dealt with Ubuntu is nice (systemd workarounds?) but is what differenciate quebesos from others here (where some could apply fixes to qubesos, but those won't stick if behavior is different between coldboot and warm boot and is not fixated across them)

Raw Translated 2 should not happen. We want AT Translated 2, otherwise we have different behaviors across OSes because of workarounds applied differently across OSes. If AT translated 2: no problem across all OSes.

The question is how to arrive to that. With hacks on coreboot disabling exit 0 on patch referred above, we arrive to that state but that is really ugly.

So this is where we are here. No real progress knowing which kernel parameters need to be set so that if firmware doesn't get in the way, we get to AT translated 2, otherwise OSes have to apply quirks of their own. Which won't work if ps2 protocol switches between AT translated 2 to Raw Translated 2 depending if we are cold booting or warm booting. I hope we are now clear on what the problem is if we haven't found proper solution... Yet.

[tlaurion]

Opened issue https://ticket.coreboot.org/issues/515