Lenovo P72 Running Ubuntu 18.10

[gstanden]

Hi I followed your instructions after reading your article here and the install went fine on Ubuntu 18.10 running on a Lenovo P72 with no issues. The IR emitters came alive on the 3rd of the

"Did your IR emitters turn on? [y/N]:"

tests. However, when I run "sudo howdy add" the IR camera blinks quickly just once and then howdy gives message after about a minute

"No face detected, aborting".

Any suggestions how I can tweak/fix this? Thanks!

[gstanden]

So I made some progress I think on this myself. I ran a "sudo howdy test" and noted that the window that opened was blank with no image at all (no images, just the box with no content at all). The error that was generated over and over was the same as the one at this issue in another GitHub project. So I followed the recipe there and added "QT_X11_NO_MITSHM=1" to /etc/environment and this seems to have "helped", at least it caused "sudo howdy test" to start working with no errors in the terminal and a VERY grainy almost now totally greyed out image appearing in the pop-up GUI box when running "sudo howdy test" but within a second or two the grained out foggy face just flips to almost a solid wall of grey and when running "sudo howdy add" it just hangs now, never gives the "no face detected, aborting" message. One other side effect of adding this parameter to /etc/environment is that now the 4th test (and not the 3rd as before) for "did you IR emitter flash Y/N" is the one that triggers the IR emitters to flash. Also, the IR emitter flashes only briefly once after running "sudo howdy add" and accepting the default name for the first record, and then there are no flashes while howdy hangs.

Trying "Integrated IR Camera: Integrate"
One of your cameras should now be on.
Did your IR emitters turn on? [y/N]: N
Interpreting as a "NO"

Trying "Integrated Camera: Integrated C"
One of your cameras should now be on.
Did your IR emitters turn on? [y/N]: N
Interpreting as a "NO"

Trying "Integrated Camera: Integrated C" <-- before /etc/environment parameter trigger was here
One of your cameras should now be on.
Did your IR emitters turn on? [y/N]: N
Interpreting as a "NO"

Trying "Integrated IR Camera: Integrate" <-- after adding /etc/environment parameter trigger is here
One of your cameras should now be on.
Did your IR emitters turn on? [y/N]: y

[gstanden]

I also tried installing "guvcview" as described here in issue #91 but this did not enable "sudo howdy add" to succeed. However, I did do some grep of dmesg and here is some output I found which looks like it might be relevant, but I'm guessing these were the IR camera tests that did not trigger the IR emitters and are expected output for non-triggering tests. Also, the emitter tested moved back to test 3 (it had moved to test 4) so that doesn't seem to be very relevant either.

ubuntu@ubuntu-ThinkPad-P72: sudo dmesg | grep "video"
[ 3.473013] videodev: Linux video capture interface: v2.00
[ 3.504020] uvcvideo: Found UVC 1.00 device Integrated Camera (04f2:b604)
[ 3.552522] uvcvideo 1-8:1.0: Entity type for entity Realtek Extended Controls Unit was not initialized!
[ 3.552524] uvcvideo 1-8:1.0: Entity type for entity Extension 4 was not initialized!
[ 3.552525] uvcvideo 1-8:1.0: Entity type for entity Processing 2 was not initialized!
[ 3.552526] uvcvideo 1-8:1.0: Entity type for entity Camera 1 was not initialized!
[ 3.554931] uvcvideo: Found UVC 1.50 device Integrated IR Camera (04f2:b605)
[ 3.558858] uvcvideo 1-12:1.0: Entity type for entity Microsoft Extended Controls Uni was not initialized!
[ 3.558859] uvcvideo 1-12:1.0: Entity type for entity Realtek Extended Controls Unit was not initialized!
[ 3.558861] uvcvideo 1-12:1.0: Entity type for entity Extension 4 was not initialized!
[ 3.558862] uvcvideo 1-12:1.0: Entity type for entity Processing 2 was not initialized!
[ 3.558863] uvcvideo 1-12:1.0: Entity type for entity Camera 1 was not initialized!
[ 3.560328] usbcore: registered new interface driver uvcvideo
ubuntu@ubuntu-ThinkPad-P72:~

Here's some other dmesg output from a grep on "Camera":

ubuntu@ubuntu-ThinkPad-P72:~ dmesg | grep Camera
[ 1.968892] usb 1-8: Product: Integrated Camera
[ 2.552299] usb 1-12: Product: Integrated IR Camera
[ 3.495867] uvcvideo: Found UVC 1.00 device Integrated Camera (04f2:b604)
[ 3.511519] uvcvideo 1-8:1.0: Entity type for entity Camera 1 was not initialized!
[ 3.511650] input: Integrated Camera: Integrated C as /devices/pci0000:00/0000:00:14.0/usb1/1-8/1-8:1.0/input/input13
[ 3.520956] uvcvideo: Found UVC 1.50 device Integrated IR Camera (04f2:b605)
[ 3.522799] uvcvideo 1-12:1.0: Entity type for entity Camera 1 was not initialized!
[ 3.522854] input: Integrated IR Camera: Integrate as /devices/pci0000:00/0000:00:14.0/usb1/1-12/1-12:1.0/input/input14
ubuntu@ubuntu-ThinkPad-P72:~$

So this is where I'm at now. I don't get any errors when I run "sudo howdy test" and I don't get any errors when I run "sudo howdy add" and it doesn't hang when running "sudo howdy add" anymore, but it aborts after a bit and there are no flashes from the IR emitters:

ubuntu@ubuntu-ThinkPad-P72:~ sudo howdy test
No face model known
[sudo] password for ubuntu:

Opening a window with a test feed

Press ctrl+C in this terminal to quit
Click on the image to enable or disable slow mode

^C
Closing window
ubuntu@ubuntu-ThinkPad-P72:~ sudo howdy add
Adding face model for the user ubuntu
Enter a label for this new model [Initial model] (max 24 characters):

Please look straight into the camera
No face detected, aborting
ubuntu@ubuntu-ThinkPad-P72:~$

[gstanden]

Here's what I get from "tail -f" of the auth.log (the entries at 9:00:56 are the entries that popped in when I ran "sudo howdy add").

Jan 27 08:55:49 ubuntu-ThinkPad-P72 sudo: pam_unix(sudo:session): session closed for user root
Jan 27 09:00:56 ubuntu-ThinkPad-P72 sudo: ubuntu : TTY=pts/0 ; PWD=/home/ubuntu ; USER=root ; COMMAND=/usr/local/bin/howdy add
Jan 27 09:00:56 ubuntu-ThinkPad-P72 sudo: pam_unix(sudo:session): session opened for user root by (uid=0)
Jan 27 09:01:08 ubuntu-ThinkPad-P72 sudo: pam_unix(sudo:session): session closed for user root

[boltgolt]

Thanks for really trying to solve this issue on own! Sounds like you might be running in the wrong resolution, though. If you run v4l2-ctl --list-formats-ext you'll get a list of supported resolutions for all your cameras. Try using the ones listed for your IR camera as frame_width and frame_height in sudo howdy config.

[gstanden]

Sorry total newb to IR emitters / IR cameras etc. Which one is my IR ? What do you recommend setting in howdy config ? Also, fyi I had to run "sudo apt install v4l-utils" for this command (it wasn't installed so I installed it).

Here's the current settings from "sudo howdy config" for "frame_width" and "frame_height"

-- Set the camera input profile to this width and height
-- The largest profile will be used if set to -1
-- Automatically ignored if not a valid profile
frame_width = -1
frame_height = -1

And here's the output from the command you provided - thanks! Which one is the IR camera ?

I tried setting width 640 and height 480 (and vice versa) in howdy config, but I still just get a single fast blink of the IR emitters when running "sudo howdy add" and then after about 10 seconds or so it times out with:

ubuntu@ubuntu-ThinkPad-P72:~$ sudo howdy add
Adding face model for the user ubuntu
Enter a label for this new model [Initial model] (max 24 characters):

Please look straight into the camera
No face detected, aborting
ubuntu@ubuntu-ThinkPad-P72:~$

ubuntu@ubuntu-ThinkPad-P72:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'MJPG' (compressed)
Name : Motion-JPEG
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 320x180
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 352x288
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 424x240
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x360
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 848x480
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 960x540
Interval: Discrete 0.033s (30.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: 'YUYV'
Name        : YUYV 4:2:2
	Size: Discrete 1280x720
		Interval: Discrete 0.100s (10.000 fps)
	Size: Discrete 320x180
		Interval: Discrete 0.033s (30.000 fps)
	Size: Discrete 320x240
		Interval: Discrete 0.033s (30.000 fps)
	Size: Discrete 352x288
		Interval: Discrete 0.033s (30.000 fps)
	Size: Discrete 424x240
		Interval: Discrete 0.033s (30.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.033s (30.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.033s (30.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.050s (20.000 fps)
	Size: Discrete 960x540
		Interval: Discrete 0.067s (15.000 fps)

ubuntu@ubuntu-ThinkPad-P72:~$

[gstanden]

Hey! I changed it to 320 x 180 and the emitter flashed for a about 3 to 5 seconds (multiple flashes) and not just a single flash, and it captured the image!! Testing it now ! I did not get the "aborting" message this time ... looks like it worked!!

I ran a "sudo reboot" and it logged me in automatically after scanning my face with the IR emitters. Howdy worked very quickly there was no lag it was very similar user experience to Windows Hello. However, once the desktop loaded, I did not yet have access to the desktop, and I got this popup and had to type in my password in the usual way.

I tried also doing a complete "shutdown -h now" and then doing a clean startup, and I always get this popup about the keyring after Howdy recognizes me and logs me into the desktop. Any way to fix this so that they keyring reads from Howdy as well ?

[gstanden]

There's another problem. Howdy works fine after I first configure it "for awhile." But if I go away from the machine for say 20 minutes or so, when I come back, the IR isn't able to make the recognition match, and I get the following message (for example from a terminal sudo session attempt) and then I have to just type in the password as usual (which Ubuntu prompts for when the face detection timeout is reached).

ubuntu@ubuntu-ThinkPad-P72:~ sudo su -
Face detection timeout reached
[sudo] password for ubuntu:
root@ubuntu-ThinkPad-P72:~#

I can fix this by doing a "sudo howdy clear" and then doing a new "sudo howdy add" and then it works fine until I go away from the machine again for awhile, and then apparently for lack of a better term, the howdy/IR setup seems to "forget" my face and I have to go through the clear/add cycle again.

[boltgolt]

So your keyring popup problem has been discussed in #39. It's not solvable without setting an empty keyring password.

The timeout is by design. You could up the timeout value in the config, but i wouldn't advice you to do so. While howdy is running you can't enter a password, so when IR isn't working due to things like sunlight, you can't log in until timeout.

[gstanden]

Agreed on the keyring popup. However, it's not the timeout that is a problem - it's the "forgetting" of the face recognition after say, 20 minutes. What I'm trying now is different resolution settings in howdy config to see if that makes any difference. For example, 424 x 280 also works for the Lenovo P72 (I was using 320 x 180) so perhaps when I find the highest resolution that will work, howdy will "remember" the IR credential image "permanently" ?

[boltgolt]

Hm, might also be a confidence problem. Try adding an additional model when Howdy does not recognize you (instead of clearing) and maybe up the certainty value to 5 in the config. A higher resolution will also help if that's the case.

[gstanden]

Thanks, it's working good on Lenovo P72 mobile workstation running Ubuntu 18.10 desktop edition now no issues and it never "forgets" anymore it remembers and this is with the following non-default settings in config:

certainty = 5
frame_width = 320
frame_height = 180

In addition, just to summarize, for this Lenovo P72 Ubuntu 18.10 systems I also did the following:

sudo apt install v4l-utils
added "QT_X11_NO_MITSHM=1" to /etc/environment

I did record two images named "ubuntu1" and "ubuntu2" as you suggested as well:

ubuntu@ubuntu-ThinkPad-P72:~$ sudo howdy list
Known face models for ubuntu:

ID  Date                 Label
0   2019-01-29 09:57:45  ubuntu1
1   2019-01-29 09:57:58  ubuntu2

ubuntu@ubuntu-ThinkPad-P72:~$

but I think it's authenticating from ubuntu1 reliably (only takes a second or two and done) and the "forgetting" problem is gone now that I'm using "certainty = 5".

Thanks for your timely replies and great help with this !!

[gstanden]

Hi just one footnote fwiw. It's all working great no "forgetting" anymore, except for one scenario, which is when the screen timeout occurs, and the desktop locks. When I go to unlock the screen, hit enter, the IR emitters flash in the normal way for a few seconds, but the face read fails sometimes in just this scenario (unlock) only. This doesn't happen on reboots, and it doesn't happen on terminal sudo sessions once the desktop is unlocked - both of those scenarios work perfect first time everytime. But screen unlock isn't reading the face authentication and is failing. Any ideas with that one? Thanks!

[boltgolt]

Sounds like one of the most reported problems, could you try setting no_confirmation to true? That sometimes throws a PamException: Conversation error with some screensavers,

[gstanden]

Ok thanks! I made the change to:

no_confirmation = true

I'll run it for a few days and see how it goes. Similar to issue #140 my lock screen interaction with howdy is also unpredictable and the failure to IR authenticate happens intermittently, while other times it works fine, which is also similar to #140

What does that setting do ? Does it make the authentication less secure ?

[boltgolt]

It simply prevents sending the "Identified face as [user]" message, which crashes some screensavers.

[gstanden]

Just a footnote to using this with the Lenovo P72 mobile workstation. The P72 mouse handling and password screen handling at the GDM login screen is kind of herky jerky uneven when logging in to Ubuntu 18.10 on this machine (and this is a brand-new unit just delivered and currently being configured for dual-NVME Ubuntu 18.10/Windows 10 multiple-display configuration). Therefore, the configuration I am using with Howdy is VERY nice for handling login to the Ubuntu 18.10 desktop: Howdy logs me in automatically at GDM login, and then the desktop now loaded presents a popup asking for the keyring password, which as noted above in this thread, has to be typed in. This is a huge improvement on the login process in general for Ubuntu 18.10 boot/restart login on the Lenovo P72. It gets you automatically past the quirky performance of GDM login page reliably and quickly, but still preserves a level of desktop login security. Thanks!