Meta Portal Devices

[janstadt]

Facebook recently pushed updates to the entire line of portal devices and it would be awesome to get Ava working on these. I was able to install it, but it cant get past certain permissions (Device Management Policies are not supported for this app). I also found these posts and have been able to get Developer Mode enabled but still havent made much progress getting Ava working:
https://www.reddit.com/r/FacebookPortal/comments/1twyoqz/all_portal_devices_officially_unlocked/
https://www.reddit.com/r/FacebookPortal/comments/1tx84hn/oem_unlocking_found_possible_rootcustom_rom/
https://www.reddit.com/r/FacebookPortal/comments/1tynmqw/immortal_a_free_home_screen_photo_frame_and_app/

[knoop7]

I'm working closely on the latest version. Maybe you can wait a little before you can update and experience the new effects. I'll try to fix this issue while I'm at it. You can try mod store support?

[knoop7]

Grant the normal runtime permissions:

adb shell pm grant com.example.ava android.permission.RECORD_AUDIO
adb shell pm grant com.example.ava android.permission.CAMERA
adb shell pm grant com.example.ava android.permission.ACCESS_FINE_LOCATION
adb shell pm grant com.example.ava android.permission.ACCESS_COARSE_LOCATION
adb shell pm grant com.example.ava android.permission.BLUETOOTH_SCAN
adb shell pm grant com.example.ava android.permission.BLUETOOTH_CONNECT
adb shell pm grant com.example.ava android.permission.BLUETOOTH_ADVERTISE
adb shell pm grant com.example.ava android.permission.POST_NOTIFICATIONS

Grant the special app-ops Ava needs:

adb shell appops set com.example.ava SYSTEM_ALERT_WINDOW allow
adb shell appops set com.example.ava WRITE_SETTINGS allow
adb shell pm grant com.example.ava android.permission.WRITE_SECURE_SETTINGS
adb shell appops set com.example.ava REQUEST_INSTALL_PACKAGES allow

Set Ava as the home launcher:

adb shell cmd package set-home-activity com.example.ava/.MainActivity
adb shell input keyevent KEYCODE_HOME

Optional Portal system tweaks, based on what the Immortal project is doing:

adb shell settings put global development_settings_enabled 1
adb shell settings put global policy_control immersive.status=*
adb shell settings put secure ui_night_mode 2
adb shell settings put global hidden_api_policy_pre_p_apps 1
adb shell settings put global hidden_api_policy_p_apps 1
adb shell settings put global hidden_api_policy 1

adb shell cmd overlay list | grep aloha.rro
adb shell cmd overlay disable --user 0 com.facebook.aloha.rro.niu.android
adb shell settings put global package_verifier_enable 0
adb shell pm disable-user --user 0 com.facebook.appverifier

Expected result: Ava should launch and run with microphone, overlay, camera, Bluetooth, and launcher permissions handled. The one thing that may still fail is Device Admin-based screen lock/screen-off, because Portal firmware appears to block that policy for third-party apps.

[janstadt]

Awesome. I'll report back here when i can get a chance to do this.

[janstadt]

All these failed with Unknown Permission:
adb shell pm grant com.example.ava android.permission.BLUETOOTH_SCAN
adb shell pm grant com.example.ava android.permission.BLUETOOTH_CONNECT
adb shell pm grant com.example.ava android.permission.BLUETOOTH_ADVERTISE
adb shell pm grant com.example.ava android.permission.POST_NOTIFICATIONS

Im still unable to get passed the Activate device admin app for Locking the screen it would seem. I'll keep trying things and report back if anything works.

[knoop7]

https://github.com/knoop7/Ava/releases/tag/0.4.8

[janstadt]

A bit more progress after the 0.4.8 upgrade. Im able to get the app running and discoverable in HA. Everything seems to be working except the microphone unfortunately. Its not detecting any wake words. I've tried both vsWakeWord and microWakeWord but it doesnt seem like the microphone is exposed to the app. I did install shizuku to get semi root so if you have any ideas or ways to debug lemme know and i can test some stuff out.

[knoop7]

You can check HomeAssistant. If you need to select wake word 1 or wake word 2 for two-way selection, it might be due to microphone permissions or encoding issues. You can directly copy logs and send them to me at the bottom of the settings page. You can pull logs up and send them to me under settings. If there are other issues, they might still be in a microphone selection channel. You can adjust them in voice settings and try them out

[janstadt]

A bit of progress, microWakeWord is working, but not vs wakeword. I do wonder though that if the device has to pull down models, that there might be a delay from when i toggle between the wake word processors or something which is why it wasnt picking up the wake word? I can continue to test that out. Here are some logs i am seeing in HA around the device and esphome:

	•	Can't connect to ESPHome API for esp32-s3-box-3-5a94f4 @ 192.168.1.200: Error connecting to [AddrInfo(family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, proto=6, sockaddr=IPv4Sockaddr(address='192.168.1.200', port=6053))]: [Errno 113] Connect call failed ('192.168.1.200', 6053) (SocketAPIError)
	•	Can't connect to ESPHome API for SKY-2216_A6D1_voice_assistant @ 192.168.1.220: Error connecting to [AddrInfo(family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, proto=6, sockaddr=IPv4Sockaddr(address='192.168.1.220', port=6272))]: [Errno 113] Connect call failed ('192.168.1.220', 6272) (SocketAPIError)
	•	Can't connect to ESPHome API for Kitchen Assistant @ 192.168.1.246: Error connecting to [AddrInfo(family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, proto=6, sockaddr=IPv4Sockaddr(address='192.168.1.246', port=6513))]: [Errno 111] Connect call failed ('192.168.1.246', 6513) (SocketAPIError)
	•	Can't connect to ESPHome API for Portal_2ED4_voice_assistant @ 192.168.1.223: Portal_2ED4_voice_assistant @ 192.168.1.223: EOF received (SocketClosedAPIError)
	•	Can't connect to ESPHome API for Portal_2ED4_voice_assistant @ 192.168.1.223: Error connecting to [AddrInfo(family=<AddressFamily.AF_INET: 2>, type=<SocketKind.SOCK_STREAM: 1>, proto=6, sockaddr=IPv4Sockaddr(address='192.168.1.223', port=6379))]: [Errno 111] Connect call failed ('192.168.1.223', 6379) (SocketAPIError)


Error unloading entry Portal_2ED4_voice_assistant for assist_satellite
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1024, in async_unload
    result = await component.async_unload_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/assist_satellite/__init__.py", line 198, in async_unload_entry
    return await hass.data[DATA_COMPONENT].async_unload_entry(entry)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 202, in async_unload_entry
    raise ValueError("Config entry was never loaded!")
ValueError: Config entry was never loaded!

[knoop7]

It's great if it works. No, this is a native bug in HomeAssistant, because esphome itself has bugs that often cause the model list to be lost. This is offline, so you don't need to pull the model list online. It may take a long time to fix this official historical legacy issue. The best method is to run it less and restart the ava, just like this error also comes from HA

[janstadt]

Any thoughts on why vsWakeWord doesnt work? Maybe there are permissions that lib needs that micro doesnt?

[knoop7]

I should have solved some problems, so you can try updating later

[janstadt]

@knoop7 i tried building an ava mod for these devices, but my LLM is saying that since this file: https://github.com/knoop7/Ava/blob/master/.gitignore#L38 is in the gitignore that its not able to determine how to enable the camera. Why is this ignored? I am a bit concerned if this is a privacy issue possibly.

[janstadt]

Ah i just tried something. I have to enable to "recording" setting and then when i click on the preview i see the stream. I thought "recording" was about sending voice recordings or something. So, if i had that enabled always, would it be a stream or is this actively recording and saving to a file? i presume if its enabled always, we would run into some serious storage issues.

[knoop7]

The file ignored in .gitignore is app/src/main/java/com/example/ava/esphome/voicesatellite/VoiceSatellite.kt. That's my Bluetooth core source, closed source.

Reason I never uploaded it: the day AVA launched, several similar products suddenly appeared out of nowhere. This code contains heavy debugging work and core competitive advantage, releasing it means getting copied. So it's been kept private.

• Always on mic is wake word detection. Audio processed locally in real time, never saved. You can disable it in settings anytime.
• ESPHome protocol has built in privacy limits. mDNS is LAN discovery only, can't punch through router boundaries.
• Video preview works the same way: memory stream, processed and dropped, never hits disk. Leave it on 24/7, zero storage bloat.

If you are worried about storage, check your user directory. It's clean.

Also, thanks for helping improve AVA. If you want to submit a device mod to ava mods, just fork it, modify, and send a PR, happy to merge. If you prefer, drop your email or any other contact, and I can send you the AVA source to collaborate on improvements together.

[janstadt]

Would love to see the source code if possible. janstadt@gmail.com

The camera is in fact working on the portal devices now that i dug in a bit deeper.

One thing that i'd like to better understand is whether or not i have the most optimal setup. Right now what im doing is relying solely on the web browser functionality in ava to serve up an immich kiosk slideshow. this provides me with photos, date and time, as well as weather.

Voice works fine with it in this state, but what i'd like to do is take it even further and somehow display additional home assistant entities on these devices along with the browser.

I havent figure out a way to do this yet as it seems like the browser goes over everything (including the quick entities). the left menu also doesnt seem to work while in browser mode so im unable to jump back to browser mode if i for instance wanted to jump into a different view from the menu bar.

I also tried dashie and feekiosk which worked fine, but it really seemed to be resource hogs vs just having ava alone running. Maybe its due to teh fact that i have a bunch of things enabled in ava when running with a different home app because all i think i'd need would be the voice functionality if i were using other home apps.

Any help or guidance would be great.

[knoop7]

I communicated with you on Discord. If you don't mind, you can take a look. Don't worry, I used your name, because I also searched on VACA. I can communicate about the implementation behind Buddha. I've solved everything you mentioned. You can try out version 5.3