Ubuntu CLI commands

[auwsom]

Hi, love that Mycroft is working on Kubuntu 18.10. I only had to fiddle with pulseaudio (restart?) to get the audio out working (although the audio test worked). I've been trying to get Voice Command to work for years, starting with WSP, Vocola and Dragonfly on Windows, but the Winapi and such calls are very limited and poorly documented. Its great that Kubuntu can use Python calls.

So the first thing I wanted to do was/is to Voice Control Vim (like in this old video using .NET? https://www.youtube.com/watch?v=TEBMlXRjhZY) or at least run some CLI commands. Unfortunately, it looks like using the KDE Plasmoid is the only way to do this? Please correct me if I'm wrong. I do see there are windows navigation controls with the Desktop Control Skill (is the plasmoid necessary for this? shouldnt there be a way to use the desktop control commands throught the command line without the whole plasmoid feature?), which would be handy, but I cant seem to get the plasmoid installed. So if there is a direct way to hook into Mycroft output and redirect it to bash or some other basic form, that would be my simplest solution. I have used the CLI debug tool which is great, but dont see how that could be redirected yet. I realize Mycroft was built to operate on devices without keyboards, but output as text to CLI commands seems like a basic tool that is fundamental, even for workarounds such as a plasmoid not installing.

Installation of the plasmoid hangs on "Installing../lottie/qmldir" and had two package install errors for qtdeclarative5-qtquick2-plugin and qtdeclarative5-models-plugin. Similar to this issue for installing on Debian (MycroftAI/installers#9), except Im using Kubuntu 18.10 Cosmic which doesnt have these packages in its repos. Im not sure if I can install them manually. Ive been using the appimage installer, but will try the manual install for Debian again. No, actually that ended where the instructions say "sudo chmod +x /usr/share/plasma/plasmoids/org.kde.plasma.mycroftplasmoid/contents/code/startservice.sh" because there is no 'code' directory created, which would have the scripts to run manually even. Not sure if I need those scripts, but after the hung appimage install I do have a plasmoid, which gives these error:

"Error loading QML file: file:///usr/share/plasma/plasmoids/org.kde.plasma.mycroftplasmoid/contents/ui/main.qml:33:34: Type FullRepresentation unavailable
file:///usr/share/plasma/plasmoids/org.kde.plasma.mycroftplasmoid/contents/ui/FullRepresentation.qml:31:1: module "Mycroft" is not installed"

I may try a restart after finishing tasks left in browser windows, but would love a path forward that doesnt require any plasmoid and all the depedencies that install required. Thanks for any pointers.

[auwsom]

Also, this makes me wonder if the plasmoid will install on the new 20.04 LTS.

I do see the Desktop Commands skill uses Python pyautogui here: https://github.com/TREE-Ind/desktop-control/blob/e3eaa3827f4644d5c4d5fc1b878ba7bb83aae657/__init__.py, which could be used in a shell script or directly in Bash. If these handful of commands are all that would need to be ported over to a CLI based integration, I would propose making such a CLI centered integration central and have the plasmoid incorporate it for its own use. That way it could act as a more universal repository of PC based actions for the community to add to. Probably a folder for each OS.

[auwsom]

Working on a solution, so far this works to capture output, extract the command from the voice.log:
tail -f -n 1 /var/log/mycroft/voice.log | grep --line-buffered -Po "(?<=').*?(?=')" | xargs -t -n1 -d '\n' echo
This will actually execute the spoken command:
tail -f -n 1 /var/log/mycroft/voice.log | grep --line-buffered -Po "(?<=').*?(?=')" | xargs -t -n1 -d'\n' -I {} sh -c "{}"
Need to make a CLI Command skill that recognizes a keyword like 'command' or 'bash' to retrieve the last phrase and excute. Also want a loop back two times before giving up after the recording timeout.

[krisgesling]

Hi there,

For "redirecting" the commands from the CLI, you probably want to look at the MessageBus and the sample client we have available:

Alternatively you can create a Mycroft Skill and call bash commands from your Python.

As for Plasmoid and the Desktop Control Skill, you're best to raise issues directly on those project repo's.

[auwsom]

Okay, thanks @krisgesling. I saw the activity on those was much older than that here in the main repo, so I thought I would try to reach out.

I am working toward creating a skill for this, a bit surprised its not already here ;), but still working through the basics. I'll at@ you on the Forum discussion about a glitch I found with pasting from the forum.. not sure if that goes to GitHub or what since it has to do with the Forum font.

I just saw how to install using a special mycroft-pip (interesting?) here mycroft-pip install mycroft-messagebus-client from that link I had been reading from, but it doesnt work. I must need to use the correct venv, but dont know where that is.. ah, just found the command in ~/mycroft-core/venv-activate.sh source venv-activate.sh, posting that for others who follow.

[krisgesling]

Yeah worth a try. Haven't seen the forum thing before. Discourse tries to do some funky stuff that I don't always agree with (like taking over your ctrl+F keyboard shortcut). So this might be related to something there.

The mycroft-pip command is available if you allowed the installer to add to your $PATH. It's just a shortcut instead of manually activating the venv and running pip install so doing as you've posted is spot on too.

[auwsom]

The test command (first one on that paeg) did work after activating the venv: python3 -m mycroft.messagebus.send 'speak' '{ "utterance": "test"}''

Actually from this page: https://mycroft-ai.gitbook.io/docs/mycroft-technologies/mycroft-core/message-types

[auwsom]

@krisgesling, I'm getting the basic setup down, but one setting looks daunting is the custom wake word. Even the 'noob' question below looks like it involves training a voice model. Is there no way to add a 'Hotword' (or is that different) based on the basic phonemes?

Sorry just saw the below.. trying to power through the setup part of this:

The simplest method to add a custom Wake Word to Mycroft is to use PocketSphinx. This is done by defining the phonemes that make up the Wake Word. It does not require any training, however is less reliable than a well-trained Precise Wake Word.

https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customizations/wake-word
https://community.mycroft.ai/t/custom-wake-word-for-noobs/7252

[auwsom]

You can probably close this as I opened an issue (MycroftAI/installers#12 above) for the plasmoid install not working.

As for the command line, you could put it as a Feature Request. I'm not to far away from finishing a Skill for it that could be included if people like it. And will report back here when its done.

[auwsom]

Here's the basic working skill called 'SealEyeCommand'. You can test it from CLI using python3 -m mycroft.messagebus.send 'recognizer_loop:utterance' '{ "utterances": ["command echo test"]}' .

from mycroft import MycroftSkill, intent_file_handler
import subprocess

class SealEyeCommand(MycroftSkill):
    def __init__(self):
        MycroftSkill.__init__(self)

    @intent_file_handler('command.cli.intent')
    def handle_testa(self, message):
        utterance = message.data.get('utterance')
        if not ' '  in utterance:  
            self.speak('please say your command')
            return True
        commandL = utterance.split(' ', 1)
        #self.log.info("commandList: "+str(commandL))
        command = utterance.split(' ', 1)[1]
        #self.log.info("command: "+str(command))
        process = subprocess.Popen(command.split(), stdout=subprocess.PIPE)
        output, error = process.communicate()
        output = output.decode('UTF-8')
        self.log.info("Output: "+str(output))
        self.log.info("Error: "+str(error))
        self.speak(str(output))

def create_skill():
    return SealEyeCommand()

 17:28:17.947 | INFO     | 21095 | Testa | commandList: ['command', 'echo test']
 17:28:17.947 | INFO     | 21095 | Testa | command: echo test
 17:28:17.957 | INFO     | 21095 | Testa | Output: test
 17:28:17.958 | INFO     | 21095 | Testa | Error: None

https://mycroft-ai.gitbook.io/docs/skill-development/marketplace-submission

[watchdominiondotcom]

Ive been trying to find a way to launch the plasma widget and i haven't been able to. beginner here. but i launched mycroft qtapplicsation but that wont connect. mycroft on konsole work though. im on latest kubuntu. i know it was said its not a mycroft core issue but this was the closest to the topic sorry. maybe someone can help or point me the way?

[auwsom]

@watchdominiondotcom, I would also put an issue in over at https://github.com/MycroftAI/installers/issues, as they are the main people dealing with the plasmoid. I havent heard any response on my issue though.

What are you trying to use the widget for? Maybe we can colab on getting the functionality to work with a regular Mycroft Skill.. Ive just finished getting a CLI Command Skill working which in theory should do just about anything youd want on a desktop. Im about to try and start porting over the basic window moving, etc, functions from the widget and the Desktop Launcher Skill.. shouldnt be too difficult

https://github.com/MycroftAI/skill-desktop-launcher

[auwsom]

@watchdominiondotcom so far it can minimize and restore a window, as well as do a basic dictation:
MycroftAI/mycroft-skills#1343

[auwsom]

Closing this as Ive added it in issue form over at mycroft-skills