Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI helper / personal assistant on command prompt #44

Closed
kensoh opened this issue Aug 7, 2017 · 5 comments
Closed

CLI helper / personal assistant on command prompt #44

kensoh opened this issue Aug 7, 2017 · 5 comments
Labels

Comments

@kensoh
Copy link
Member

kensoh commented Aug 7, 2017

Referencing - https://github.com/tebelorg/TagUI/issues/42#issuecomment-320465044

I would now really want to make a CLI helper which can help to run scripts. The TagUI scripts are already in natural-language-like syntax to convert to JS code. I want to have a helper that can convert natural-language-like instructions on the command line to call the right scripts with right parameters.

Instead of typing tagui abc_bank_balance on command prompt, I think it is more intuitive to support typing erina get my abc bank balance or something like that directly on command prompt. And let the magic happens behind the scenes to return the results.

Having this opens up voice-control possibility later on. And it has synergy with chat user interfaces.

@kensoh kensoh self-assigned this Aug 7, 2017
@kensoh kensoh added the feature label Aug 7, 2017
@kensoh
Copy link
Member Author

kensoh commented Aug 8, 2017

There are 4 types of sentence structures in English. Giving instructions and asking a question should be able to be done with type 1 - simple sentence. A simple sentence is an independent clause, which is made up of verb, pronoun, noun, adjective, adverb, conjunction, etc. Some more dev notes below.

http://www.english-grammar-revolution.com/sentence-structure.html

Types of intentions - 1. instruction, 2. question, 3. other types not yet thought of

  1. An instruction probably starts with a verb (get something, do this, send that)
  2. If it is a question instead, it starts with who, what, when, where, why, how?

Handling intention types - rough ideas how to approach

  1. an instruction should have automation handlers defined, if no match error out
  2. a question without defined handler can be passed to WolframAlpha or Google

Context contains more info than opening verb / 5w1h

  • The key is interpreting and matching the context correctly, as the opening verb or 5w1h can be inferred or somewhat ignored if context is understood. But understanding the opening verb correctly or the 5w1h is useless if context is derived incorrectly. Focus is to make out context, then address action.

Filler words, parts of grammar with low info-density

  • There are probably filler words which contain very little useful info but is part of normal grammar. For eg what is my next appointment. is has little meaning and the full meaning can be captured in context of appointment, next to convey option / parameters, and lastly what to convey it as a question. my can possibly be considered a filler as the context by default assumes context of the user. Intuitively, getting this right is important is order to trim down an instruction or question to its core intention. Before finding a match in defined automation flows. Otherwise with many filler words a match becomes much harder to find to match exactly the instruction or question.

Initial general structure of command line input

  • erina 5w1h / verb fillers options / modifiers fillers context

Technical design for initial implementation

  • erina bash script and windows batch file. this name is chosen because 3 syllabus, potentially can be used as trigger word for voice control. and it is a russian-sounding name, which I like. as a batch file / bash script, users can rename it to whatever they wish.

  • after calling script from commandline, pass all parameters into a PHP interpreter script. all the natural language interpretation is gonna be challenging or impossible to be done within bash script / batch file. let a PHP helper do it. and spits out the command to be executed as output. it is also not good to do the execution within PHP, as users needs to override system default security setting to allow running of executables. that's a really bad idea for security reasons. better to let PHP script do the interpretation and the final command eg tagui download_sales_report june be returned to bash/batch script for execution.

  • PHP helper to parse inputs base on above general structure.

    • 1. remove fillers - is, are, was, were, my, me
    • 2. check 5w1h / verb _ context for matching automation flow names
    • 3. if match found returns 5w1h / verb_context options / modifiers
    • 4. if not found and not a question, just show message don't understand, and maybe with a link to readme section on how to use / set up command line helper. link not needed i guess, users would probably have seen the readme on this feature before knowing to using this helper.
    • 5. if not found but is a question, return as a custom WolframAlpha automation flow that asks a question there, formats the result and returns as a sentence. UPDATE - reduce scope, don't get answer from Wolfram. Doing some trials shows it is very hard to structure the response returned into a proper sentence, due to the wide range of questions and returning answer structures. Alternatively, searching google and returning the small box with answer is a great idea. But in practice users running the flow from different regions will definitely run into issues as Google will auto-redirect to local versions, and the website layout will be different. Taking that approach will simply end up in disappointment when it fails for users due to different google instance. This idea is implementable if invoked as a web-service to run from the cloud (fixed IP, region, google instance etc). shelf the idea for now. I still like the Wolfram idea, but the response returned is too technical for normal user's expectation. Instead of a paragraph of bio or answer, it is a dynamic tables structure with different fields, images, notes etc. Better to drop for now as it is hard to control quality of interpretation response by TagUI.
  • for users to create custom automation, the file name will be 5w1h / verb _ context and taking in options / modifiers as parameters. that translates to input variables p1 to p9. that can be used in the automation flow to handle actions dynamically on the target webpage.

  • default execution option should be in quiet mode to avoid showing all the unnecessary details of execution. only the result matters, which can be shown using echo step in the automation flow. default should be using phantomjs, so that there's no disruption to user web browser if he / she is using chrome browser. if user want to run in visible mode or save the hassle of coding login steps into the automation flow, then user can type using chrome behind. That means need to first extract out TagUI run options from behind the command, before start interpreting. Other variants can be using firefox and using headless for running on Firefox and headless Chrome respectively. Also added with chrome, with firefox, with headless.

Enough planning to start coding

  • placeholder for more notes

kensoh referenced this issue Aug 9, 2017
Commit for php helper and linux/macos shell runner, outstanding is
windows batch file.

The TagUI scripts are already in natural-language-like syntax to
convert to JS code. I want to have a helper that can convert
natural-language-like instructions on the command line to call the
right scripts with right parameters.

Instead of typing tagui download_bank_report june on command prompt, I
think it is more intuitive to support typing erina download my june
bank report or something like that directly on command prompt. And let
the magic happens behind the scenes to return the results.

running this -
erina 5w1h/verb fillers options/modifiers fillers
single-or-multi-word-context

results in running -
tagui 5w1h/verb_context quiet options/modifiers

calling using chrome/firefox/headless or with chrome/firefox/headless
at the end of erina … will call tagui in visible chrome, firefox,
headless chrome respectively.

this feature allows future extension to natural language voice control.
the trigger word erina can be changed by just renaming the erina file
(macos/linux) or erina.cmd (windows) file.
@kensoh
Copy link
Member Author

kensoh commented Aug 9, 2017

committed for php helper that does the interpretation and the macos/linux runner. outstanding is windows erina.cmd batch file.

kensoh added a commit that referenced this issue Aug 10, 2017
Commit for windows batch file. For main commit for php helper and
macos/linux see -
tebelorg@3e3f924
015a14502

The TagUI scripts are already in natural-language-like syntax to
convert to JS code. I want to have a helper that can convert
natural-language-like instructions on the command line to call the
right scripts with right parameters.

Instead of typing tagui download_bank_report june on command prompt, I
think it is more intuitive to support typing erina download my june
bank report or something like that directly on command prompt. And let
the magic happens behind the scenes to return the results.

running this -
erina 5w1h/verb fillers options/modifiers fillers
single-or-multi-word-context

results in running -
tagui 5w1h/verb_context quiet options/modifiers

calling using chrome/firefox/headless or with chrome/firefox/headless
at the end of erina … will call tagui in visible chrome, firefox,
headless chrome respectively.

this feature allows future extension to natural language voice control.
the trigger word erina can be changed by just renaming the erina file
(macos/linux) or erina.cmd (windows) file.
kensoh added a commit that referenced this issue Aug 10, 2017
- define php path for windows cli helper and output carriage return +
line feed for generated tagui_helper.cmd

- baseline option set flow file name variable correctly
@kensoh
Copy link
Member Author

kensoh commented Aug 10, 2017

tested alright for windows. macos/linux/windows all ok. closing issue.

next is update readme for various commits / new features,
and publishing package installations for TagUI v2.3

@kensoh kensoh closed this as completed Aug 10, 2017
@kensoh kensoh removed their assignment Aug 31, 2017
@Vibra24
Copy link

Vibra24 commented Nov 9, 2018

Mr Kensoh, can you help me with this command:

@echo off
:A
start chrome
goto :A
&&
Timeout 4
&&
Shutdown /s

I want the first command (start chrome) to run, before the time and shutdown do. Like it should run and keep running as my PC is shutting down

@kensoh
Copy link
Member Author

kensoh commented Nov 10, 2018

Hi @Vibra24 I've left AI Singapore and now @Aussiroth is maintaining TagUI, copying him.

This isn't a TagUI question but a question on how to program Windows batch file. Normally && is used to combine command and only runs the next command when the first one comes back successfully.

command a && command b && command c

I don't think start command will come back and return control to the batch file until Chrome browser quits. Not sure what you are trying to achieve, but here is the wrong place to ask about batch files, somewhere else with Windows batch file experts or perhaps stack overflow would be better. For example, below post uses start "" command

https://stackoverflow.com/questions/17201507/how-to-use-the-start-command-in-a-batch-file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants