Feature Request: Interactive JavaScript Shell

[personalizedrefrigerator]

#!python

import os, sys, re
import subprocess, tempfile

from argparse import ArgumentParser

VALID_JS_NAME = re.compile(r"^[a-zA-Z_][a-zA-Z0-9_]+$")

LONG_DESCRIPTION = """jsi executes JavaScript by running it within a WkWebView. As such, all standard browser APIs should be available.""".strip()

SHELL_BANNER = "Type 'help' for help and 'quit' to exit."

SHELL_HELP = \
"""This program is a simple, interactive JavaScript shell.
In addition to browser-provided APIs, the following commands are provided:
    print(message)
    println(message)
        Prints [message] to the terminal.
        `println` prints a trailing newline after printing [message].

Note that changes to `document` won't be visible unless jsi is run with the '-w' or '--in-window' option. If this option is not given, code is run in a background WebView.
"""

JS_UTILITIES = r"""
/**
 * Additional functions for script output
 */

let print = (...output) => {
    _jsi.notePrintFn(window.print, output);
    return window.print.apply(this, output);
};

let println = (...output) => {
    _jsi.notePrintFn(window.println, output);
    return window.println.apply(this, output);
};

let print_error = (...output) => {
    _jsi.notePrintFn(window.print_error, output);
    return window.print_error.apply(this, output);
};

// Define the _jsi namespace.
(function() {

if (!window._jsi) {
    window._jsi = {};
    _jsi.uncollectedPrints = [];
}

/**
 * Print unprinted output.
 */
_jsi.collectPrints = () => {
    if (_jsi.uncollectedPrints) {
        for (let item of _jsi.uncollectedPrints) {
            item.printFn.apply(item.printFn, item.args);
        }
    }

    _jsi.uncollectedPrints = [];
};

_jsi.pauseSavingFuturePrints = () => {
    _jsi.savingPrintsPaused = true;
};

_jsi.saveFuturePrints = () => {
    _jsi.savingPrintsPaused = false;
};

/**
 * Note that we have called a print
 * function. This allows its output to
 * be collected later.
 */
_jsi.notePrintFn = (fctn, args) => {
    if (_jsi.savingPrintsPaused) {
        return;
    }

    _jsi.uncollectedPrints.push({
        printFn: fctn,
        args: args
    });
};

/**
 * @param obj Entity to stringify and print.
 * @param recurseDepth Maximum depth to print properties of sub-objects.
 * @param indentation How much to indent the region. Optional.
 * @param toplevel Optional; used internally.
 * @return a string description of [obj].
 */
_jsi.stringify_obj = function (obj, recurseDepth, indentation, toplevel) {
    /* Fill default arguments. */
    indentation = indentation === undefined ? "" : indentation;
    toplevel = toplevel === undefined ? true : toplevel;
    recurseDepth = recurseDepth === undefined ? 1 : recurseDepth;

    /* Accumulator for output. */
    let result = "";
    let formatLine = "";

    /**
     * Indent [content] with given indententation and additional characters [additional].
     */
    const indent = (content, additional) => {
        additional = additional || "";

        return (indentation + additional + content).split('\n').join('\n' + indentation + additional);
    };

    const termEscape = (text) => {
        return text.split('\033').join('\\033').split('\0').join('\\0').split('\\').join('\\\\');
    }

    /**
     * Surrounds text with multiline quotation characters, escapes text content.
     */
    const inQuotes = (text) => {
        return "`" + termEscape(text).split('`').join('\\`') + "`";
    };

    /**
     * Append [content] to result with proper indentation and a newline.
     */
    const outputLine = (content) => {
        result += indent(content) + "\n";
    };

    /* If an object, list its properties */
    if (recurseDepth >= 0 && typeof(obj) == 'object') {
        result += '' + obj;
        result += " {\n";

        let propDescriptions = [];

        for (let prop in obj) {
            result += indent(prop + ": ", "  ");

            /* Permission errors may prevent us from accessing/enumerating properties of obj[prop]. */
            try {
                result += _jsi.stringify_obj(obj[prop], recurseDepth - 1,indentation + "  ", false);
            } catch(e) {
                result += "(error `" + e + "`)";
            }

            result += ",\n";
        }

        outputLine("}");
        formatLine = "js";
    } else if (typeof (obj) == "string") {
        const quoted = inQuotes(obj);
        const lines = quoted.split('\n');
        formatLine = "js";

        if (quoted.length > 100 && recurseDepth < 0) {
            result += quoted.substring(0, 100) + ' `...';
        } else if (lines.length == 1) {
            result += quoted;
        } else {
            if (obj.search(/[<][\/][a-zA-Z0-9 \t]+[>]/) >= 0 && recurseDepth >= 0) {
                for (const line of lines) {
                    outputLine(line);
                }

                formatLine = "html";
            } else {
                result += quoted;
            }
        }
    } else if (typeof (obj) == 'function') {
        if (recurseDepth < 0) {
            result += "[ Function ]";
        } else {
            const lines = termEscape(obj + '').split('\n');

            for (let i = 0; i < lines.length; i++) {
                if (i == 0) {
                    result += lines[i] + '\n';
                } else if (i == lines.length - 1) {
                    result += indent(lines[i]);
                } else {
                    outputLine(lines[i]);
                }
            }
        }

        formatLine = "js";
    } else if (obj == undefined) {
        result += "undefined";
        formatLine = "undefined";
    } else {
        result += termEscape('' + obj);
    }

    formatLine += " " + result.split("\n").length;

    if (!toplevel) {
        formatLine = "";
    } else {
        result = "\n" + result;
        formatLine = "\n" + formatLine;
    }

    return result + formatLine;
};

})(); /* End declaration of _jsi "namespace". */
"""

def getJSInfo(text, stopIdx=-1):
    """
    Returns information about [text] if
    interpreted as JavaScript. Information
    is returned as a dictionary in the form,
    {
        endsWithOp: '+', '-', '*', '/', '&', '|', '^', or None
        parenLevel: Number of unclosed nested parens
        squareBracketLevel: Number of unclosed nested '[' ']' brackets
        curlyBraceLevel: Number of unclosed nested '{' '}' braces.
        escaped: True iff the last character processed was escaped escaped
        inComment: The last character processed was in some (single or multi-line) comment
        inMultiLineComment: True iff the last character processed was in a multi-line comment
        inQuote: '`', '"', "'", or None
        trailingEmptyLineCount: Number of lines with no non-space characters processed at the end of the input.
        endsWithSemicolon: Whether the last character in the input is ';'
    }
    """

    inQuote = None
    inSLComment = False
    inMLComment = False
    escaped = False
    parenLevel = 0
    curlyBraceLevel = 0
    squareBracketLevel = 0
    endsWithOperator = None
    emptyLineCount = 0
    endsWithSemicolon = False

    if stopIdx > len(text) or stopIdx < 0:
        stopIdx = len(text)

    for i in range(0, stopIdx):
        char = text[i]
        nextChar = ''
        if i + 1 < len(text):
            nextChar = text[i + 1]

        isSpaceChar = char == ' ' or char == '\t' or char == '\n'

        if not isSpaceChar:
            endsWithOperator = None
            emptyLineCount = 0
        endsWithSemicolon = False

        if escaped:
            escaped = False
        elif char == '\n':
            emptyLineCount += 1
            inSLComment = False
        elif char == '*' and nextChar == '/':
            inMLComment = False
        elif inMLComment or inSLComment:
            pass
        elif char == '"' or char == "'" or char == '`':
            if inQuote is None:
                inQuote = char
            elif char == inQuote:
                inQuote = None
        elif inQuote:
            pass
        elif char == '/' and nextChar == '*':
            inMLComment = True
        elif char == '/' and nextChar == '/':
            inSLComment = True
        elif char == '\\':
            escaped = True
        elif char == '(':
            parenLevel += 1
        elif char == ')':
            parenLevel -= 1
        elif char == '{':
            curlyBraceLevel += 1
        elif char == '}':
            curlyBraceLevel -= 1
        elif char == '[':
            squareBracketLevel += 1
        elif char == ']':
            squareBracketLevel -= 1
        elif char == ';':
            endsWithSemicolon = True
        elif char == '+' or char == '-' or char == '*' or char == '/' or char == '&' or char == '^' or char == '|':
            endsWithOperator = char

    result = {}
    result['endsWithOp'] = endsWithOperator
    result['parenLevel'] = parenLevel
    result['squareBracketLevel'] = squareBracketLevel
    result['curlyBraceLevel'] = curlyBraceLevel
    result['escaped'] = escaped
    result['inComment'] = inMLComment or inSLComment
    result['inMultiLineComment'] = inMLComment
    result['inQuote'] = inQuote
    result['trailingEmptyLineCount'] = emptyLineCount
    result['endsWithSemicolon'] = endsWithSemicolon
    return result


def shouldNewlineAccept(text):
    """
    Returns true iff pressing Return
    should append a newline to text.
    """

    ctx = getJSInfo(text)
    hasUnfinishedCode = ctx['squareBracketLevel'] > 0 or \
            ctx['curlyBraceLevel'] > 0 or\
            ctx['parenLevel'] > 0 or \
            ctx['inQuote'] or \
            ctx['inMultiLineComment'] or \
            ctx['endsWithOp']
#            ctx['endsWithSemicolon']

    return ctx['trailingEmptyLineCount'] < 2 and hasUnfinishedCode or ctx['escaped']


try:
    import pygments
    from pygments.lexers.javascript import JavascriptLexer
    from pygments.lexers.html import HtmlLexer
    from pygments.formatters import TerminalFormatter
    havePygments = True
except:
    havePygments = False

try:
    import prompt_toolkit, prompt_toolkit.filters, prompt_toolkit.validation
    from prompt_toolkit.lexers import PygmentsLexer
    from prompt_toolkit.completion import Completion, Completer, NestedCompleter

    class NewlineAcceptTester(prompt_toolkit.filters.Condition, prompt_toolkit.validation.Validator):
        def __init__(self, session):
            self._session = session
            self._text = ""
        def validate(self, document):
            self._text = document.text
        def __call__(self):
            return shouldNewlineAccept(self._text)
        def __bool__(self):
            return True

    class JSCompleter(Completer):
        completions = {
            "window": None,
            "if": None,
            "else": None,
            "var": None,
            "let": None,
            "for": None,
            "of": None,
            "in": None,
            "while": None,
            "const": None,
            "class": None,
            "extends": None,
            "new": None,
            "function": None,
            "println": None,
            "print": None,
            "Math": {
                "random": None,
                "sin": None,
                "cos": None,
                "asin": None,
                "acos": None,
                "atan": None,
                "atan2": None,
                "tan": None,
                "pow": None,
                "log": None,
                "abs": None,
                "cbrt": None,
                "sqrt": None,
                "exp": None,
                "floor": None,
                "round": None,
                "ceil": None,
                "PI": None,
                "E": None,
                "SQRT2": None,
                "LN2": None,
                "LN10": None
            },
            "BigInt": None,
            "Number": None,
            "Date": None,
            "help": None,
            "quit": None
        }

        def __init__(self, runJS):
            self._userPopulated = {}
            self._runJS = runJS
            self._populateFromJS("window")
            self.completions["window"]["window"] = self.completions["window"]
            for key in self.completions["window"]:
                self.completions[key] = self.completions["window"][key]

        def get_completions(self, document, complete_event):
            cursor = document.cursor_position
            text = document.text
            word = document.get_word_before_cursor(pattern=re.compile(r"[a-z0-9A-Z._]+"))

            # First, if we're in a quote or comment, quit.
            ctx = getJSInfo(text, cursor)
            if ctx['inComment'] or ctx['inQuote']:
                return

            def fromDict(parts, completionDict):
                """ Yields completions matching parts from completionDict """
                result = []

                if len(parts) > 0:
                    for key in completionDict:
                        if len(key) >= len(parts[0]) and key[:len(parts[0])] == parts[0]:
                            if len(parts) == 1:
                                result.append(Completion(key, start_position = -len(parts[0])))
                            elif completionDict[key]:
                                result += fromDict(parts[1:], completionDict[key])
                return result

            parts = word.split('.')
            completionCount = 0

            for completion in fromDict(parts, self.completions):
                completionCount += 1
                yield completion

            if completionCount == 0 and len(word) > 0 and len(parts) >= 2:
                # If the word's parent isn't in this' completion dict,
                # stop.
                checkDict = self.completions
                for part in parts[:-2]:
                    if not part in checkDict:
                        return
                    checkDict = checkDict[part]

                if word[-1] == '.':
                    word = word[:-1]
                populateFrom = ".".join(parts[:-1])

                # If we've already tried to populate suggestions
                # for this, stop. There may not be any
                if populateFrom in self._userPopulated:
                    return

                self._populateFromJS(populateFrom)
                self._userPopulated[populateFrom] = True

        def _populateFromJS(self, base, parentDict = None, depth = 0, maxDepth = 0):
            if depth > maxDepth:
                return

            if parentDict is None:
                parts = base.split('.')
                parentDict = self.completions
                baseSoFar = []

                for part in parts:
                    baseSoFar.append(part)
                    if not part in parentDict or not parentDict[part]:
                        partExistsResult = self._runJS("""
                        try {
                            %s;
                            println('success');
                        } catch(e) {
                            println('error');
                        }
                        """ % ".".join(baseSoFar)).strip()

                        # If it doesn't exist, don't generate completions
                        # for it!
                        if partExistsResult != "success":
                            return

                        parentDict[part] = {}
                    parentDict = parentDict[part]

            out = self._runJS("""{
            let result = '';
            try
            {
                for (let key in eval(%s)) {
                    if (key.indexOf('\\n') == -1) {
                        result += key + "\\n";
                    }
                }
            } catch(e) { }
            println(result); }""" % jsQuote(base))

            for key in out.split('\n'):
                if key != "" and VALID_JS_NAME.match(key):
                    parentDict[key] = { }
                    self._populateFromJS(base + "." + key, parentDict[key], depth + 1, maxDepth)

    promptTkSession = None
    if sys.stdin.isatty():
        promptTkSession = prompt_toolkit.PromptSession()
except Exception as e:
    print ("Warning: Unable to import prompt_toolkit: " + str(e))
    promptTkSession = None

class SourceFormatter:
    def __init__(self, disableColor=False):
        self._disableColor = disableColor or not havePygments

    def isHighlightingEnabled(self):
        return not self._disableColor

    def formatHtml(self, text):
        """ Returns [text] highlighted via ASCII escape sequences """

        if self._disableColor:
            return text
        else:
            return self._highlight(text, HtmlLexer())

    def formatJavascript(self, text):
        """
            Returns [text] with ASCII escape sequences inserted to provide
        syntax highlighting
        """

        if self._disableColor:
            return text
        else:
            return self._highlight(text, JavascriptLexer()).rstrip()

    def getJavascriptLexer(self):
        """
            Returns JavascriptLexer, or None, depending on whether coloring is enabled.
        """

        if self._disableColor:
            return None
        else:
            return JavascriptLexer

    def formatError(self, text):
        if self._disableColor:
            return text
        else:
            return "\033[92m%s\033[0m" % text

    def _highlight(self, text, lexer):
        return pygments.highlight(text, lexer, TerminalFormatter())

class InputReader:
    def __init__(self, disableFormatting, promptText, runJS):
        self._formattingDisabled = disableFormatting or not promptTkSession
        self._promptText = promptText

        if not self._formattingDisabled:
            self._makeInteractive()
            self._lexer = PygmentsLexer(JavascriptLexer)
            self._shouldNewlineAcceptTest = NewlineAcceptTester(promptTkSession)
            self._completer = JSCompleter(runJS)

    def prompt(self):
        if self._formattingDisabled:
            print(self._promptText, end='')
            result = str(input())
            while shouldNewlineAccept(result):
                result += "\n" + str(input())
            return result
        else:
            return promptTkSession.prompt(self._promptText,
                    lexer=self._lexer,
                    multiline=self._shouldNewlineAcceptTest,
                    validator=self._shouldNewlineAcceptTest,
                    completer=self._completer)

    def _makeInteractive(self):
        runJS("window.interactiveCommandRunning = true;", inTermContext = True)

class Printer:
    PRIMARY_COLOR="\033[94m"
    SECONDARY_COLOR="\033[33m"
    HIGHLIGHT_COLOR="\033[93m"
    NO_COLOR="\033[0m"

    def __init__(self, disableColorization):
        self._colorDisabled = disableColorization

    def print(self, text, end='\n'):
        """ Print [text] with no additional formatting. """

        self._print(text, end=end)

    def printPrimary(self, text, end='\n'):
        """ Print [text] with primary formatting. """

        self._print(text, self.PRIMARY_COLOR, end=end)

    def printSecondary(self, text, end='\n'):
        """ Print [text] with secondary formatting. """

        self._print(text, self.SECONDARY_COLOR, end=end)

    def printHighlight(self, text, end='\n'):
        """ Print [text], highlighted """

        self._print(text, self.HIGHLIGHT_COLOR, end=end)

    def _print(self, text, colorize=None, end='\n'):
        if colorize is None or self._colorDisabled:
            print(text, end=end)
        else:
            print("%s%s%s" % (colorize, text, self.NO_COLOR), end=end)


## Returns [text] within a string, line breaks, etc. escaped.
def jsQuote(text):
    result = '`'
    for char in text:
        if char == '`' or char == '\\':
            result += '\\'
        result += char
    result += '`'

    return result

def exportVarDefs(js):
    VARDECL_LETTERS = { '', 'l', 'e', 't', 'c', 'o', 'n', 's', 't' }
    result = ''

    inQuote = None
    escaped = False
    buff = ''
    bracketLevel = 0
    inSingleLineComment = False
    inMultiLineComment = False

    def flushBuff(nextChar):
        nonlocal buff, result
        if nextChar == '\t' or nextChar == '\n' or nextChar == ' ':
            if buff.strip() == 'let' or buff.strip() == 'const':
                if bracketLevel == 0 and not inSingleLineComment and not inMultiLineComment and inQuote is None:
                    buff = 'var'
        result += buff
        buff = ''

    for i in range(0, len(js)):
        char = js[i]
        nextChar = ''

        if i + 1 < len(js):
            nextChar = js[i + 1]

        inComment = inSingleLineComment or inMultiLineComment

        buff += char
        if escaped:
            escaped = False
            continue
        elif inSingleLineComment and char == '\n':
            inSingleLineComment = False
        elif char == '*' and nextChar == '/':
            inMultiLineComment = False
        elif inComment:
            continue
        elif char == '"' or char == "'" or char == '`':
            if char == inQuote:
                inQuote = None
            else:
                inQuote = char
        elif inQuote:
            continue
        elif char == '/' and nextChar == '/':
            inSingleLineComment = True
        elif char == '/' and nextChar == '*':
            inMultiLineComment = True
        elif char == '{':
            bracketLevel += 1
        elif char == '}':
            bracketLevel -= 1
        elif char == '\\':
            escaped = True

        if not nextChar in VARDECL_LETTERS or not char in VARDECL_LETTERS:
            flushBuff(nextChar)

    result += buff
    return result

## Evaluates [content] as JavaScript. If
##  [inTermContext], [content] is evaluated
##  in the same context as the terminal UI.
## Returns the output, if any, produced by [content].
##  If [collectPrints], any async/delayed print statements' output is also returned.
def runJS(content, inTermContext=False, recurseDepth=1, sourceFormatter=SourceFormatter(), collectPrints = False):
    toRun = JS_UTILITIES

    if collectPrints:
        toRun += "_jsi.collectPrints();\n"

    # Don't re-print what was printed while jsc was running
    toRun += "_jsi.pauseSavingFuturePrints();\n"
    toRun += "print(_jsi.stringify_obj(eval(%s), %d));" % (jsQuote(exportVarDefs(content)), recurseDepth)

    # Save any async prints for later collection
    toRun += "_jsi.saveFuturePrints();"

    outFile = tempfile.NamedTemporaryFile(delete=False)
    outFile.write(bytes(toRun, "utf-8"))
    outFile.close()

    args = ["jsc"]
    if inTermContext:
        args.append("--in-window")
    args.append(os.path.relpath(outFile.name))

    output = subprocess.check_output(args)
    output = output.decode("utf-8").rstrip(" \r\n")

    os.unlink(outFile.name)

    ## The last line is always a format specifier, unless an error occurred.
    outLines = output.split('\n')

    if len(outLines) == 0:
        return "No output"

    lastLine = outLines[-1].split(" ")
    outLines = outLines[:-1]

    # If we don't have two words in the output's
    # format specification line, an error occurred.
    if len(lastLine) != 2 or not lastLine[1].isdecimal():
        return sourceFormatter.formatError(output)

    formatType = lastLine[0]
    formatLineCount = int(lastLine[1])

    formatLines = "\n".join(outLines[-formatLineCount:])
    unformatLines = "\n".join(outLines[:-formatLineCount])

    if formatType == 'js':
        formatLines = sourceFormatter.formatJavascript(formatLines)
    elif formatType == 'html':
        formatLines = sourceFormatter.formatHtml(formatLines)
    elif formatType == "undefined":
        formatLines = ""

        # Remove trailing linebreak from unformatted lines, we don't need it.
        if len(unformatLines) > 1 and unformatLines[-1] == '\n':
            unformatLines = unformatLines[:-1]

    result = unformatLines + formatLines
    return result

## Parse commandline arguments and take action on them, if necessary.
## Returns a map with the following keys:
## {
##  prompt: The prompt to use when interacting with the user.
##  omitIntro: False iff a brief introductory help message should be printed.
##  inTermWebView: True iff scripts should be run in the same WkWebView as the terminal UI.
##  noColor: True iff output should have no color formatting.
## }
def parseCmdlineArguments():
    args = ArgumentParser(
            description="An interactive JavaScript shell",
            epilog=LONG_DESCRIPTION
    )

    isTTY = True
    omitIntro = False
    defaultPrompt = "% "

    # If we're getting input from a pipe (i.e. cat foo.js | jsi)
    # don't print a prompt (by default).
    if not sys.stdin.isatty() or not sys.stdout.isatty():
        defaultPrompt = ""
        omitIntro = True
        isTTY = False

    args.add_argument("--prompt", default=defaultPrompt)
    args.add_argument("--no-banner",
            default=omitIntro, action="store_true", dest="omitIntro")
    args.add_argument("--no-color",
            default=not isTTY,
            dest="noColor",
            action="store_true", help="Disable colorized output")
    args.add_argument("-d", "--inspect-depth",
            default=0,
            type=int,
            help="When printing the details of an object, how many levels of sub-objects to inspect. This should be a small number (e.g. 0, 1, or 2).",
            dest="inspectDepth")
    args.add_argument("-w", "--in-window",
            action="store_true", dest="inTermWebView",
            help="Run JavaScript in the same WkWebView as the terminal UI.")

    result = args.parse_args()

    if not "omitIntro" in result:
        result.omitIntro = omitIntro

    return result


if __name__ == "__main__":
    args = parseCmdlineArguments()
    promptText = args.prompt
    response = ''

    formatter = SourceFormatter(args.noColor)
    execJS = lambda js, collectPrints=False: \
            runJS(js,
                    args.inTermWebView,
                    args.inspectDepth,
                    formatter,
                    collectPrints=collectPrints)

    out = Printer(args.noColor)
    readIn = InputReader(args.noColor, promptText, execJS)

    if not args.omitIntro:
        out.printPrimary(SHELL_BANNER)

    # If input is not directly from a user,
    # run input to this, then exit.
    if not sys.stdin.isatty():
        toRun = sys.stdin.read()
        out.print(execJS(toRun))
        sys.exit(0)

    while True:
        try:
            response = readIn.prompt()
        except EOFError:
            print()
            break
        except KeyboardInterrupt:
            print("quit")
            break
        if response == "quit":
            break
        elif response == "help":
            out.printPrimary(SHELL_HELP)
        else:
            out.print(execJS(response,  collectPrints=True))

It would be great to have this in a-Shell! I'm not an iOS developer, so I'm not comfortable opening a pull request.

[holzschu]

That looks great, and a great addition to a-Shell! Thank you very much.

I have a question: line 12 refers to ~/Documents/bin/Includes/libJS.js, which does not exist. Is that going to be an issue, or is it just for customizing?

[personalizedrefrigerator]

~/Documents/bin/Includes/libJS.js, which does not exist. Is that going to be an issue, or is it just for customizing?

See the comment above. It could definitely be used for user-customization.

[personalizedrefrigerator]

I need to clean this up (use tempfile library, etc) :). Closing the issue for now (I intend to re-open it soon!)

[holzschu]

Okay, thanks. I think you can use the environment variable TMPDIR, which is defined.

[ifuchs]

Sorry if this should be obvious but the lines:
from Includes.printUtil import *
from Includes.argsUtil import *
result in Module not found. Are there additional modules that must be installed to run this script?

[personalizedrefrigerator]

from Includes.printUtil import *
from Includes.argsUtil import *

Sorry!!! As I stated above, I need to clean this up! I copy-pasted this script from my ~/Documents/bin directory (I have a sub-directory Includes with printUtil.py and argsUtil.py files). Expect this to be fixed soon!

[personalizedrefrigerator]

This script's help text is currently somewhat inaccurate. See #197

[holzschu]

Thanks a lot for this command. I'm going to copy and test this in a-Shell.
I was going to ask about IO redirection (command > file or command | otherCommand), but it's an interactive shell, so this does not apply.

[personalizedrefrigerator]

It looks like multi-line input isn't working as expected...

% print(`
a
b
c`)

should not produce a syntax error...

Edit: This has been fixed. I apologize for re-opening this before having fixed several issues!

[personalizedrefrigerator]

The --no-intro argument was broken! I've fixed it and updated the script!

[holzschu]

I can understand expert users wanting to run jsc on the main webView, but I think jsi should always act on the background webView (which is the situation right now, but I wanted your opinion on the subject).
Also, I would suggest inverting the "--no-intro" behaviour, to be more in line with other commands: no text is the default behaviour, "-h" or "--help" shows the help text.

[personalizedrefrigerator]

I can understand expert users wanting to run jsc on the main webView, but I think jsi should always act on the background webView (which is the situation right now, but I wanted your opinion on the subject).

I would suggest the reverse! Commands run from jsi probably don't need to interact with the user via the keyboard, while commands run from jsc might. Of course, advanced users might want to use jsc to run JavaScript on the main WebView, so I suggest making jsc have an argument like --on-main-webview to determine where its content is run.

Also, I would suggest inverting the "--no-intro" behaviour, to be more in line with other commands: no text is the default behaviour, "-h" or "--help" shows the help text.

I was trying to mirror python and bc. Both print a brief message on launch.

[holzschu]

I would suggest the reverse! Commands run from jsi probably don't need to interact with the user via the keyboard, while commands run from jsc might. Of course, advanced users might want to use jsc to run JavaScript on the main WebView, so I suggest making jsc have an argument like --on-main-webview to determine where its content is run.

What I have in mind is:

• by default, both jsi and jsc are running on the secondary webView.
• jsc will have an option to run on the main webView, with a warning that it can break everything.
• jsi does not need that option.

It's not just about keyboard interaction, but also about not redefining keyboard input by mistake.

Also, I would suggest inverting the "--no-intro" behaviour, to be more in line with other commands: no text is the default behaviour, "-h" or "--help" shows the help text.

I was trying to mirror python and bc. Both print a brief message on launch.

I see your point, but both messages are much shorter. Other interactive commands like nslookup have no information before the prompt.

[personalizedrefrigerator]

... but both messages are much shorter.

$ jsi
Type 'help' for help and 'quit' to exit.
% 

I think the message is reasonably short. :)

jsi does not need that option

I, previously, used jsi to apply quick stylistic changes to a-Shell (e.g. document.body.style.filter = 'saturate(200%);) that I only wanted to persist for a session.
For my own use case, I would very much like jsi to have the option to run in the main WebView.

[holzschu]

I'm terribly sorry. In reading the code, I made a confusion between SHELL_HELP and SHELL_HEADER. I should have tested it instead of reading.

For the second point, you could either pass the non-recognized options sent to jsi to jsc, or recognize the --inWindow option when parsing arguments and then passing it to jsc.

[personalizedrefrigerator]

--inWindow option when parsing arguments and then passing it to jsc.

I made it --in-window, to fit the style of the other arguments to jsi.

Thank you for the feedback!

[personalizedrefrigerator]

I've added syntax highlighting for outputted HTML/JavaScript. Code has been updated above.

[personalizedrefrigerator]

Status update:

• Fixed an issue where nested quotes (e.g. "this 'is a test") would make input difficult.
• E.g.

% var ss = document.createElement("div");
% ss.innerHTML = "This is 'a test";

// Command still not accepted (thinks quote is unclosed).

[aramb-dev]

Show source

#!python



import os, sys

import subprocess, tempfile



from argparse import ArgumentParser



LONG_DESCRIPTION = """jsi executes JavaScript by running it within a WkWebView. As such, all standard browser APIs should be available.""".strip()



JS_UTILITIES = """

/**

 * Additional functions for script output

 */







/**

 * @param obj Entity to stringify and print.

 * @param recurseDepth Maximum depth to print properties of sub-objects.

 * @param indentation How much to indent the region.

 * @param highlightingJS Whether the region is within a highlighting block.

 * @return a string description of [obj].

 */

function _jsi_print_obj(obj, recurseDepth, indentation, highlightingJS) {

    /* Fill default arguments. */

    recurseDepth = recurseDepth === undefined ? 1 : recurseDepth;

    indentation = indentation === undefined ? "" : indentation;



    // If we're

    let highlightJSLevel = highlightingJS ? 1 : 0;



    /**

     * Indent [content] with given indententation and additional characters [additional].

     */

    const indent = (content, additional) => {

        additional = additional || "";



        return (indentation + additional + content).split('\\n').join('\\n' + indentation + additional);

    };



    const termEscape = (text) => {

        return text.split('\\033').join('\\\\033').split('\\0').join('\\\\0').split('\\\\').join('\\\\\\\\');

    }



    /**

     * Surrounds text with multiline quotation characters, escapes text content.

     */

    const inQuotes = (text) => {

        return "`" + termEscape(text).split('`').join('\\\\`') + "`";

    };



    /**

     * Print [content] with proper indentation.

     */

    const outputLine = (content) => {

        println(indent(content));

    };



    const startHighlightJS = () => {

        highlightJSLevel ++;

        if (highlightJSLevel == 1) {

            println('(((highlight js ');

        }

    };



    const stopHighlightJS = () => {

        highlightJSLevel --;

        if (highlightJSLevel == 0) {

            print(')))');

        }

    };



    /* If an object, list its properties */

    if (recurseDepth >= 0 && typeof(obj) == 'object') {

        // Highlight object outputs as JavaScript.

        startHighlightJS();



        print('' + obj);

        println(" {");



        let propDescriptions = [];



        for (let prop in obj) {

            print(indent(prop + ": ", "  "));



            /* Permission errors may prevent us from accessing/enumerating properties of obj[prop]. */

            try {

                _jsi_print_obj(obj[prop], recurseDepth - 1, indentation + "  ", true);

            } catch(e) {

                print("(error " + e + ")");

            }



            println(",");

        }



        outputLine("}");



        stopHighlightJS();

    } else if (typeof (obj) == "string") {

        const quoted = inQuotes(obj);

        const lines = quoted.split('\\n');



        if (quoted.length > 100 && recurseDepth < 0) {

            print(quoted.substring(0, 100) + ' `...');

            return;

        }



        if (lines.length == 1) {

            print(quoted);

        } else {

            if (obj.search(/[<][\\/][a-zA-Z0-9 \\t]+[>]/) >= 0 && recurseDepth >= 0) {

                stopHighlightJS();



                println('(((highlight html ');



                for (const line of lines) {

                    outputLine(line);

                }



                print(indent(')))'));



                startHighlightJS();

            } else {

                print(quoted);

            }

        }

    } else if (typeof (obj) == 'function') {

        if (recurseDepth < 0) {

            print("[ Function ]");

        } else {

            startHighlightJS();

            const lines = termEscape(obj + '').split('\\n');



            for (let i = 0; i < lines.length; i++) {

                if (i == 0) {

                    println(lines[i]);

                } else if (i == lines.length - 1) {

                    print(indent(lines[i]));

                } else {

                    outputLine(lines[i]);

                }

            }



            stopHighlightJS();

        }

    } else {

        print(termEscape('' + obj));

    }

}

"""



SHELL_BANNER = "Type 'help' for help and 'quit' to exit."



SHELL_HELP = """

 This program is a simple, interactive JavaScript shell.

 In addition to browser-provided APIs, the following commands are provided:

    print(message)

    println(message)

        Prints [message] to the terminal.

        `println` prints a trailing newline after printing [message].

"""



try:

    import pygments

    from pygments.lexers.javascript import JavascriptLexer

    from pygments.lexers.html import HtmlLexer

    from pygments.formatters import TerminalFormatter

    havePygments = True

except:

    havePygments = False



class SourceFormatter:

    def __init__(self, disableColor=False):

        self._disableColor = disableColor or not havePygments



    def formatHtml(self, text):

        """ Returns [text] highlighted via ASCII escape sequences """



        if self._disableColor:

            return text

        else:

            return self._highlight(text, HtmlLexer())



    def formatJavascript(self, text):

        """

            Returns [text] with ASCII escape sequences inserted to provide

        syntax highlighting

        """



        if self._disableColor:

            return text

        else:

            return self._highlight(text, JavascriptLexer()).rstrip()



    def _highlight(self, text, lexer):

        return pygments.highlight(text, lexer, TerminalFormatter())



    def formatMixedForattingRegion(self, sourceText):

        result = ''

        buff = ''

        trippleParenLevel = 0

        currentChunk = ''

        charsSinceEnteredTrippleParen = 0

        escaped = False

        acceptingHighlightType = False

        highlightType = None



        def flushBuff():

            nonlocal currentChunk, buff

            currentChunk += buff

            buff = ''



        def flushChunk():

            nonlocal result, currentChunk



            if highlightType == 'js' or highlightType == 'string':

                result += self.formatJavascript(currentChunk)

            elif highlightType == 'html':

                result += self.formatHtml(currentChunk)

            else:

                result += currentChunk



            currentChunk = ''



        for char in sourceText:

            buffstart = ''

            if len(buff) > 0:

                buffstart = buff[0]



            if acceptingHighlightType:

                if char == ' ':

                    acceptingHighlightType = False

                    flushChunk()

                    highlightType = buff.strip()

                    flushBuff()

            elif char == '(' and buffstart != '(' or char != '(' and buffstart == '(':

                if buff == '(((':

                    trippleParenLevel += 1

                    charsSinceEnteredTrippleParen = 0

                if buff == '(((' or char != '(' or buffstart != '(':

                    flushBuff()

            elif trippleParenLevel > 0 and char == ')':

                if buffstart != ')':

                    flushBuff()

                elif buff == ')))':

                    trippleParenLevel -= 1

                    if highlightType and trippleParenLevel == 0:

                        flushChunk()

                        highlightType = None

                    flushBuff()

            elif trippleParenLevel > 0 and buff == 'highlight' and char == ' ':

                if charsSinceEnteredTrippleParen == len('highlight'):

                    flushBuff()

                    acceptingHighlightType = trippleParenLevel == 1

            elif char == '`' and trippleParenLevel == 0 and not escaped:

                flushBuff()

                flushChunk()

                if highlightType == 'string':

                    highlightType = None

                else:

                    highlightType = 'string'

            elif char == '\\':

                escaped = not escaped

            elif highlightType == 'string' and escaped and char == '`':

                char = ' `'

                escaped = False

            elif escaped:

                escaped = False

            buff += char

            charsSinceEnteredTrippleParen += 1

        flushBuff()

        flushChunk()



        return result





class Printer:

    PRIMARY_COLOR="\033[94m"

    SECONDARY_COLOR="\033[33m"

    HIGHLIGHT_COLOR="\033[93m"

    NO_COLOR="\033[0m"



    def __init__(self, disableColorization):

        self._colorDisabled = disableColorization



    def print(self, text, end='\n'):

        """ Print [text] with no additional formatting. """



        self._print(text, end=end)



    def printPrimary(self, text, end='\n'):

        """ Print [text] with primary formatting. """



        self._print(text, self.PRIMARY_COLOR, end=end)



    def printSecondary(self, text, end='\n'):

        """ Print [text] with secondary formatting. """



        self._print(text, self.SECONDARY_COLOR, end=end)



    def printHighlight(self, text, end='\n'):

        """ Print [text], highlighted """



        self._print(text, self.HIGHLIGHT_COLOR, end=end)



    def _print(self, text, colorize=None, end='\n'):

        if colorize is None or self._colorDisabled:

            print(text, end=end)

        else:

            print("%s%s%s" % (colorize, text, self.NO_COLOR), end=end)





## Returns [text] within a string, line breaks, etc. escaped.

def jsQuote(text):

    result = '`'

    for char in text:

        if char == '`' or char == '\\':

            result += '\\'

        result += char

    result += '`'



    return result



## Evaluates [content] as JavaScript. If

##  [inTermContext], [content] is evaluated

##  in the same context as the terminal UI.

## Returns the output, if any, produced by [content].

def runJS(content, inTermContext=False, recurseDepth=1, sourceFormatter=SourceFormatter()):

    toRun = JS_UTILITIES

    toRun += "_jsi_print_obj(eval(%s), %d);" % (jsQuote(content), recurseDepth)



    outFile = tempfile.NamedTemporaryFile(delete=False)

    outFile.write(bytes(toRun, "utf-8"))

    outFile.close()



    args = ["jsc"]

    if inTermContext:

        args.append("--in-window")

    args.append(os.path.relpath(outFile.name))



    output = subprocess.check_output(args)

    output = output.decode("utf-8").rstrip(" \r\n")



    os.unlink(outFile.name)

    return sourceFormatter.formatMixedForattingRegion(output)



## Parse commandline arguments and take action on them, if necessary.

## Returns a map with the following keys:

## {

##  prompt: The prompt to use when interacting with the user.

##  omitIntro: False iff a brief introductory help message should be printed.

##  inTermWebView: True iff scripts should be run in the same WkWebView as the terminal UI.

##  noColor: True iff output should have no color formatting.

## }

def parseCmdlineArguments():

    args = ArgumentParser(

            description="An interactive JavaScript shell",

            epilog=LONG_DESCRIPTION

    )



    isTTY = True

    omitIntro = False

    defaultPrompt = "% "



    # If we're getting input from a pipe (i.e. cat foo.js | jsi)

    # don't print a prompt (by default).

    if not sys.stdin.isatty() or not sys.stdout.isatty():

        defaultPrompt = ""

        omitIntro = True

        isTTY = False



    args.add_argument("--prompt", default=defaultPrompt)

    args.add_argument("--no-banner",

            default=omitIntro, action="store_true", dest="omitIntro")

    args.add_argument("--no-color",

            default=not isTTY,

            dest="noColor", action="store_true", help="Disable colorized output")

    args.add_argument("-d", "--inspect-depth",

            default=0,

            type=int,

            help="When printing the details of an object, how many levels of sub-objects to inspect. This should be a small number (e.g. 0, 1, or 2).",

            dest="inspectDepth")

    args.add_argument("-w", "--in-window",

            action="store_true", dest="inTermWebView",

            help="Run JavaScript in the same WkWebView as the terminal UI.")



    result = args.parse_args()



    if not "omitIntro" in result:

        result.omitIntro = omitIntro



    return result



## Gets user input from stdin.

## Throws EOFError on EOF.

def getInput():

    result = ""



    emptylineCount = 0

    bracketLevel = 0

    parenLevel = 0

    inQuote = None

    inComment = False

    escaped = False



    firstLine = True



    while bracketLevel > 0 or parenLevel > 0 or inQuote or inComment or escaped or firstLine:

        firstLine = False

        escaped = False



        line = str(input()).lstrip()



        if line == "":

            emptylineCount += 1

        else:

            emptylineCount = 0



        # If the user enters more than two empty lines, stop.

        # Assume an error in previous entries.

        if emptylineCount > 2:

            break



        previousChar = ''

        for char in line:

            if escaped:

                escaped = False

                previousChar = ''

                continue

            if not inComment and (

                    # If the character could start/end a string,

                    char == '"' or char == "'" or char == '`'

                        or (char == '/' and previousChar != '/')

                    ):

                if inQuote == char:

                    inQuote = None

                elif inQuote is None:

                    inQuote = char

            elif inQuote:

                pass

            elif char == '*' and previousChar == '/':

                inComment = True

            elif char == '/' and previousChar == '*':

                inComment = False

            elif inComment:

                pass

            # If in a single-line comment, stop immediately.

            # We don't care about any input after the start

            # of a single-line comment.

            elif char == '/' and previousChar == '/':

                break

            elif char == '(':

                parenLevel += 1

            elif char == ')':

                parenLevel -= 1

            elif char == '{':

                bracketLevel += 1

            elif char == '}':

                bracketLevel -= 1

            elif char == '\\':

                escaped = not escaped

            previousChar = char

        if not inQuote:

            line = line.rstrip()

        result += line + '\n'

    return result



if __name__ == "__main__":

    args = parseCmdlineArguments()

    promptText = args.prompt

    response = ''



    out = Printer(args.noColor)

    formatter = SourceFormatter(args.noColor)



    if not args.omitIntro:

        out.printPrimary(SHELL_BANNER)



    while True:

        out.print(promptText, end='')

        try:

            response = getInput().strip()

        except EOFError:

            print()

            break



        if response == "quit":

            break

        elif response == "help":

            out.printPrimary(SHELL_HELP)

        else:

            out.print(runJS(response, args.inTermWebView, args.inspectDepth, formatter))

It would be great to have this in a-Shell! I'm not an iOS developer, so I'm not comfortable opening a pull request.

@personalizedrefrigerator I would love that feature as well!

[personalizedrefrigerator]

a-Shell seems to come with prompt_toolkit! I might be able to use this to add auto-complete to jsi!

[holzschu]

I've updated jsi to the latest version of the code, and I get this strange behaviour: print() adds undefined after its argument.

[personalizedrefrigerator]

That's because the expression evaluates to 'undefined'. I just checked, and the old version of 'jsi' does this, too.

If the output of a command is 'undefined', we could, alternatively not print anything, but this makes expressions like window.thisdoesnotexist print nothing...

Perhaps if an expression inputted by the user ends with a semicolon and outputs 'undefined', nothing should be printed.

[holzschu]

Ah, I see. It kinda makes sense, from JS point of view.
Maybe there is something I could change in the definition of print so that it does not evaluate to undefined?

function print(printString) {
	window.webkit.messageHandlers.aShell.postMessage('print:' + printString);
}

[personalizedrefrigerator]

Status update

• Input uses prompt_toolkit: Provides (very basic) input tab-completion and better multi-line editing.

Maybe there is something I could change in the definition of print so that it does not evaluate to undefined?

I've changed jsi such that it no longer prints undefined as a default output to fix this.

[personalizedrefrigerator]

Status Update

• Smarter tab completion
• Correctly handle KeyboardInterrupts (Ctrl + C)

[personalizedrefrigerator]

Currently, variables defined using let and const aren't accessible by following runs of JavaScript:

% let a = 6;
% a
Jsc: Error...
% var a = 12;
% a
12

A while ago, I wrote a solution to this (published here: https://github.com/personalizedrefrigerator/LibJS/blob/1d18cafa6503bc293b5ed5f592ba7ef2ec6266af/Libs/JSHelper.js#L206 ). I would prefer not to re-write it, so may include this file.

[holzschu]

In SceneDelegate.swift, I enclose the javascript code between curly braces:

a-shell/a-Shell/SceneDelegate.swift

Line 475 in e9dba50

javascript = "{" + javascript + "}"

This was done to avoid influencing the main WkWebView too much, which is not so much of an issue now that javascript is executed in a different webView. Would removing the braces help in this issue?

[personalizedrefrigerator]

Would removing the braces help in this issue?

I think so.

[holzschu]

I tried: even without the braces the issue is still present.

[personalizedrefrigerator]

To workaround the let/const declaration issue, jsi now replaces let and const within the global scope (and not in strings, comments, etc.) with var. For example, const a = 3; { let b = 6; } becomes var a = 3; { let b = 6; }.

Of course, this means that const-ness for variables declared in the shell is not respected.

The other approach mentioned above requires jsc to not exit until we have the result of a Promise. (The other approach uses nested evals and async/await; e.g.

 var codeToKeepAcceptingEvals = `
eval((await getNewCode()) + codeToKeepAcceptingEvals)`; 
(async function() { eval(codeToKeepAcceptingEvals); })();

)

[personalizedrefrigerator]

At present, print statements aren't working when delayed/when they run after jsc returns:

I'm not sure how this should be addressed...

[holzschu]

delaying things in the webView JS interpreter doesn't work as I expect in general, and it tends to treat the timeout delay as a suggestion more than a requirement.
Same with promises and async/await. Loading webAssembly code or JS modules is done without async() because it would not work otherwise.

[personalizedrefrigerator]

jsi now overrides print, println, and print_error for commands it runs to allow printed content to be collected, and prints collected content at the beginning of output from new commands (then clears the collected content):

jsi now also treats input from stdin that is not from a TTY as a single command.

[personalizedrefrigerator]

It looks like build 157 contains a version of jsi without auto-complete...

Thank you so much for helping me with this!

[holzschu]

Yes, there are a few days of delay when uploading, plus a few days of delay when I forget to copy the latest version. I'd say the version of jsi in build 157 is 3-4 days old.

[personalizedrefrigerator]

I think I'm satisfied with the current functionality of jsi, so (hopefully) am done making changes. Please let me know if there is anything you would like me to add!