Browse files

Initial commit for new public phpsh repo.

This also marks the release of phpsh 1.1.

Reviewed By: ccheever

Test Plan: Installing, running test/'s.  Has also been deployed at fb
for a week or so.

Revert Plan: ok
  • Loading branch information...
0 parents commit 9317b56fd47fc702109a6b3e6a5765ae1c109aac Daniel Corson committed Oct 13, 2008
Showing with 1,875 additions and 0 deletions.
  1. +10 −0 LICENSE
  2. +44 −0 README
  3. +40 −0 phpshrc.example.php
  4. +20 −0 setup.py
  5. +576 −0 src/__init__.py
  6. +25 −0 src/ansicolor.py
  7. +141 −0 src/cmd_util.py
  8. +87 −0 src/ctags.py
  9. +425 −0 src/html2text.py
  10. +85 −0 src/manual.py
  11. BIN src/php_manual.db
  12. +148 −0 src/phpsh
  13. +210 −0 src/phpsh.php
  14. +1 −0 src/phpsh.py
  15. +4 −0 src/phpsh_check_syntax
  16. +38 −0 src/phpshrc.php
  17. +6 −0 test/all_good.pht
  18. +15 −0 test/err_and_multiout.pht
10 LICENSE
@@ -0,0 +1,10 @@
+Copyright (c) 2006, Dan Corson, Charles Cheever, Facebook, inc.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+ * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+ * Neither the name of Facebook, inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
44 README
@@ -0,0 +1,44 @@
+To install:
+~ python setup.py build
+~ sudo python setup.py install
+(Will download and install the pysqlite dependency for you if needed.)
+
+The buildscript uses setuptools. This comes with python >= 2.5.
+If you are using older python, go grab EasyInstall:
+http://peak.telecommunity.com/DevCenter/EasyInstall#installing-easy-install
+
+To set up after install:
+- If you just want to use vanilla php, just do ~ phpsh and have fun.
+- To connect to an actual codebase, e.g.:
+ ~ cd ~/www
+ ~ ctags -R # for phpsh ctags integration, recommended
+ ~ phpsh lib/init.php # or some file(s) that load codebase libraries
+- To get autoloading, modify /etc/phpsh/phpshrc.php. After that it's just:
+ ~ cd ~/www
+ ~ phpsh
+ And for times when you just want vanilla php:
+ ~ phpsh -c none
+- For individual configuration, also see phpshrc.example.php in the php
+ distribution.
+
+Contact phpsh@facebook.com with any questions.
+
+
+Note to ppl hacking on phpsh:
+- For faster iteration, after installing once, you can run phpsh from src/
+directly without reinstalling. E.g.:
+ ~ cd ~/www
+ ~ ~/projects/phpsh/src/phpsh
+
+Todo for after phpsh 1.1
+- Simple phpsh breakpoints that you can insert into your php code.
+- Maybe phpsh_check_syntax shouldn't actually be installed as a script, and
+ just be a pkg_resource? Didn't want to worry about zip file overhead, and
+ installing didn't seem like a big deal.
+- Similarly, php_manual.db should probably go in share/ not etc/ but similarly
+ was worried about b.s. with setuptools..
+- Paging for long php> d .. results? Or is terminal scroll fine..
+- Thread loading ctags and starting php? Would speed start but not restart.
+- Command-line apc for faster php startup for large codebases.
+- Note on php start error to start from codebase place?
+- Make tab to show function signature work with multiline func sigs.
40 phpshrc.example.php
@@ -0,0 +1,40 @@
+<?php
+# copy this to ~/.phpshrc.php
+
+# load any system defaults / codebase-modes
+require_once '/etc/phpsh/phpshrc.php';
+
+# the examples here are some functions i use for easy io with the outside world
+
+define('DEFAULT_IO_FILE', getenv('HOME').'/o');
+
+/**
+ * append array or var to ~/o
+ * @author dcorson
+ */
+function o($x, $fn = DEFAULT_IO_FILE) {
+ $f = fopen($fn, 'a');
+ if (is_array($x)) {
+ fwrite($f, implode("\n", $x)."\n");
+ } else {
+ fwrite($f, $x."\n");
+ }
+ fclose($f);
+ return true;
+}
+
+/**
+ * strip last char (typically used to kill "\n") from line
+ * @author dcorson
+ */
+function _rstrip($l) {
+ return substr($l, 0, strlen($l) - 1);
+}
+
+/**
+ * read array from ~/o
+ * @author dcorson
+ */
+function i($fn = DEFAULT_IO_FILE) {
+ return array_map('_rstrip', file($fn));
+}
20 setup.py
@@ -0,0 +1,20 @@
+#!/usr/bin/env python
+from setuptools import setup
+
+setup(
+ name='phpsh',
+ version='1.1',
+ description='interactive shell into a php codebase',
+ author='facebook',
+ author_email='phpsh@facebook.com',
+ url='http://www.phpsh.org/',
+ packages=['phpsh'],
+ package_dir={'phpsh': 'src'},
+ package_data={'phpsh': ['*.php']},
+ scripts=['src/phpsh', 'src/phpsh_check_syntax'],
+ data_files=[
+ ('/etc/phpsh', ['src/phpshrc.php']),
+ ('/etc/phpsh', ['src/php_manual.db']),
+ ],
+ install_requires=['pysqlite'],
+)
576 src/__init__.py
@@ -0,0 +1,576 @@
+from subprocess import Popen, PIPE
+import ansicolor as clr
+import cmd_util as cu
+import os
+import re
+import readline
+import select
+import sys
+import tempfile
+import time
+
+comm_poll_timeout = 0.01
+
+def help_message():
+ return """\
+-- Help --
+Type php commands and they will be evaluted each time you hit enter. Ex:
+php> $msg = "hello world"
+
+Put = at the beginning of a line as syntactic sugar for return. Ex:
+php> = 2 + 2
+4
+
+phpsh will print any returned value (in yellow) and also assign the last
+returned value to the variable $_. Anything printed to stdout shows up blue,
+and anything sent to stderr shows up red.
+
+You can enter multiline input, such as a multiline if statement. phpsh will
+accept further lines until you complete a full statement, or it will error if
+your partial statement has no syntactic completion. You may also use ^C to
+cancel a partial statement.
+
+You can use tab to autocomplete function names, global variable names,
+constants, classes, and interfaces. If you are using ctags, then you can hit
+tab again after you've entered the name of a function, and it will show you
+the signature for that function. phpsh also supports all the normal
+readline features, like ctrl-e, ctrl-a, and history (up, down arrows).
+
+Note that stdout and stderr from the underlying php process are line-buffered;
+so php> for ($i = 0; $i < 3; $i++) {echo "."; sleep(1);}
+will print the three dots all at once after three seconds.
+(echo ".\n" would print one a second.)
+
+See phpsh -h for invocation options.
+
+-- phpsh quick command list --
+ h Display this help text.
+ r Reload (e.g. after a code change). args to r append to add
+ includes, like: php> r ../lib/username.php
+ (use absolute paths or relative paths from where you start phpsh)
+ R Like 'r', but change includes instead of appending.
+ d Get documentation for a function or other identifier.
+ ex: php> d my_function
+ D Like 'd', but gives more extensive documentation for builtins.
+ v Open vim read-only where a function or other identifer is defined.
+ ex: php> v some_function
+ V Open vim (not read-only) and reload (r) upon return to phpsh.
+ e Open emacs where a function or other identifer is defined.
+ ex: php> e some_function
+ c Append new includes without restarting; display includes.
+ C Change includes without restarting; display includes.
+ ! Execute a shell command.
+ ex: php> ! pwd
+ q Quit (ctrl-D also quits)
+"""
+
+def do_sugar(line):
+ line = line.lstrip()
+ if line.startswith("="):
+ line = "return " + line[1:]
+ if line:
+ line += ";"
+ return line
+
+def line_encode(line):
+ return cu.multi_sub({'\n': '\\n', '\\': '\\\\'}, line) + "\n"
+
+def inc_args(s):
+ """process a string of includes to a set of them"""
+ return set([inc.strip() for inc in s.split(" ") if inc.strip()])
+
+class PhpMultiliner:
+ """This encapsulates the process and state of intaking multiple input lines
+ until a complete php expression is formed, or (hopefully eventually..)
+ detecting a syntax error.
+
+ Note: this is not perfectly encapsulated while the parser has global state
+ """
+
+ complete = "complete"
+ incomplete = "incomplete"
+ syntax_error = "syntax_error"
+
+ def __init__(self):
+ self.partial = ""
+
+ def check_syntax(self, line):
+ p = Popen(["phpsh_check_syntax", line], stderr=PIPE)
+ p.wait()
+ l = p.stderr.readline()
+ if l.find('syntax error') != -1:
+ if l.find('unexpected $end') != -1:
+ return (self.incomplete, l)
+ return (self.syntax_error, l)
+ return (self.complete, l)
+
+ def input_line(self, line):
+ if self.partial:
+ self.partial += "\n"
+ self.partial += line
+ partial_mod = do_sugar(self.partial)
+ if not partial_mod:
+ return (self.complete, "")
+
+ # There is a terrible bug in php/eval where and unclosed ' only creates
+ # a warning! (" and ` correctly give syntax errors.)
+ # So we have to explicitly and hackily check for this error..
+ #
+ # If this does _not_ error, then you have an unclosed '
+ may_be_right = True
+ (syntax_info, result_str) = self.check_syntax(partial_mod + ";())';")
+ if syntax_info == self.complete:
+ may_be_right = False
+
+ if may_be_right:
+ (syntax_info, result_str) = self.check_syntax(partial_mod)
+ if syntax_info == self.complete:
+ # multiline inputs are encoded to one line
+ partial_mod = line_encode(partial_mod)
+ self.clear()
+ return (syntax_info, partial_mod)
+ # need to pull off syntactic sugar ; to see if line failed the syntax
+ # check because of syntax_error, or because of incomplete
+ return self.check_syntax(partial_mod[:-1])
+
+ def clear(self):
+ self.partial = ""
+
+class ProblemStartingPhp(Exception):
+ def __init__(self, file_name = None, line_num = None):
+ self.file_name = file_name
+ self.line_num = line_num
+
+class PhpshState:
+ """This doesn't perfectly encapsulate state (e.g. the readline module has
+ global state), but it is a step in the
+ right direction and it already fulfills its primary objective of
+ simplifying the notion of throwing a line of input (possibly only part of a
+ full php line) at phpsh.
+ """
+
+ phpsh_root = os.path.dirname(os.path.realpath(__file__))
+
+ php_prompt = "php> "
+ php_more_prompt = " ... "
+
+ no_command = "no_command"
+ yes_command = "yes_command"
+ quit_command = "quit_command"
+
+ def __init__(self, cmd_incs, do_color, do_echo, codebase_mode,
+ do_autocomplete, do_ctags, interactive):
+ """start phpsh.php and do other preparations (colors, ctags)
+ """
+
+ self.do_echo = do_echo
+ self.temp_file_name = tempfile.mktemp()
+ self.comm_base = "php " + self.phpsh_root + "/phpsh.php " + \
+ self.temp_file_name + " " + cu.arg_esc(codebase_mode)
+ if not do_color:
+ self.comm_base += " -c"
+ if not do_autocomplete:
+ self.comm_base += " -A"
+ self.cmd_incs = cmd_incs
+
+ # so many colors, so much awesome
+ if not do_color:
+ self.clr_cmd = ""
+ self.clr_err = ""
+ self.clr_help = ""
+ self.clr_announce = ""
+ self.clr_default = ""
+ else:
+ self.clr_cmd = clr.Green
+ self.clr_err = clr.Red
+ self.clr_help = clr.Green
+ self.clr_announce = clr.Magenta
+ self.clr_default = clr.Default
+
+ # ctags integration
+ self.ctags = None
+ if do_ctags and os.path.isfile("tags"):
+ print self.clr_cmd + "Loading ctags" + self.clr_default
+ try:
+ import ctags
+ self.ctags = ctags.Ctags()
+ try:
+ self.function_signatures = \
+ ctags.CtagsFunctionSignatures().function_signatures
+ except Exception, e:
+ self.function_signatures = {}
+ print self.clr_err + \
+ "Problem loading function signatures" + \
+ self.clr_default
+ except Exception, e:
+ print self.clr_err + "Problem loading ctags" + self.clr_default
+
+ import rlcompleter
+ input_rc_file = os.path.join(os.environ["HOME"], ".inputrc")
+ if os.path.isfile(input_rc_file):
+ readline.read_init_file(input_rc_file)
+ readline.parse_and_bind("tab: complete")
+
+ # persistent readline history
+ # we set the history length to be something reasonable
+ # so that we don't write a ridiculously huge file every time
+ # someone executes a command
+ self.history_file = os.path.join(os.environ["HOME"], ".phpsh.history")
+ readline.set_history_length(100)
+
+ try:
+ readline.read_history_file(self.history_file)
+ except IOError:
+ # couldn't read history (probably one hasn't been created yet)
+ pass
+
+ self.autocomplete_identifiers = []
+ self.autocomplete_cache = None
+ self.autocomplete_match = None
+ self.autocomplete_signature = None
+
+ self.show_incs(start=True)
+ self.php_open_and_check()
+
+ def tab_complete(text, state):
+ """The completer function is called as function(text, state),
+ for state in 0, 1, 2, ..., until it returns a non-string value."""
+
+ size = len(text)
+ if size == 0:
+ # currently there is a segfault in readline when you complete
+ # on nothing. so just don't allow completing on that for now.
+ # in the long term, we may use ipython's prompt code instead
+ # of readline
+ return None
+ if state == 0:
+ self.autocomplete_cache = []
+ for identifier in self.autocomplete_identifiers:
+ if identifier[0:size] == text:
+ self.autocomplete_cache.append(identifier)
+
+ if self.function_signatures.has_key(text):
+ for sig in self.function_signatures[text]:
+ self.autocomplete_cache.append(sig)
+ try:
+ return self.autocomplete_cache[state]
+ except IndexError:
+ return None
+
+ readline.set_completer(tab_complete)
+
+ # print welcome message
+ if interactive:
+ print self.clr_help + \
+ "type 'h' or 'help' to see instructions & features" + \
+ self.clr_default
+
+ def do_expr(self, expr):
+ self.p.stdin.write(expr)
+ self.wait_for_comm_finish()
+ return self.result
+
+ def wait_on_ready(self):
+ while True:
+ a = self.comm_file.readline()
+ if a:
+ break
+ time.sleep(comm_poll_timeout)
+
+ def php_open_and_check(self):
+ self.p = None
+ while not self.p:
+ try:
+ self.php_open()
+ except ProblemStartingPhp, e:
+ print self.clr_cmd + """phpsh failed to initialize PHP.
+Fix the problem and hit enter to reload or ctrl-C to quit."""
+ if e.line_num:
+ print "Type V to vim to %s: %s" % (e.file_name, e.line_num)
+ print self.clr_default
+ if raw_input() == "V":
+ Popen("vim +" + str(e.line_num) + " " + e.file_name,
+ shell=True).wait()
+ else:
+ print self.clr_default
+ raw_input()
+ # this file is how phpsh.php tells us it is done with a command
+ self.comm_file = open(self.temp_file_name)
+ self.wait_on_ready()
+ self.wait_for_comm_finish()
+
+ def php_restart(self):
+ self.initialized_successfully = False
+ try:
+ self.p.stdout.close()
+ self.p.stderr.close()
+ self.p.stdin.close()
+ self.p.wait()
+ except IOError:
+ pass
+
+ return self.php_open_and_check()
+
+ def php_open(self):
+ self.autocomplete_identifiers = []
+ cmd = " ".join([self.comm_base] + list(self.cmd_incs))
+ self.p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE,
+ preexec_fn=os.setsid)
+ p_line = self.p.stdout.readline().rstrip()
+ if p_line != "#start_autocomplete_identifiers":
+ err_lines = self.p.stderr.readlines();
+ if len(err_lines) >= 1:
+ err_str = err_lines[-1].rstrip()
+ else:
+ err_str = "UNKNOWN ERROR (maybe php build does not support signals/tokenizer?)"
+ print self.clr_err + err_str + self.clr_default
+ m = re.match("PHP Parse error: .* in (.*) on line ([0-9]*)", err_str)
+ if m:
+ file_name, line_num = m.groups()
+ raise ProblemStartingPhp(file_name, line_num)
+ else:
+ raise ProblemStartingPhp()
+ while True:
+ p_line = self.p.stdout.readline().rstrip()
+ if p_line == "#end_autocomplete_identifiers":
+ break
+ self.autocomplete_identifiers.append(p_line)
+
+ def wait_for_comm_finish(self):
+ try:
+ # wait for signal that php command is done
+ # keep checking for death
+ out_buff = ["", ""]
+ buffer_size = 4096
+ self.result = ""
+ died = False
+
+ debug = False
+ #debug = True
+
+ while True:
+ if debug:
+ print 'polling'
+ ret_code = self.p.poll()
+ if debug:
+ print 'ret_code: ' + str(ret_code)
+ if ret_code != None:
+ if debug:
+ print 'NOOOOO'
+ died = True
+ break
+ while not died:
+ # line-buffer stdout and stderr
+ if debug:
+ print 'start loop'
+ s = select.select([self.p.stdout, self.p.stderr], [], [],
+ comm_poll_timeout)
+ if s == ([], [], []):
+ if debug:
+ print 'empty'
+ break
+ if debug:
+ print s[0]
+ for r in s[0]:
+ if r is self.p.stdout:
+ out_buff_i = 0
+ else:
+ out_buff_i = 1
+ buff = os.read(r.fileno(), buffer_size)
+ if not buff:
+ # process has died
+ died = True
+ break
+ out_buff[out_buff_i] += buff
+ last_nl_pos = out_buff[out_buff_i].rfind('\n')
+ if last_nl_pos != -1:
+ l = out_buff[out_buff_i][:last_nl_pos + 1]
+ self.result += l
+ if self.do_echo:
+ if r is self.p.stdout:
+ sys.stdout.write(l)
+ else:
+ l = self.clr_err + l + self.clr_default
+ sys.stderr.write(l)
+ out_buff[out_buff_i] = out_buff[out_buff_i][last_nl_pos + 1:]
+ # don't sleep if the command is already done
+ # (even tho sleep period is small; maximize responsiveness)
+ if self.comm_file.readline():
+ break
+ time.sleep(comm_poll_timeout)
+
+ if died:
+ self.show_incs("PHP died. ")
+ self.php_open_and_check()
+
+ except KeyboardInterrupt:
+ self.show_incs("Interrupt! ")
+ self.php_restart()
+
+ def show_incs(self, pre_str="", restart=True, start=False):
+ s = self.clr_cmd + pre_str
+ inc_str = str(list(self.cmd_incs))
+ if start or restart:
+ if start:
+ start_word = "Starting"
+ else:
+ start_word = "Restarting"
+ if self.cmd_incs:
+ s += start_word + " php with extra includes: " + inc_str
+ else:
+ s += start_word + " php"
+ else:
+ s += "Extra includes are: " + inc_str
+ print s + self.clr_default
+
+ def try_command(self, line):
+ if line == "r" or line.startswith("r "):
+ # add args to phpsh.php (includes), reload
+ self.cmd_incs = self.cmd_incs.union(inc_args(line[2:]))
+ self.show_incs()
+ self.php_restart()
+ elif line == "R" or line.startswith("R "):
+ # change args to phpsh.php (includes), reload
+ self.cmd_incs = inc_args(line[2:])
+ self.show_incs()
+ self.php_restart()
+ elif line == "c" or line.startswith("c "):
+ # add args to phpsh.php (includes)
+ self.cmd_incs = self.cmd_incs.union(inc_args(line[2:]))
+ self.show_incs(restart=False)
+ self.p.stdin.write("\n")
+ elif line == "C" or line.startswith("C "):
+ # change args to phpsh.php (includes)
+ self.cmd_incs = inc_args(line[2:])
+ self.show_incs(restart=False)
+ self.p.stdin.write("\n")
+ elif line.startswith("d ") or line.startswith("D "):
+ identifier = line[2:]
+ if identifier.startswith("$"):
+ identifier = identifier[1:]
+
+ print self.clr_help
+
+ lookup_tag = False
+ ctags_error = "ctags not enabled"
+ try:
+ if self.ctags:
+ tags = self.ctags.py_tags[identifier]
+ ctags_error = None
+ lookup_tag = True
+ except KeyError:
+ ctags_error = "no ctag info found for '" + identifier + "'"
+ if lookup_tag:
+ print repr(tags)
+ for t in tags:
+ try:
+ file = self.ctags.tags_root + os.path.sep + t["file"]
+ doc = ""
+ append = False
+ line_num = 0
+ for line in open(file):
+ line_num += 1
+ if not append:
+ if line.find("/*") != -1:
+ append = True
+ doc_start_line = line_num
+ if append:
+ if line.find(t["context"]) != -1:
+ print "%s, lines %d-%d:" % (file, doc_start_line, line_num)
+ print doc
+ break
+ if line.find("*") == -1:
+ append = False
+ doc = ""
+ else:
+ doc += line
+ except:
+ pass
+ import manual
+ manual_ret = manual.get_documentation_for_identifier(identifier,
+ short=line.startswith("d "))
+ if manual_ret:
+ print manual_ret
+ if not manual_ret and ctags_error:
+ print "could not find in php manual and " + ctags_error
+ print self.clr_default
+ elif line.startswith("v "):
+ self.editor_tag(line[2:], "vim", read_only=True)
+ elif line.startswith("V "):
+ self.editor_tag(line[2:], "vim")
+ elif line.startswith("e "):
+ self.editor_tag(line[2:], "emacs")
+ elif line.startswith("!"):
+ # shell command
+ Popen(line[1:], shell=True).wait()
+ elif line == "h" or line == "help":
+ print self.clr_help + help_message() + self.clr_default
+ elif line == "q" or line == "exit" or line == "exit;":
+ return self.quit_command
+ else:
+ return self.no_command
+ return self.yes_command
+
+ def editor_tag(self, tag, editor, read_only=False):
+ if tag.startswith("$"):
+ tag = tag[1:]
+
+ def not_found():
+ print self.clr_cmd + "no tag '" + tag + "' found" + self.clr_default
+ self.p.stdin.write("\n")
+
+ if not self.ctags.py_tags.has_key(tag):
+ not_found()
+ return
+
+ if editor == "emacs":
+ t = self.ctags.py_tags[tag][0]
+ # get line number (or is there a way to start emacs at a
+ # particular tag location?)
+ try:
+ file = self.ctags.tags_root + os.path.sep + t["file"]
+ doc = ""
+ append = False
+ line_num = 1
+ found_tag = False
+ for line in open(file):
+ line_num += 1
+ if line.find(t["context"]) != -1:
+ emacs_line = line_num
+ found_tag = True
+ break
+ except:
+ pass
+ if found_tag:
+ # -nw opens it in the terminal instead of using X
+ cmd = "emacs -nw +%d %s" % (emacs_line, file)
+ p_emacs = Popen(cmd, shell=True)
+ p_emacs.wait()
+ self.p.stdin.write("\n")
+ else:
+ not_found()
+ return
+ else:
+ if read_only:
+ vim = "vim -R"
+ else:
+ vim = "vim"
+ vim += ' -c "set tags=' + self.ctags.tags_file + '" -t '
+ p_vim = Popen(vim + tag, shell=True)
+ p_vim.wait()
+ self.p.stdin.write("\n")
+ if not read_only:
+ self.show_incs()
+ self.php_open_and_check()
+
+ def write(self):
+ try:
+ readline.write_history_file(self.history_file)
+ except IOError:
+ print >> sys.stderr, \
+ "Could not write history file. No write permissions?"
+
+ def close(self):
+ self.write()
+ print self.clr_default
+ os.remove(self.temp_file_name)
25 src/ansicolor.py
@@ -0,0 +1,25 @@
+# ansi escape sequences to create terminal colors
+
+Default = '\033[0m'
+
+Black = '\033[30m'
+Red = '\033[31m'
+Green = '\033[32m'
+Yellow = '\033[33m'
+Blue = '\033[34m'
+Magenta = '\033[35m'
+Cyan = '\033[36m'
+White = '\033[37m'
+
+Reset = '\033[0;0m'
+Bold = '\033[1m'
+Reverse = '\033[2m'
+
+Blackbg = '\033[40m'
+Redbg = '\033[41m'
+Greenbg = '\033[42m'
+Yellowbg = '\033[43m'
+Bluebg = '\033[44m'
+Magentabg = '\033[45m'
+Cyanbg = '\033[46m'
+Whitebg = '\033[47m'
141 src/cmd_util.py
@@ -0,0 +1,141 @@
+# utility functions for running shell commands
+
+import re
+import subprocess as spc
+import threading as thg
+import sys
+import time
+
+PIPE = spc.PIPE
+
+def multi_sub(reps, s):
+ """multi-substitute: do replacements (dictionary of strings to strings)
+ simultaneously
+ """
+ i = 0
+ ret = ""
+ for m in re.finditer("(" + "|".join(map(re.escape, reps.keys())) + ")", s):
+ ret += s[i:m.start()] + reps[m.group()]
+ i = m.end()
+ ret += s[i:]
+ return ret
+
+# the simpler way:
+# s.replace("\\", "\\\\").replace("'", "'\\''")
+# does _not_ work when there are backslashes on the scene:
+# you need simultaneous substitution bc \ inside ' is weird in shell-scripting
+# (this really does come up, such double escaping for dsh e.g.)
+arg_esc_subs = {
+ "'": "'\\''",
+ "\\": "'\\\\'"
+}
+
+def arg_esc(s):
+ """escape an argument for a shell command"""
+ return "'" + multi_sub(arg_esc_subs, s) + "'"
+
+def args_esc(ss):
+ """escape arguments for a shell command"""
+ return tuple([arg_esc(s) for s in ss])
+
+def error_out(s):
+ raise Exception(s + " ..aborting!")
+
+def cmd_run(cmd, shell=True, stdout=None, stdin=None, stderr=None):
+ """
+ run a command, applying escaping properly on array.
+ basically this does what Popen should do already
+ (Popen is wontfix busted on unix for passing multiple args as array,
+ since it just does bash -c "$@" which silently kills args. pretty lame)
+ """
+ if type(cmd) == type([]):
+ cmd = " ".join([arg_esc(a) for a in cmd])
+ return spc.Popen(cmd, shell=shell, stdout=stdout, stdin=stdin,
+ stderr=stderr)
+
+def cmd_wait(cmd, shell=True, stdout=None, stderr=None, can_fail=False):
+ """wait for a command to finish and return the subprocess object"""
+ p = cmd_run(cmd, shell=shell, stdout=stdout, stderr=stderr)
+ ret = p.wait()
+ if not can_fail and ret != 0:
+ error_out("got ret " + str(ret) + " from command:\n" + str(cmd))
+ return p
+
+def cmd_output(cmd, can_fail=False):
+ """wait for a command to finish and return the output"""
+ ls = cmd_wait(cmd, stdout=spc.PIPE, can_fail=can_fail).stdout.readlines()
+ return [l[:-1] for l in ls]
+
+def are_you_sure(timeout=5, msg=""):
+ print "WARNING: YOU ARE PERFORMING A POTENTIALLY DANGEROUS ACTION"
+ if msg:
+ print
+ print msg
+ print
+ print "waiting " + str(timeout) + " seconds before continuing."
+ print "^C TO CANCEL"
+ timeout_list = range(1, timeout + 1)
+ timeout_list.reverse()
+ for i in timeout_list:
+ print i
+ time.sleep(1)
+
+def try_rep(n, cmd):
+ """try a command repeatedly until it works,
+ erroring after some number of failures
+ """
+ for i in xrange(n):
+ if i > 0:
+ print >> sys.stderr, "retrying " + cmd
+ ret = spc.Popen(cmd, shell=True).wait()
+ if ret == 0:
+ return ret
+ print >> sys.stderr, "***** FAILED ***** (with ret %d): %s" % (ret, cmd)
+ return ret
+
+default_fanout = 5
+# note this to retry the ssh connection/execution
+# not to retry your command, e.g.: dsh(hosts, 'false') will only be tried
+# once on each host, even though false "fails" every time it is run
+default_retry_num = 10
+
+def dsh(hosts, cmd, fanout=default_fanout, retry_num=default_retry_num):
+ """runs a command on several boxes via ssh"""
+ def my_cmd_fcn(x):
+ return "ssh " + x + " " + cu.arg_esc(cmd)
+ dsh_fcn(hosts, my_cmd_fcn, fanout, retry_num)
+
+def dsh_fcn(hosts, cmd_fcn, fanout=default_fanout,
+ retry_num=default_retry_num):
+ """very generic form of dsh function"""
+ print "Runinng " + cu.arg_esc(cmd_fcn("<x>")) + ":" + \
+ "\n- with fanout " + str(fanout) + " and retry " + str(retry_num) + \
+ "\n- on <x> in " + str(hosts)
+
+ fanout_sema = thg.BoundedSemaphore(value=fanout)
+
+ class ShellCommThread(thg.Thread):
+ def __init__(self, cmd, retry_num):
+ thg.Thread.__init__(self)
+ self.cmd = cmd
+ self.retry_num = retry_num
+ def run(self):
+ fanout_sema.acquire()
+ try:
+ spc.Popen(self.cmd, shell=True).wait()
+ finally:
+ fanout_sema.release()
+
+ threads = []
+ for host in hosts:
+ t = ShellCommThread(cmd_fcn(host), retry_num)
+ threads.append(t)
+ t.start()
+ for t in threads:
+ t.join()
+
+def sql_esc(s):
+ return '"' + s.replace('"', '\"').replace('\\', '\\\\') + '"'
+
+def sql_like_esc(s):
+ return s.replace('%', '\\%').replace('_', '\\_')
87 src/ctags.py
@@ -0,0 +1,87 @@
+import re # regular expressions used for parsing the tags file
+import os # to get the current working directory
+import os.path
+
+__context_regular_expression = re.compile(r".*\t\/\^(.*)\$\/;\"\t.*")
+
+class CantFindTagsFile(Exception):
+ pass
+
+class Ctags:
+ def __init__(self, tags_file_path=None):
+ if not tags_file_path:
+ tags_file_path = find_tags_file()
+ self.tags_file = tags_file_path
+ self.tags_root = os.path.dirname(tags_file_path)
+ self.py_tags = parse_tags_file(tags_file_path)
+
+class CtagsFunctionSignatures:
+ def __init__(self, tags_file_path=None):
+ if not tags_file_path:
+ tags_file_path = find_tags_file()
+ self.tags_file = tags_file_path
+ self.tags_root = os.path.dirname(tags_file_path)
+ self.function_signatures = parse_function_signatures(tags_file_path)
+
+def find_tags_file(dir=None, tags_file_name="tags"):
+ "looks for a file in the current or given directory and all parent directories of that directory up until the root"
+ file_in_dir = lambda d: d + os.path.sep + tags_file_name
+ if not dir:
+ dir = os.getcwd()
+ while not os.path.isfile(file_in_dir(dir)):
+ dir = os.path.normpath(dir)
+ if dir == os.path.sep:
+ raise CantFindTagsFile
+ dir += os.path.sep + os.path.pardir
+
+ return file_in_dir(os.path.normpath(dir))
+
+def parse_tags_file(tags_file_path):
+ "parses a tags file generated by ctags, returns a dict of identifiers with info about them"
+ tags_file = open(tags_file_path)
+ py_tags = {}
+ for line in tags_file:
+ if line[:6] == '!_TAG_': # tag program metadata
+ continue
+ cols = line.rstrip().split("\t")
+ identifier = cols[0]
+ file_path = cols[1]
+ type = cols[-1] # type is the last field
+ try:
+ (context, ) = __context_regular_expression.match(line).groups()
+ except ValueError:
+ continue
+ except AttributeError:
+ continue
+
+ if not py_tags.has_key(identifier):
+ py_tags[identifier] = []
+ py_tags[identifier].append({"file": file_path, "context": context, "type": type})
+
+ return py_tags
+
+__function_args_regular_expression = re.compile(r".*function\s+([^\);]*[\)]*)")
+def parse_function_signatures(tags_file_path):
+ tags_file = open(tags_file_path)
+ functions_to_signatures = {}
+ for line in tags_file:
+ if line[:6] == '!_TAG_' or line.rstrip()[-1] != "f":
+ continue
+ try:
+ context = __context_regular_expression.match(line).groups()[0]
+ signature = __function_args_regular_expression.match(context).groups()[0]
+ except ValueError:
+ continue
+ except AttributeError:
+ continue
+ except IndexError:
+ continue
+
+ cols = line.rstrip().split("\t")
+ identifier = cols[0]
+ try:
+ functions_to_signatures[identifier].append(signature)
+ except KeyError:
+ functions_to_signatures[identifier] = [signature]
+
+ return functions_to_signatures
425 src/html2text.py
@@ -0,0 +1,425 @@
+"""html2text: Turn HTML into equivalent Markdown-structured text."""
+__version__ = "2.24"
+__author__ = "Aaron Swartz (me@aaronsw.com)"
+__copyright__ = "(C) 2004 Aaron Swartz. GNU GPL 2."
+__contributors__ = ["Martin 'Joey' Schulze", "Ricardo Reyes"]
+
+# TODO:
+# Support decoded entities with unifiable.
+# Relative URL resolution
+
+if not hasattr(__builtins__, 'True'): True, False = 1, 0
+import re, sys, urllib, htmlentitydefs, codecs, StringIO, types
+import sgmllib
+sgmllib.charref = re.compile('&#([xX]?[0-9a-fA-F]+)[^0-9a-fA-F]')
+
+try: from textwrap import wrap
+except: pass
+
+# Use Unicode characters instead of their ascii psuedo-replacements
+UNICODE_SNOB = 0
+
+# Put the links after each paragraph instead of at the end.
+LINKS_EACH_PARAGRAPH = 0
+
+# Wrap long lines at position. 0 for no wrapping. (Requires Python 2.3.)
+BODY_WIDTH = 0
+
+### Entity Nonsense ###
+
+def name2cp(k):
+ if k == 'apos': return ord("'")
+ if hasattr(htmlentitydefs, "name2codepoint"): # requires Python 2.3
+ return htmlentitydefs.name2codepoint[k]
+ else:
+ k = htmlentitydefs.entitydefs[k]
+ if k.startswith("&#") and k.endswith(";"): return int(k[2:-1]) # not in latin-1
+ return ord(codecs.latin_1_decode(k)[0])
+
+unifiable = {'rsquo':"'", 'lsquo':"'", 'rdquo':'"', 'ldquo':'"',
+'copy':'(C)', 'mdash':'--', 'nbsp':' ', 'rarr':'->', 'larr':'<-', 'middot':'*',
+'ndash':'-', 'oelig':'oe', 'aelig':'ae',
+'agrave':'a', 'aacute':'a', 'acirc':'a', 'atilde':'a', 'auml':'a', 'aring':'a',
+'egrave':'e', 'eacute':'e', 'ecirc':'e', 'euml':'e',
+'igrave':'i', 'iacute':'i', 'icirc':'i', 'iuml':'i',
+'ograve':'o', 'oacute':'o', 'ocirc':'o', 'otilde':'o', 'ouml':'o',
+'ugrave':'u', 'uacute':'u', 'ucirc':'u', 'uuml':'u'}
+
+unifiable_n = {}
+
+for k in unifiable.keys():
+ unifiable_n[name2cp(k)] = unifiable[k]
+
+def charref(name):
+ if name[0] in ['x','X']:
+ c = int(name[1:], 16)
+ else:
+ c = int(name)
+
+ if not UNICODE_SNOB and c in unifiable_n.keys():
+ return unifiable_n[c]
+ else:
+ return unichr(c)
+
+def entityref(c):
+ if not UNICODE_SNOB and c in unifiable.keys():
+ return unifiable[c]
+ else:
+ try: name2cp(c)
+ except KeyError: return "&" + c
+ else: return unichr(name2cp(c))
+
+def replaceEntities(s):
+ s = s.group(1)
+ if s[0] == "#":
+ return charref(s[1:])
+ else: return entityref(s)
+
+r_unescape = re.compile(r"&(#?[xX]?(?:[0-9a-fA-F]+|\w{1,8}));")
+def unescape(s):
+ return r_unescape.sub(replaceEntities, s)
+
+def fixattrs(attrs):
+ # Fix bug in sgmllib.py
+ if not attrs: return attrs
+ newattrs = []
+ for attr in attrs:
+ newattrs.append((attr[0], unescape(attr[1])))
+ return newattrs
+
+### End Entity Nonsense ###
+
+def onlywhite(line):
+ """Return true if the line does only consist of whitespace characters."""
+ for c in line:
+ if c is not ' ' and c is not ' ':
+ return c is ' '
+ return line
+
+def optwrap(text):
+ """Wrap all paragraphs in the provided text."""
+ if not BODY_WIDTH:
+ return text
+
+ assert wrap # Requires Python 2.3.
+ result = ''
+ newlines = 0
+ for para in text.split("\n"):
+ if len(para) > 0:
+ if para[0] is not ' ' and para[0] is not '-' and para[0] is not '*':
+ for line in wrap(para, BODY_WIDTH):
+ result += line + "\n"
+ result += "\n"
+ newlines = 2
+ else:
+ if not onlywhite(para):
+ result += para + "\n"
+ newlines = 1
+ else:
+ if newlines < 2:
+ result += "\n"
+ newlines += 1
+ return result
+
+def hn(tag):
+ if tag[0] == 'h' and len(tag) == 2:
+ try:
+ n = int(tag[1])
+ if n in range(1, 10): return n
+ except ValueError: return 0
+
+class _html2text(sgmllib.SGMLParser):
+ def __init__(self, out=sys.stdout.write):
+ sgmllib.SGMLParser.__init__(self)
+
+ if out is None: self.out = self.outtextf
+ else: self.out = out
+ self.outtext = u''
+ self.quiet = 0
+ self.p_p = 0
+ self.outcount = 0
+ self.start = 1
+ self.space = 0
+ self.a = []
+ self.astack = []
+ self.acount = 0
+ self.list = []
+ self.blockquote = 0
+ self.pre = 0
+ self.startpre = 0
+ self.lastWasNL = 0
+ self._bold = 0
+ self._underline = 0
+ self._color = 32
+ self.outtext += ("\033[" + str(self._color) + "m")
+
+ def outtextf(self, s):
+ if type(s) is type(''): s = codecs.utf_8_decode(s)[0]
+ self.outtext += s
+
+ def close(self):
+ sgmllib.SGMLParser.close(self)
+
+ self.pbr()
+ self.o('', 0, 'end')
+
+ return self.outtext
+
+ def handle_charref(self, c):
+ self.o(charref(c))
+
+ def handle_entityref(self, c):
+ self.o(entityref(c))
+
+ def unknown_starttag(self, tag, attrs):
+ self.handle_tag(tag, attrs, 1)
+
+ def unknown_endtag(self, tag):
+ self.handle_tag(tag, None, 0)
+
+ def previousIndex(self, attrs):
+ """ returns the index of certain set of attributes (of a link) in the
+ self.a list
+
+ If the set of attributes is not found, returns None
+ """
+ if not attrs.has_key('href'): return None
+
+ i = -1
+ for a in self.a:
+ i += 1
+ match = 0
+
+ if a.has_key('href') and a['href'] == attrs['href']:
+ if a.has_key('title') or attrs.has_key('title'):
+ if (a.has_key('title') and attrs.has_key('title') and
+ a['title'] == attrs['title']):
+ match = True
+ else:
+ match = True
+
+ if match: return i
+
+ def handle_tag(self, tag, attrs, start):
+ attrs = fixattrs(attrs)
+
+ if hn(tag):
+ self.p()
+ if start: self.o(hn(tag)*"#" + ' ')
+
+ if tag in ['p', 'div']: self.p()
+
+ if tag == "br" and start: self.o(" \n")
+
+ if tag == "hr" and start:
+ self.p()
+ self.o("_" * 80)
+ self.p()
+
+ if tag in ["head", "style", 'script']:
+ if start: self.quiet += 1
+ else: self.quiet -= 1
+
+ if tag == "blockquote":
+ if start:
+ self.p(); self.o('> ', 0, 1); self.start = 1
+ self.blockquote += 1
+ else:
+ self.blockquote -= 1
+ self.p()
+
+ if tag in ['em', 'i', 'u']:
+ if start:
+ self.o("\033[4m")
+ self._underline += 1
+ else:
+ if self._underline:
+ self.o("\033[0;" + str(self._color) + "m")
+ self._underline -= 1
+ if self._bold:
+ self.out("\033[1m")
+
+ if tag in ['strong', 'b']:
+ if start:
+ self._bold += 1
+ self.out("\033[1m")
+ else:
+ if self._bold:
+ self._bold -= 1
+ self.out("\033[0;" + str(self._color) + "m")
+ if self._underline:
+ self.o("\033[4m")
+
+ if tag == "code" and not self.pre: self.o('`') #TODO: `` `this` ``
+
+ if tag == "a":
+ if start:
+ attrsD = {}
+ for (x, y) in attrs: attrsD[x] = y
+ attrs = attrsD
+ if attrs.has_key('href'):
+ self.astack.append(attrs)
+ self.o("[")
+ else:
+ self.astack.append(None)
+ else:
+ if self.astack:
+ a = self.astack.pop()
+ if a:
+ i = self.previousIndex(a)
+ if i is not None:
+ a = self.a[i]
+ else:
+ self.acount += 1
+ a['count'] = self.acount
+ a['outcount'] = self.outcount
+ self.a.append(a)
+ self.o("][" + `a['count']` + "]")
+
+ if tag == "img" and start:
+ attrsD = {}
+ for (x, y) in attrs: attrsD[x] = y
+ attrs = attrsD
+ if attrs.has_key('src'):
+ attrs['href'] = attrs['src']
+ alt = attrs.get('alt', '')
+ i = self.previousIndex(attrs)
+ if i is not None:
+ attrs = self.a[i]
+ else:
+ self.acount += 1
+ attrs['count'] = self.acount
+ attrs['outcount'] = self.outcount
+ self.a.append(attrs)
+ self.o("![")
+ self.o(alt)
+ self.o("]["+`attrs['count']`+"]")
+
+ if tag == 'dl' and start: self.p()
+ if tag == 'dt' and not start: self.pbr()
+ if tag == 'dd' and start: self.o(' ')
+ if tag == 'dd' and not start: self.pbr()
+
+ if tag in ["ol", "ul"]:
+ if start:
+ self.list.append({'name':tag, 'num':0})
+ else:
+ if self.list: self.list.pop()
+
+ self.p()
+
+ if tag == 'li':
+ if start:
+ self.pbr()
+ if self.list: li = self.list[-1]
+ else: li = {'name':'ul', 'num':0}
+ self.o(" "*len(self.list)) #TODO: line up <ol><li>s > 9 correctly.
+ if li['name'] == "ul": self.o("* ")
+ elif li['name'] == "ol":
+ li['num'] += 1
+ self.o(`li['num']`+". ")
+ self.start = 1
+ else:
+ self.pbr()
+
+ if tag in ['tr']: self.pbr()
+
+ if tag == "pre":
+ if start:
+ self.startpre = 1
+ self.pre = 1
+ else:
+ self.pre = 0
+ self.p()
+
+ def pbr(self):
+ if self.p_p == 0: self.p_p = 1
+
+ def p(self): self.p_p = 2
+
+ def o(self, data, puredata=0, force=0):
+ if not self.quiet:
+ if puredata and not self.pre:
+ data = re.sub('\s+', ' ', data)
+ if data and data[0] == ' ':
+ self.space = 1
+ data = data[1:]
+ if not data and not force: return
+
+ if self.startpre:
+ #self.out(" :") #TODO: not output when already one there
+ self.startpre = 0
+
+ bq = (">" * self.blockquote)
+ if not (force and data and data[0] == ">") and self.blockquote: bq += " "
+
+ if self.pre:
+ bq += " "
+ data = data.replace("\n", "\n"+bq)
+
+ if self.start:
+ self.space = 0
+ self.p_p = 0
+ self.start = 0
+
+ if force == 'end':
+ # It's the end.
+ self.p_p = 0
+ self.out("\n")
+ self.space = 0
+
+
+ if self.p_p:
+ self.out(('\n'+bq)*self.p_p)
+ self.space = 0
+
+ if self.space:
+ if not self.lastWasNL: self.out(' ')
+ self.space = 0
+
+ if self.a and ((self.p_p == 2 and LINKS_EACH_PARAGRAPH) or force == "end"):
+ if force == "end": self.out("\n")
+
+ newa = []
+ for link in self.a:
+ if self.outcount > link['outcount']:
+ self.out(" ["+`link['count']`+"]: " + link['href']) #TODO: base href
+ if link.has_key('title'): self.out(" ("+link['title']+")")
+ self.out("\n")
+ else:
+ newa.append(link)
+
+ if self.a != newa: self.out("\n") # Don't need an extra line when nothing was done.
+
+ self.a = newa
+
+ self.p_p = 0
+ self.out(data)
+ self.lastWasNL = data and data[-1] == '\n'
+ self.outcount += 1
+
+ def handle_data(self, data):
+ self.o(data, 1)
+
+ def unknown_decl(self, data): pass
+
+def html2text_file(html, out=sys.stdout.write):
+ h = _html2text(out)
+ h.feed(html)
+ h.feed("")
+ return h.close()
+
+def html2text(html):
+ return optwrap(html2text_file(html, None))
+
+if __name__ == "__main__":
+ if sys.argv[1:]:
+ arg = sys.argv[1]
+ if arg.startswith('http://'):
+ data = urllib.urlopen(arg).read()
+ else:
+ data = open(arg, 'r').read()
+ else:
+ data = sys.stdin.read()
+ html2text_file(data)
+
85 src/manual.py
@@ -0,0 +1,85 @@
+from pysqlite2 import dbapi2 as sqlite
+#from subprocess import
+import html2text
+import logging
+import re
+
+__author__ = "ccheever" # Charlie Cheever <charlie@facebook.com>
+__date__ = "Mon Jun 26 03:04:55 PDT 2006"
+
+php_manual_url = "http://us2.php.net/distributions/manual/php_manual_en.html.gz"
+max_doc_size = 30000
+
+_identifier_match = re.compile('<div id="([^"]*)')
+
+def _get_documentation(url=php_manual_url):
+ # todo: actually grab here if grabbing manually is annoying enough
+ return open("php_manual_en.html")
+
+def esc(s):
+ return '"' + s.replace('\\', '\\\\').replace('"', '""') + '"'
+
+def _insert_documentation_into_db():
+ """goes through the documentation file and pulls out the relevant code and
+ then inserts stuff into the db. run from phpsh src/ dir."""
+ logging.basicConfig(level=logging.INFO)
+
+ conn = sqlite.connect('php_manual.db')
+ lines = _get_documentation()
+
+ documentation = ""
+ found_first = False
+ name = None
+
+ cursor = conn.cursor()
+ cursor.execute('CREATE TABLE php_manual ' +
+ '(identifier VARCHAR(255) PRIMARY KEY, doc TEXT)')
+ conn.commit()
+
+ for line in lines:
+ matches = _identifier_match.search(line)
+ if matches:
+ if found_first:
+ if len(documentation) > max_doc_size:
+ documentation = documentation[:max_doc_size]
+ logging.warn("truncating documentation for %s" % name)
+ cursor = conn.cursor()
+ sql = \
+ 'REPLACE INTO php_manual (identifier, doc) VALUES (%s, %s)' \
+ % (esc(name.lower()), esc(documentation))
+ cursor.execute(sql)
+ conn.commit()
+ else:
+ found_first = True
+ documentation = ""
+
+ name = matches.group(1)
+ print name
+ logging.debug("name=%s" % name)
+ else:
+ documentation += line
+ conn.close()
+
+
+def get_documentation_for_identifier(identifier, short=True):
+ identifier = identifier.replace('_', '-').lower()
+ conn = sqlite.connect('/etc/phpsh/php_manual.db')
+ cursor = conn.cursor()
+
+ sql = "SELECT doc FROM php_manual " + \
+ "WHERE identifier = %s OR identifier = %s LIMIT 1" % \
+ (esc('function.' + identifier), esc(identifier))
+ cursor.execute(sql)
+
+ try:
+ rows = cursor.fetchall()
+ except:
+ logging.error("Query to get documentation from php manual failed")
+ return "Could not query manual db"
+ conn.close()
+
+ if rows:
+ ((doc,),) = rows
+ if short:
+ doc = doc[:doc.find("Example")]
+ return html2text.html2text(doc).strip()
BIN src/php_manual.db
Binary file not shown.
148 src/phpsh
@@ -0,0 +1,148 @@
+#!/usr/bin/env python
+
+__version__ = "1.1"
+__author__ = "phpsh@facebook.com"
+__date__ = "Sep 29, 2008"
+
+from optparse import OptionParser
+from phpsh import PhpshState, PhpMultiliner, do_sugar, line_encode
+import sys
+
+usage = """~/www> phpsh [options] [extra-includes]
+phpsh is an interactive shell into a php codebase."""
+p = OptionParser(usage)
+p.add_option("-c", "--codebase-mode",
+ help="""Use "-c none" to load no codebase.
+See /etc/phpsh/phpshrc.php for other codebase modes.""")
+p.add_option("-t", "--test-file",
+ help="""Run a saved-phpsh-session unit test file.
+See test/ in the phpsh distribution for examples.""")
+p.add_option("-v", "--version", action="store_true",
+ help="""Display version info and exit.""")
+# are we cool with these negated opts? at least they indicate the defaults..
+p.add_option("-A", "--no-autocomplete", action="store_true")
+p.add_option("-C", "--no-color", action="store_true")
+p.add_option("-M", "--no-multiline", action="store_true")
+p.add_option("-T", "--no-ctags", action="store_true")
+(opts, cmd_incs) = p.parse_args()
+
+if opts.version:
+ print "phpsh version " + __version__
+ print "http://www.phpsh.org"
+ print __author__
+ sys.exit(0)
+
+# default codebase_mode is "" (don't want None)
+if not opts.codebase_mode:
+ opts.codebase_mode = ""
+
+do_multiline = not opts.no_multiline
+
+s = PhpshState(cmd_incs=set(cmd_incs),
+ do_color=not opts.no_color and not opts.test_file,
+ do_echo=not opts.test_file, codebase_mode=opts.codebase_mode,
+ do_autocomplete=not opts.no_autocomplete, do_ctags=not opts.no_ctags,
+ interactive=not opts.test_file)
+
+if opts.test_file:
+ # TODO support multiline in test-mode
+ # TODO? test-mode shouldn't support r/c/i etc should it? maybs r? but q?
+ # parse test file
+ # this is not perfect since output lines could start with "php> " (!!)
+ test_f = file(opts.test_file)
+ in_line = None
+ in_line_n = None
+ out_lines = []
+ test_pairs = []
+ line_n = 1
+ while True:
+ l = test_f.readline()
+ if not l:
+ break
+ l = l[:-1]
+ if l.startswith(s.php_prompt):
+ if in_line:
+ test_pairs.append((in_line, out_lines, in_line_n))
+ out_lines = []
+ in_line = l[len(s.php_prompt):]
+ in_line_n = line_n
+ elif in_line:
+ out_lines.append(l)
+ line_n += 1
+ if in_line:
+ test_pairs.append((in_line, out_lines, in_line_n))
+ test_f.close()
+ test_pairs_iter = test_pairs.__iter__()
+ test_cur = None
+
+ # run through test pairs
+ error_num = 0
+ for in_line, out_lines, in_line_n in test_pairs:
+ out_lines = "\n".join(out_lines)
+ out_lines_now = s.do_expr(line_encode(do_sugar(in_line)))[:-1]
+ if out_lines_now != out_lines:
+ error_num += 1
+ print s.clr_err + "ERROR Line " + str(in_line_n) + " mismatch:"
+ print "---Command:---"
+ print in_line
+ print "---Expected:---"
+ print out_lines
+ print "---Got:---"
+ print out_lines_now
+ print "----------" + s.clr_default
+ s.close()
+ if error_num:
+ print "%d of %d tests failed." % (error_num, len(test_pairs))
+ sys.exit(-1)
+ else:
+ print "All %d tests passed." % len(test_pairs)
+ sys.exit(0)
+
+
+if do_multiline:
+ m = PhpMultiliner()
+
+# main loop
+new_expr = True
+while True:
+ if new_expr:
+ prompt = s.php_prompt
+ else:
+ prompt = s.php_more_prompt
+ try:
+ line = raw_input(prompt)
+ except EOFError:
+ break
+ except KeyboardInterrupt:
+ print
+ if do_multiline:
+ m.clear()
+ new_expr = True
+ continue
+ if new_expr:
+ t_c_ret = s.try_command(line)
+ if t_c_ret == PhpshState.quit_command:
+ break
+ elif t_c_ret == PhpshState.yes_command:
+ continue
+ line_ready = None
+ if do_multiline:
+ (m_res, m_str) = m.input_line(line)
+ if m_res == PhpMultiliner.complete:
+ line_ready = m_str
+ elif m_res == PhpMultiliner.syntax_error:
+ print s.clr_err + \
+ "Multiline input has no syntactic completion:"
+ print m_str + s.clr_default
+ m.clear()
+ line_ready = ""
+ else:
+ line_ready = line_encode(do_sugar(line))
+ if line_ready != None:
+ if line_ready:
+ s.write()
+ s.do_expr(line_ready)
+ new_expr = True
+ else:
+ new_expr = False
+s.close()
210 src/phpsh.php
@@ -0,0 +1,210 @@
+#!/usr/bin/env php
+<?php
+// Copyright 2004-2007 Facebook. All Rights Reserved.
+// this is used by phpshell.py to exec php commands and maintain state
+// @author ccheever
+// @author dcorson
+// @author warman (added multiline input support \ing)
+// @date Thu Jun 15 22:27:46 PDT 2006
+//
+// usage: this is only called from phpsh (the python end), as:
+// phpsh.php <comm-file> <codebase-mode> [-c]
+//
+// use '' for default codebase-mode, define others in phpshrc
+// -c turns off color
+
+// set the TFBENV to script
+$_SERVER['TFBENV'] = 16777216;
+
+// FIXME: www/lib/thrift/packages/falcon/falcon.php is huge
+// this is probably not the right fix, but we need it for now
+ini_set('memory_limit', ini_get('memory_limit') * 2 . 'M');
+
+// we buffer the output on includes so that output that gets generated by includes
+// doesn't interfere with the secret messages we pass between php and python
+// we'll capture any output and show it when we construct the shell object
+ob_start();
+
+$___phpshell___codebase_mode = $argv[2];
+$___phpshell___homerc = getenv('HOME').'/.phpshrc.php';
+if (file_exists($___phpshell___homerc)) {
+ require_once $___phpshell___homerc;
+} else {
+ require_once '/etc/phpsh/phpshrc.php';
+}
+
+$___phpshell___do_color = true;
+$___phpshell___do_autocomplete = true;
+$___phpshell___options_possible = true;
+foreach (array_slice($GLOBALS['argv'], 3) as $___phpshell___arg) {
+ $___phpshell___did_arg = false;
+ if ($___phpshell___options_possible) {
+ switch ($___phpshell___arg) {
+ case '-c':
+ $___phpshell___do_color = false;
+ $___phpshell___did_arg = true;
+ break;
+ case '-A':
+ $___phpshell___do_autocomplete = false;
+ $___phpshell___did_arg = true;
+ break;
+ case '--':
+ $___phpshell___options_possible = false;
+ $___phpshell___did_arg = true;
+ break;
+ }
+ if ($___phpshell___did_arg) {
+ continue;
+ }
+ }
+
+ include_once $___phpshell___arg;
+}
+
+$___phpshell___output_from_includes = ob_get_contents();
+ob_end_clean();
+
+// unset all the variables we don't absolutely need
+unset($___phpshell___arg);
+unset($___phpshell___did_arg);
+unset($___phpshell___options_possible);
+$___phpshell___ = new ___PHPShell___($___phpshell___output_from_includes,
+ $___phpshell___do_color, $___phpshell___do_autocomplete, $argv[1]);
+
+/**
+ * An instance of a phpshell interactive loop
+ *
+ * @author ccheever
+ * @author dcorson
+ *
+ * This class mostly exists as a proxy for a namespace
+ */
+class ___PHPShell___ {
+ var $_handle = STDIN;
+ var $_comm_handle;
+ var $_MAX_LINE_SIZE = 262144;
+
+ /**
+ * Constructor - actually runs the interactive loop so that all we have to do is construct it to run
+ * @param list $extra_include Extra files that we want to include
+ *
+ * @author ccheever
+ * @author dcorson
+ */
+ function __construct($output_from_includes='', $do_color, $do_autocomplete,
+ $comm_filename) {
+ $this->_comm_handle = fopen($comm_filename, 'w');
+
+ $this->__send_autocomplete_identifiers($do_autocomplete);
+
+ // now it's safe to send any output the includes generated
+ print $output_from_includes;
+ fwrite($this->_comm_handle, "ready\n");
+
+ $this->_interactive_loop($do_color);
+ }
+
+ /**
+ * Destructor - just closes the handle to STDIN
+ *
+ * @author ccheever
+ */
+ function __destruct() {
+ fclose($this->_handle);
+ }
+
+ /**
+ * Sends the list of identifiers that phpshell should know to tab-complete to python
+ *
+ * @author ccheever
+ */
+ function __send_autocomplete_identifiers($do_autocomplete) {
+ // send special string to signal that we're sending the autocomplete identifiers
+ print "#start_autocomplete_identifiers\n";
+
+ if ($do_autocomplete) {
+ // send function names -- both user defined and built-in
+ // globals, constants, classes, interfaces
+ $defined_functions = get_defined_functions();
+ $methods = array();
+ foreach (($classes = get_declared_classes()) as $class) {
+ foreach (get_class_methods($class) as $class_method) {
+ $methods[] = $class_method;
+ }
+ }
+ foreach (array_merge($defined_functions['user'], $defined_functions['internal'], array_keys($GLOBALS), array_keys(get_defined_constants()), $classes, get_declared_interfaces(), $methods, array('instanceof')) as $identifier) {
+ // exclude the phpshell internal variables from the autocomplete list
+ if ((substr($identifier, 0, 14) != "___phpshell___") && ($identifier != '___PHPShell___')) {
+ print "$identifier\n";
+ } else {
+ unset($$identifier);
+ }
+ }
+ }
+
+ // string signalling the end of autocmplete identifiers
+ print "#end_autocomplete_identifiers\n";
+ }
+
+ /**
+ * The main interactive loop
+ *
+ * @author ccheever
+ * @author dcorson
+ *
+ * We prefix our vars here to prevent accidental name collisions :(
+ */
+ function _interactive_loop($do_color) {
+ extract($GLOBALS);
+
+ $buf_len = 0;
+
+ while (!feof($this->_handle)) {
+ // indicate to phpsh (parent process) that we are ready for more input
+ fwrite($this->_comm_handle, "ready\n");
+
+ // multiline inputs are encoded to one line
+ $buffer_enc = fgets($this->_handle, $this->_MAX_LINE_SIZE);
+ $buffer = stripcslashes($buffer_enc);
+ $buf_len = strlen($buffer);
+
+ // evaluate what the user's entered
+ if ($do_color) {
+ print "\033[33m"; // yellow
+ }
+ try {
+ $evalue = eval($buffer);
+ } catch (Exception $e) {
+ // unfortunately, almost all exceptions that aren't explicitly thrown
+ // by users are uncatchable :(
+ fwrite(STDERR, 'Uncaught exception: '.get_class($e).': '.
+ $e->getMessage()."\n");
+ $evalue = null;
+ }
+
+ // if any value was returned by the evaluated code, print it
+ if (isset($evalue)) {
+ if ($do_color) {
+ print "\033[36m"; // cyan
+ }
+ if ($evalue === true) {
+ print "true";
+ } elseif ($evalue === false) {
+ print "false";
+ } elseif ($evalue === null) {
+ print "null";
+ } else {
+ print_r($evalue);
+ }
+ // set $_ to be the value of the last evaluated expression
+ $_ = $evalue;
+ }
+ // back to normal for prompt
+ if ($do_color) {
+ print "\033[0m";
+ }
+ // newline so we end cleanly
+ print "\n";
+ }
+ }
+}
1 src/phpsh.py
4 src/phpsh_check_syntax
@@ -0,0 +1,4 @@
+#!/usr/bin/env php
+<?php
+error_reporting(E_PARSE);
+eval('return;'.$argv[1]);
38 src/phpshrc.php
@@ -0,0 +1,38 @@
+<?php
+
+// Have this file include/do anything to set up your php codebase environment.
+// Expect it to run from the top of your repo, where you should start phpsh
+// from. Run ctags from the top of your repo periodically too, to get the
+// benefits of phpsh ctags integration.
+
+// You may make use of the string $___phpshell___codebase_mode which may be
+// specified on phpsh startup via -C. The string is '' by default.
+
+
+// E_ALL catches certain coding errors and we recommend it. Comment this out
+// if it produces too many warnings you don't have time to fix.
+// You could also try moving it to the bottom of this file, after you load your
+// codebase libraries, to have fewer pre-existing warnings show up, but still
+// get added safety in phpsh.
+error_reporting(E_ALL);
+
+
+switch ($___phpshell___codebase_mode) {
+case '':
+ // Put default library includes here.
+ //require_once 'relative-path-from-repo-head/lib/codebase_include.php';
+ break;
+
+case 'none':
+ // Vanilla php
+ break;
+
+// Put any custom codebase modes here, e.g.:
+//case 'core':
+// // Would only load 'core' library includes here, perhaps for faster startup
+// // in a very large codebase.
+
+default:
+ fwrite(STDERR, 'Unknown codebase mode '.$___phpshell___codebase_mode."\n");
+ break;
+}
6 test/all_good.pht
@@ -0,0 +1,6 @@
+Loading ctags
+Starting php
+type 'h' or 'help' to see instructions & features
+php> =4
+4
+php>
15 test/err_and_multiout.pht
@@ -0,0 +1,15 @@
+Loading ctags
+Starting php
+type 'h' or 'help' to see instructions & features
+php> =4
+5
+php> ="4\n5"
+4
+5
+php>
+php>
+php>
+php>
+php> ='hi'
+hi
+php>

0 comments on commit 9317b56

Please sign in to comment.