Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

--- IN ATTIC BECAUSE: import problems

Fetching latest commit…

Octocat-spinner-32-eaf2f5

Cannot retrieve the latest commit at this time

Octocat-spinner-32 ChangeLog
Octocat-spinner-32 README
Octocat-spinner-32 TODO
Octocat-spinner-32 pymacs-services
Octocat-spinner-32 pymacs.el
Octocat-spinner-32 pymacs.py
Octocat-spinner-32 setup
README
* README for `Pymacs'				allout -*- outline -*-

.. Presentation.

. : What is Pymacs?

    Pymacs is a powerful tool which, once started from Emacs, allows both-way
    communication between Emacs LISP and Python.  Pymacs aims at using
    Python as an extension language for Emacs.  One may load and use Python
    modules from within Emacs LISP code.  Python functions may themselves
    use Emacs services, and handle LISP objects kept in LISP space.

    The goals are to write "naturally" in both languages, debug with ease,
    fall back gracefully on errors, and allow full cross-recursivity.

    It is not expected that Pymacs users have a deep knowledge of both Emacs
    LISP and Python, yet knowledge always helps!  As Python scripting is the
    main goal for Pymacs, you'll find at the end of this documentation a few
    examples meant for Python users having a limited experience with Emacs.

    It is very easy to install Pymacs, as neither Emacs nor Python need to
    be compiled nor relinked.  Emacs merely starts Python as a subprocess,
    and Pymacs implements a communication protocol between both processes.

. : Warning to Pymacs users.

    This is beta status software: specifications are only slightly frozen,
    you might have to later adapt your code to specification changes.  Report
    the problems you see to François Pinard at `pinard@iro.umontreal.ca'.
    For discussing specifications or making suggestions, please also copy
    the `python-list@python.org' mailing list, to help brain-storming! :-)

. : History and references.

    Pymacs revisits previous Cedric Adjih's works about running Python as a
    process separate from Emacs.  See `http://www.crepuscule.com/pyemacs/',
    or write Cedric at `adjih-pam@crepuscule.com'.  Cedric presented
    `pyemacs' to me as a proof of concept.  I spiced the `pyemacs' concept
    with a few simplification ideas on my own, and decided to drop the `e'
    from `pyemacs' to witness that simplification :-).  Cedric told me that
    there also exist some older patches for linking Python right into XEmacs.

    Brian McErlean contacted me, as he independently and simultaneously
    wrote a very similar project.  Amusing coincidence, he even chose
    `pymacs' as a name.  As he paid good attention to complex details that
    escaped my courage, his help and collaboration have been beneficial.
    You may reach Brian at `brianmce@crosswinds.net'.

    One other reference of interest is Doug Bagley shoot out project,
    which compares the relative speed of many popular languages.  See
    `http://www.bagley.org/~doug/shootout/' for more information.

.. Installation.

. : Execute the `setup' script.

    To install Pymacs, `cd' into its distribution, then run `./setup -ie'.
    This will invite you to interactively confirm installation directories.
    Without `-ie', Pymacs will be installed in automatically guessed places.
    Use `-n' to known about these guesses without proceeding to the actual
    installation.  You may use options to select various directories or
    executables for Emacs or Python, try `./setup -H' for a complete list.

    To check that `pymacs.el' is properly installed, start Emacs and give
    it the command `M-x load-library RET pymacs': you should not receive
    any error.  To check that `pymacs.py' is properly installed, start
    an interactive Python session and type `import pymacs': you should
    not receive any error.  To check that `pymacs-services' is properly
    installed, type `pymacs-services </dev/null' in a shell; you should
    then receive two lines: one ending with "(pymacs-version VERSION)",
    and another saying: "Protocol error: `>' expected.".

. : Prepare your `.emacs' file.

    The ".emacs" file is not given in the distribution, you likely have
    one already in your home directory.  You need to add these lines:

       (autoload 'pymacs-load "pymacs" nil t)
       (autoload 'pymacs-eval "pymacs" nil t)
       (autoload 'pymacs-apply "pymacs")

    If the file "$HOME/.emacs" does not exist, merely create it with the
    above lines.  You are now all set to use Pymacs.

    To check this, start a fresh Emacs session, and type `M-x pymacs-eval'.
    Emacs should prompt you for a Python expression.  Try "`2L**111`" (type
    the backquotes, but not the external double-quotes).  The minibuffer
    should display `2596148429267413814265248164610048L'. `M-x pymacs-load'
    should prompt you for a Python module name.  Reply `os'.  After Emacs
    prompts you for a prefix, merely hit Enter to accept the default prefix.
    This should have the effect of importing the Python "os" module within
    Emacs.  Typing `M-: (os-getcwd)' should echo the current directory in
    the message buffer, as returned by the `os.getcwd' Python function.

. : Caveats.

    Some later versions of Emacs 20 silently ignore the request for
    creating weak hash tables, they create an ordinary table instead.
    Older Emacses just do not have hash tables.  Pymacs should run on
    all, yet for these, memory will leak on the Python side whenever
    complex objects get transmitted to Emacs, as these objects will not
    be reclaimed on the Python side once Emacs is finished with them.
    It should not be a practical problem in most simple cases.

.. LISP structures and Python objects.

. : Conversions.

    Whenever LISP calls Python functions giving them arguments, these
    arguments are LISP structures that should be converted into Python
    objects in some way.  Conversely, whenever Python calls LISP functions,
    the arguments are Python objects that should be received as LISP
    structures.  We need some conventions for doing such conversions.

    Python is meant to be an extension language for LISP, rather than
    the other way around.  So by default, conversions generally transmit
    mutable LISP structures as mutable objects on the Python side, in
    such a way that transforming the object in Python will effectively
    transform the structure on the LISP side.  This implies that there are
    more frequent communications between Emacs and Python, with less data
    in each communication.  But this is not true the other way around,
    Python objects transmitted to LISP will often loose their mutability,
    transforming the LISP structure will not be reflected on the Python side.
    This implies that there are fewer communications between Emacs and
    Python, yet these might sometimes be a bit more bulky.

. : Simple objects.

    LISP `nil' and the equivalent LISP `()' yield Python `None'.  Python
    `None' and the Python empty list `[]' are returned as `nil' in LISP.

    LISP numbers, either integer or floating, are converted in equivalent
    Python numbers.  LISP characters are really numbers and yield Python
    numbers.  In the other direction, Python numbers are converted into LISP
    numbers, with the exception of long Python integers and complex numbers.

    LISP strings are usually converted into equivalent Python narrow strings.
    As Python strings do not have text properties, these are not reflected.
    This may be changed by setting the `pymacs-mutable-strings' option: if
    this variable is not `nil', LISP strings are then transmitted opaquely.
    Python strings, except Unicode, are always converted into LISP strings.

    LISP symbols yield the special `lisp.SYMBOL' or `lisp[STRING]'
    notations on the Python side.  The first notation is used when the
    LISP symbol starts with a letter, and contains only letters, digits and
    hyphens, in which case LISP hyphens get replaced by Python underscores.
    This convention is welcome, as LISP programmers commonly prefer using
    dashes, where Python programmers use underlines.  Otherwise, the second
    notation is used.  Conversely, `lisp.SYMBOL' on the Python side yields a
    LISP symbol with underscores replaced with hyphens, while `lisp[STRING]'
    corresponds to a LISP symbol printed with that STRING which, of course,
    should then be a valid LISP symbol name.

. : Sequences.

    The case of strings has been discussed in the previous section.

    Proper LISP lists, those for which the `cdr' of last cell is `nil', are
    normally transmitted opaquely to Python.  If `pymacs-forget-mutability'
    is set, or if Python later asks for these to be expanded, proper LISP
    lists get converted into Python lists.  In the other direction, Python
    lists are always converted into proper LISP lists, if we except the
    empty list, which is always converted as Python `None'.

    LISP vectors are normally transmitted opaquely to Python.  However, if
    `pymacs-forget-mutability' is set, or if Python later asks for these
    to be expanded, LISP vectors get converted into Python tuples.  In the
    other direction, Python tuples are always converted into LISP vectors.

    Remember the rule: round parentheses correspond to square brackets!
    It works for lists, vectors, tuples, seen from either LISP or Python.

    The above choices were debatable.  Since LISP proper lists and Python
    lists are the bread-and-butter of algorithms modifying structures, at
    least in my experience, I guess they are more naturally mapped into one
    another, this spares many casts in practice.  While in Python, the most
    usual idiom for growing lists is appending to their end, the most usual
    idiom in LISP to grow a list is by cons'ing new items at its beginning:

       (setq accumulator (cons 'new-item accumulator))

    or more simply:

       (push accumulator 'new-item)

    So, in case speed is especially important and many modifications happen
    in a row on the same side, while order of elements ought to be preserved,
    some (nreverse ...) on the LISP side or .reverse() on the Python side
    side might be needed.  Surely, proper lists in LISP and lists in Python
    are the normal structure for which length is easily modified.

    We cannot so easily change the size of a vector, the same as it requires
    a bit more stunts to "modify" a tuple.  The shape of these objects is
    fixed.  Mapping vectors to tuples, which is admittedly strange, will
    only be done if the Python side requests an expanded copy, otherwise
    an opaque LISP object is seen in Python.  In the other direction,
    whenever a LISP vector is needed, one has to write `tuple(python_list)'
    while transmitting the object.  Such transmissions are most probably
    to be unusual, as people are not going to blindly transmit whole big
    structures back and forth between Emacs and Python, they would rather
    do it once in a while only, and do only local modifications afterwards.
    The infrequent casting to `tuple' for getting a LISP vector seems to
    suggest that we did a reasonable compromise.

    In Python, both tuples and lists have O(1) access, so there is no real
    speed consideration there.  LISP is different: vectors have O(1) access
    while lists have O(N) access.  The rigidity of LISP vectors is such that
    people do not resort to vectors unless there is a speed issue, so in
    real LISP practice, vectors are used parsimoniously.  So parsimoniously,
    in fact, that LISP vectors are overloaded for what they are not meant:
    for example, very small vectors are used to represent X events in
    key-maps, programmers only want to test for the vector type, and users
    like bracketed syntax, in which case speed of access is hardly an issue.

. : Opaque objects.

.  , Lisp handles.

     When a Python function is called from LISP, the function arguments
     have already been converted to Python types from LISP types and the
     function result is going to be converted back to LISP.

     Several LISP objects do not have Python equivalents, like for Emacs
     windows, buffers, markers, overlays, etc.  It is nevertheless useful
     to pass them to Python functions, hoping that these Python functions
     will "operate" on these LISP objects.  Of course, the Python side may
     not itself modify such objects, it has to call for Emacs services to
     do so.  LISP handles are a mean to ease this communication.

     Whenever a LISP object may not be converted to a Python object, an
     LISP handle is created and used instead.  Whenever that LISP handle is
     returned into LISP from a Python function, or is used as an argument
     to a LISP function from Python, the original LISP object behind the
     LISP handle is automatically retrieved.

     LISP handles are either instances of the `pymacs.Lisp' class, or of one
     of its subclasses.  If `object' is a LISP handle, and if the underlying
     LISP object is a LISP sequence, then whenever `object[index]',
     `object[index] = value' and `len(object)' are meaningful, these
     may be used to fetch or alter an element of the sequence directly
     in LISP space.  Also, if `object' corresponds to a LISP function,
     `object(ARGUMENTS)' may be used to apply the LISP function over the
     given arguments.  Since arguments have been evaluated the Python way on
     the Python side, it would be conceptual overkill evaluating them again
     the LISP way on the LISP side, so Pymacs manage to quotes arguments for
     defeating LISP evaluation.  The same logic applies the other way around.

     LISP handles have a `value()' method, which merely returns self.
     They also have a `copy()' method, which tries to "open the box"
     if possible.  LISP proper lists are turned into Python lists, LISP
     vectors are turned into Python tuples.  Then, modifying the structure
     of the copy on the Python side has no effect on the LISP side.

.  , Python handles.

     The same as LISP handles are useful to handle LISP objects on the
     Python side, Python handles are useful to handle Python objects on
     the LISP side.

     Many Python objects do not have direct LISP equivalents, including
     long integers, complex numbers, Unicode strings, modules, classes,
     instances and surely a lot of others.  When such are being transmitted
     to the LISP side, Pymacs use Python handles.  These are automatically
     recovered into the original Python objects whenever transmitted back to
     Python, either as arguments to a Python function, as the Python function
     itself, or as the return value of a LISP function called from Python.

     The objects represented by these Python handles may be inspected or
     modified using the basic library of Python functions.  For example, in:

        (setq matcher (pymacs-eval "re.compile('PATTERN').match"))
        (pymacs-apply matcher (list ARGUMENT))

     the initial `setq' above could be decomposed into:

        (setq compiled (pymacs-eval "re.compile('PATTERN')")
              matcher (pymacs-apply "getattr" (list compiled "match")))

     This example shows that one may use `pymacs-apply' with "getattr"
     as the function, to get a wanted attribute for a Python object.

.. Usage on the LISP side.

. : `pymacs-eval'.

    Function `(pymacs-eval TEXT)' gets TEXT evaluated as a Python expression,
    and returns the value of that expression converted back to LISP.

. : `pymacs-apply'.

    Function `(pymacs-apply FUNCTION ARGUMENTS)' will get Python to apply the
    given FUNCTION over the given ARGUMENTS.  ARGUMENTS is a list containing
    all arguments, or `nil' if there is none.  FUNCTION is either a Python
    string holding an expression yielding a Python function, or else, a
    Python handle previously received from Python, and hopefully holding
    a callable Python object.  Each argument gets separately converted
    to Python before the function is called.  `pymacs-apply' returns the
    resulting value of the function call, converted back to LISP.

. : `pymacs-load'.

    Function `(pymacs-load MODULE PREFIX)' imports the Python MODULE into
    LISP space.  MODULE is the name of the file containing the module,
    without any `.py' or `.pyc' extension.  If the directory part is omitted
    in MODULE, the module will be looked into the current Python search path.
    Each top-level function in the module produces a trampoline function in
    LISP having the same name, except that underlines in Python names are
    turned into dashes in LISP, and that PREFIX is uniformly added before
    the LISP name (as a way to avoid name clashes).  PREFIX may be omitted,
    in which case it defaults to base name of MODULE with underlines turned
    into dashes, and followed by a dash.

    The return value of a successful `pymacs-load' is the module object.
    An optional third argument, NOERROR, when given and not `nil', will
    have `pymacs-load' to return `nil' instead of raising an error, if
    the Python module could not be found.

    When later calling one of these functions, all provided arguments are
    converted to Python and transmitted, and the function return value
    is later converted back to LISP.  It is left to the Python side to
    check for argument consistency.  However, for an interactive function,
    the interaction specification drives some checking on the LISP side.
    There is no provision to collect keyword arguments in LISP.

. : Expected usage.

    We do not expect that `pymacs-eval' or `pymacs-apply' will be much used,
    if ever.  In practice, the LISP side of a Pymacs application might
    call `pymacs-load' a few times for linking into the Python modules,
    with the indirect effect of defining trampoline functions for these
    modules on the LISP side, than can be called like usual LISP functions.

    These imported functions are really those which are of interest for
    the user, and the preferred way to call Python services with Pymacs.

. : Special LISP variables.

    Users could alter the inner working of Pymacs through a few variables,
    which are documented here.  Except for `pymacs-load-path', which should
    be set before the first call to `pymacs-eval' or `python-load', the
    value of these variables can be changed at any time.

.  , pymacs-load-path

     Users might want to use special directories for holding their Python
     modules, when these modules are meant to be used from Emacs.  Best is
     to preset `pymacs-load-path, `nil' by default, to a list of these
     directory names.  (Tilde expansions and such occur automatically.)

.  , pymacs-trace-transit

     The `*Pymacs*' buffer, within Emacs, holds a trace of transactions
     between Emacs and Python.  When `pymacs-trace-transit' is `nil',
     and this is the default setting, the buffer only holds the last
     bi-directional transaction (a request and a reply).  If that variable
     is not `nil', all transactions are kept.  This could be useful for
     debugging, but the drawback is that this buffer could grow big over
     time, to the point of diminishing Emacs performance.

.  , pymacs-forget-mutability

     The default behaviour of Pymacs is to transmit LISP objects
     to Python in such a way thay they are fully modifiable from the
     Python side, would it mean triggering LISP functions to act on them.
     When `pymacs-forget-mutability' is not `nil', the behaviour is changed,
     and the flexibility is lost.  Pymacs then tries to expand proper lists
     and vectors as full copies when transmitting them on the Python side.
     This variable, seen as a user setting, is best left to `nil'.  It may
     be temporarily overriden within some functions, when deemed useful.

     There is no corresponding variable from objects transmitted to Emacs
     from Python.  Pymacs automatically expands what gets transmitted.
     Mutability is preserved only as a side-effect of not having a natural
     LISP representation for the Python object.  This assymetry is on
     purpose, yet debatable.  Maybe Pymacs could have a variable telling
     that mutability _is_ important for Python objects?  That would give
     Pymacs users the capability of restoring the symmetry somewhat, if
     they have a strong appetite for it.  But I'm not sure it would be
     worth the effort: I merely tried to guess what's most useful.

.  , pymacs-mutable-strings

     Strictly speaking, Emacs LISP strings are mutable. Yet, it does not
     come naturally to a Python programmer to modify a string "in-place",
     as Python strings are never mutable.  When `pymacs-mutable-strings'
     is `nil', which is the default setting, LISP strings are transmitted
     to Python as Python strings, and so, loose their mutability.  Also,
     with the default setting, text properties are not reflected.  If that
     variable is not `nil', LISP strings are rather passed as LISP handles.
     This variable is ignored whenever `pymacs-forget-mutability' is set.

.. Usage on the Python side.

. : Python setup.

    Pymacs requires little or no setup in the Python modules which are
    meant to be used from Emacs, for the simple situations where these
    modules receive nothing but Emacs nil, numbers or strings, or return
    nothing but Python None, numbers or strings.

    For other cases, Python modules might use `from pymacs import lisp'
    near their beginning or rather, for better flexibility:

        import pymacs
        lisp = pymacs.lisp

. : Response mode.

    When Python receives a request from Emacs in the context of Pymacs,
    and until it returns the reply, Emacs keeps listening to serve
    Python requests.  Emacs is not listening otherwise.  Consequently,
    Python should never attempt calling for Emacs services at other times.
    Other Python threads may not call Emacs without careful synchronisation.

. : LISP symbols.

    `lisp' is a special object which has useful built-in magic.
    Its attributes do nothing but represent LISP symbols, created on the
    fly as needed (symbols also have their built-in magic).

    Except for `lisp.nil' or `lisp["nil"]', which are the same as `None',
    both `lisp.SYMBOL' and `lisp[STRING]' yield objects of `pymacs.Symbol'
    type.  These are genuine Python objects, that could be referred to
    by simple Python variables.  One can write `quote = lisp.quote',
    for example, and use `quote' afterwards to mean that LISP symbol.
    Here are other examples.  If a Python function received a LISP symbol
    as an argument, it can check with `==' if that argument is `lisp.never'
    or `lisp.ask'.  And a Python function may choose to return `lisp.t'.

    In Python, writing `lisp.SYMBOL = VALUE' or `lisp[STRING] = VALUE' does
    assign VALUE to the corresponding symbol in LISP space.  Beware that
    in such cases, the `lisp.' prefix may not be spared.  One cannot write
    `result = lisp.result' and hope that a later `result = 3' will have any
    effect in the LISP space: this would merely change the Python variable
    `result', which was a reference to a `pymacs.Symbol' instance, so it
    is now a reference to the number 3.

    The `pymacs.Symbol' class has `value()' and `copy()' methods.  One can
    use either `lisp.SYMBOL.value()' or `lisp.SYMBOL.copy()' to access
    the LISP value of a symbol, after conversion to some Python object,
    of course.  However, if `value()' would have given a LISP handle,
    `lisp.SYMBOL.copy()' has the same effect of `lisp.SYMBOL.value().copy()',
    that is, it returns the value of the symbol as opened as possible.

    A symbol may also be as if it was a Python function, in which case it
    really names a LISP function that should be applied over the following
    function arguments.  The result of the LISP function becomes the value
    of the call, with all due conversions of course.

. : Dynamic bindings.

    As Emacs LISP uses dynamic bindings, it is common that LISP programs
    use `let' for temporarily setting new values for some LISP variables
    having global scope.  These variables recover their previous value
    automatically when the `let' gets completed, even if an error occurs
    which interrupts the normal flow of execution.

    Pymacs has a `pymacs.Let' class to represent such temporary settings.
    Suppose for example that you want to recover the value of `lisp.mark()'
    when the transient mark mode is active on the LISP side.  One could
    surely use `lisp.mark(lisp.t)' to "force" reading the mark in such cases,
    but for the sake of illustration, let's ignore that, and temporarily
    deactivate transient mark mode instead.  This could be done this way:

    try:
        let = pymacs.Let()
        let.push(transient_mark_mode=None)
        ... USER CODE ...
    finally:
        let.pop()

    `let.push()' accepts any number of keywords arguments.  Each keyword name
    is interpreted as a LISP symbol written the Pymacs way, with underlines.
    The value of that LISP symbol is saved on the Python side, and the value
    of the keyword becomes the new temporary value for this LISP symbol.
    A later `let.pop()' restores the previous value for all symbols which
    were saved together at the time of the corresponding `let.push()'.
    There may be more than one `let.push()' call for a single `let' instance,
    they stack within that instance.  Each `let.pop()' will undo one and
    only one `let.push()' from the stack, in the reverse order or the pushes.

    When the `let' instance disappears, either because the programmer does
    `del let' or `let = None', or just because the Python `let' variable
    goes out of scope, all remaining `let.pop()' get automatically executed,
    so the `try'/'finally' statement may be omitted in practice.  For this
    omission to work flawlessly, the programmer should be careful at not
    keeping extra references to the instance.

    The constructor call `let = pymacs.Let()' also has an implied initial
    `.push()' over all given arguments, so the explicit `let.push()'
    may be omitted as well.  In practice, this sums up and the above code
    could be reduced to a mere:

    let = pymacs.Let(transient_mark_mode=None)
    ... USER CODE ...

    Be careful at assigning the result of the constructor to some Python
    variable.  Otherwise, the instance would disappear immediately after
    having been created, restoring the LISP variable much too soon.

    Any variable to be bound with `pymacs.Let' should have been bound
    in advance on the LISP side.  This restriction usually does no kind
    of harm.  Yet, it will likely be lifted in some later version of Pymacs.

    The `pymacs.Let' class has other methods meant for some macros which
    are common in Emacs LISP programming, in the spirit of `let' bindings.
    These method names look like `push_*' or `pop_*', where LISP macros are
    `save-*'.  One has to use the matching `pop_*' for undoing the effect of
    a given `push_*' rather than a mere `.pop()': the Python code is clearer,
    this also ensures that things are undone in the proper order.  The same
    `let' instance may use many `push_*' methods, their effects nest.

    `push_excursion()' and `pop_excursion' save and restore the current
    buffer, point and mark.  `push_match_data()' and `pop_match_data()'
    save and restore the state of the last regular expression match.
    `push_restriction()' and `pop_restriction()' save and restore
    the current narrowing limits.  `push_selected_window()' and
    `pop_selected_window()' save and restore the fact that a window holds
    the cursor.  `push_window_excursion()' and `pop_window_excursion()'
    save and restore the current window configuration in the Emacs display.

    For example, one may write:

    let = pymacs.Let()
    if 1:
        let.push_excursion()
        ... USER CODE ...
    del let

    The `if 1:' (some people might prefer writting `if let:'), has the only
    goal of indenting USER CODE, so the scope of the `let' variable is made
    quite explicit.  This is purely stylistic, and not at all necessary.
    The last `del let' might be omitted in a few circumstances, for example
    if the excursion lasts until the end of the Python function.

. : Raw LISP expressions.

    Pymacs offers a device for evaluating a raw LISP expression expressed
    as a string.  One merely uses `lisp' as a function, like this:

    lisp("""
    ...
    POSSIBLY-LONG-LISP-EXPRESSION
    ...
    """)

    The LISP value of the expression becomes the value of the `lisp' call,
    after conversion back to Python.

. : User interaction.

    Emacs functions have the concept of user interaction for completing
    the specification of their arguments while being called.  This happens
    only when a function is interactively called by the user, it does
    not happen when a function is programmatically called by another.
    As Python does not have a corresponding facility, a bit of trickery
    was needed to retrofit that facility on the Python side.

    Whenever a Python module contains an `interactions' global variable
    which is a dictionary, Pymacs uses that dictionary at load time to find
    out the interaction specification of each imported function from this
    module.  If the dictionary has no entry for a given function, or if the
    dictionary value is None, that function is not going to be interactive
    on the Emacs side.  Otherwise, the function is interactive, and the
    dictionary value should either be a string or another Python function.

    If the interaction specification is a string, that string is used
    verbatim as the argument to the `(interactive ...)' function on the
    Emacs side.  Within Emacs, you may execute `C-h f interactive' to get
    a short reminder about how this string is interpreted on the Emacs side.
    For an interactive function having no arguments, use the empty string:


        import pymacs
        interactions = {}

        def hello_world():
            "Hello world from python"
            pymacs.lisp.insert("Hello from python")
        interactions[hello_world] = ''


    If the interaction specification is not a string, but another Python
    function, that function is called without arguments and should return
    a list of the actual arguments to be used for the original function.

    One might wonder why we did not use `lisp.interactive(...)' from Python.
    The requirement that "(interactive ...)" be first in an Emacs LISP
    `defun' suggests that there is some magic in the Emacs LISP interpreter
    itself, which looks for that call _before_ the function is actually
    entered.  Looking for `lisp.interactive' would be the difficult part,
    as this Python code is already in compiled form when it gets executed.
    Pymacs could try to scan the generated code, but given the computation of
    `lisp.interactive' arguments could get arbitrarily complex, it would
    be rather difficult to un-compile that evaluation into LISP code.

    The ideal might have been to use `interactive' as a function attribute,
    following the ideas found within PEP 232.  We might come naturally
    to that one of these days, if this PEP ever gets implemented.  In the
    meantime, it is plain better to use more widespread methods.

. : Keybindings.

    An interactive function may be bound to a key sequence.

    To translate bindings like `C-x w', say, one might have to know a
    bit more how LISP processes string escapes like `\C-x' or `\M-\C-x'
    in LISP, and emulate it within Python strings, since Python does not
    have such escapes.  `\C-L', where L is an upper case letter, produces a
    character which ordinal is the result of subtracting 0x40 from ordinal
    of `L'.  `\M-' has the ordinal one gets by adding 0x80 to the ordinal
    of following described character.  So people can use self-inserting
    non-ASCII characters, `\M-' is given another representation, which is
    to replace the addition of 0x80 by prefixing with `ESC', that is 0x1b.

    So "\C-x" in Emacs is '\x18' in Python.  This is easily found, using an
    interactive Python session, by givin it: chr(ord('X') - ord('A') + 1).
    An easier way would be using the `kbd' function on the LISP side,
    like with lisp.kbd('C-x w') or lisp.kbd('M-<f2>').

    To bind the F1 key to the `helper' function in some `module':

        lisp.global_set_key((lisp.f1,), lisp.module_helper)

    (item,) is a Python tuple yielding a LISP vector.  `lisp.f1' translates
    to the LISP symbol `f1'.  So, Python `(lisp.f1,)' is LISP `[f1]'.
    Keys like `[M-f2]' might require some more ingenuity, one may write
    either (lisp['M-f2'],) or (lisp.M_f2,) on the Python side.

.. Debugging.

. : The `*Pymacs*' buffer.

    Emacs and Python are two separate processes (well, each may use more
    than one process).  Pymacs implements a simple communication protocol
    between both, and does whatever needed so the programmers do not have to
    worry about details.  The main debugging tool is the communication buffer
    between Emacs and Python, which is named `*Pymacs*'.  To make good use
    of it, first set `pymacs-trace-transit' to `t', so all exchanges are
    accumulated in that buffer.  It helps understanding the communication
    protocol, so it is shortly explained here.  Consider:

    ---------------------------------------------------------------------->
    (pymacs-eval "lisp('(pymacs-eval \"`2L**111`\")')")
    "2596148429267413814265248164610048L"
    ----------------------------------------------------------------------<

    Here, Emacs asks Python to ask Emacs to ask Python for a simple bignum
    computation.  Note that Emacs does not natively know how to handle
    big integers, nor has an internal representation for them.  This is
    why I use backticks, so Python returns a string representation of the
    result, instead of the result itself.  Here is a trace for this example.
    The `<' character flags a message going from Python to Emacs and is
    followed by an expression written in LISP.  The '>' character flags
    a message going from Emacs to Python and is followed by a expression
    written in Python.  The number gives the length of the message.

    ---------------------------------------------------------------------->
    <22   (pymacs-version "0.3")
    >49   eval("lisp('(pymacs-eval \"`2L**111`\")')")
    <25   (pymacs-eval "`2L**111`")
    >18   eval("`2L**111`")
    <47   (pymacs-reply "2596148429267413814265248164610048L")
    >45   reply("2596148429267413814265248164610048L")
    <47   (pymacs-reply "2596148429267413814265248164610048L")
    ----------------------------------------------------------------------<

    Python evaluation is done in the context of the `pymacs' module,
    so a mere `reply' really means `pymacs.reply'.  On the LISP side,
    there is no concept of module namespaces, so we use the `pymacs-'
    prefix as an attempt to stay clean.  Of course, users should ideally
    refrain from naming their LISP objects with a `pymacs-' prefix.

    `pymacs.reply' and `pymacs-reply' are special functions meant to indicate
    that an expected result is finally transmitted.  `pymacs.error' and
    `pymacs-error' are special functions that introduce a string which
    explains an exception which recently occurred.  `pymacs-expand' is
    a special function implementing the `copy()' methods of LISP handles
    or symbols.  In all other cases, the expression is a request for the
    other side, that request stacks until a corresponding reply is received.

    Part of the protocol manages memory, and this management generates
    some extra-noise in the `*Pymacs*' buffer.  Whenever Emacs passes a
    structure to Python, an extra pointer is generated on the Emacs side to
    inhibit garbage collection by Emacs.  Python garbage collector detects
    when the received structure is no longer needed on the Python side,
    at which time the next communication will tell Emacs to remove the
    extra pointer.  It works symmetrically as well, that is, whenever Python
    passes a structure to Emacs, an extra Python reference is generated to
    inhibit garbage collection on the Python side.  Emacs garbage collector
    detects when the received structure is no longer needed on the Emacs
    side, after which Python will be told to remove the extra reference.
    For efficiency, those allocation-related messages are delayed, merged and
    batched together within the next communication having another purpose.

. : Emacs usual debugging.

    If cross-calls between LISP and Python nest deeply, an error will raise
    successive exceptions alternatively on both size as requests unstack,
    and the diagnostic gets transmitted back and forth, slightly growing
    as we go.  So, errors will eventually be reported by Emacs.  I made
    no kind of effort to transmit the LISP backtrace on the Python side,
    as I do not see a purpose for it: all debugging is done within Emacs
    windows anyway.

    On recent Emacses, the Python backtrace gets displayed in the
    mini-buffer, and the LISP backtrace is simultaneously shown in the
    `*Backtrace*' window.  One useful thing is to allow to mini-buffer to
    grow big, so it has more chance to fully contain the Python backtrace,
    the last lines of which are often especially useful.  Here, I use:

    (setq resize-mini-windows t
          max-mini-window-height .85)

    in my `.emacs' file, so the mini-buffer may use 85% of the screen,
    and quickly shrinks when fewer lines are needed.  The mini-buffer
    contents disappear at the next keystroke, but you can recover the
    Python backtrace by looking at the end of the `*Messages*' buffer.
    In which case the `ffap' package in Emacs may be yet another friend!
    From the `*Messages*' buffer, once `ffap' activated, merely put the
    cursor on the file name of a Python module from the backtrace, and
    `C-x C-f RET' will quickly open that source for you.

. : Auto-reloading on save.

    I found useful to automatically `pymacs-load' some Python files whenever
    they get saved from Emacs.  Here is how I do it.  The code below assumes
    that Python files meant for Pymacs are kept in `~/share/emacs/python'.

    (defun fp-maybe-pymacs-reload ()
      (let ((pymacsdir (expand-file-name "~/share/emacs/python/")))
        (when (and (string-equal (file-name-directory buffer-file-name)
                                 pymacsdir)
                   (string-match "\\.py\\'" buffer-file-name))
          (pymacs-load (substring buffer-file-name 0 -3)))))
    (add-hook 'after-save-hook 'fp-maybe-pymacs-reload)

.. Exemples.

. : Paul Winkler's

.  , The problem

     Let's say I have a a module, let's call it manglers.py, containing
     this simple python function:


     def break_on_whitespace(some_string):
         words = some_string.split()
         return '\n'.join(words)


     The goal is telling Emacs about this function so that I can call it on
     a region of text and replace the region with the result of the call.
     And bind this action to a key, of course, let's say `[f7]'.

     The Emacs buffer ought to be handled in some way.  If this is not on
     the LISP side, it has to be on the Python side, but we cannot escape
     handling the buffer.  So, there is an equilibrium in the work to do
     for the user, that could be displaced towards LISP or towards Python.

.  , Python side

     Here is a first draft for the Python side of the problem:


     from pymacs import lisp

     def break_on_whitespace():
         start = lisp.point()
         end = lisp.mark(lisp.t)
         if start > end:
             start, end = end, start
         text = lisp.buffer_substring(start, end)
         words = text.split()
         replacement = '\n'.join(words)
         lisp.delete_region(start, end)
         lisp.insert(replacement)

     interactions = {break_on_whitespace: ''}


     For various stylistic reasons, this could be rewritten into:


     import pymacs, string
     lisp = pymacs.lisp
     interactions = {}

     def break_on_whitespace():
         start, end = lisp.point(), lisp.mark(lisp.t)
         words = lisp.buffer_substring(start, end).split()
         lisp.delete_region(start, end)
         lisp.insert(string.join(words, '\n'))

     interactions[break_on_whitespace] = ''


     The above relies, in particular, on the fact that for those LISP
     functions used here, `start' and `end' may be given in any order.

.  , Emacs side

     On the Emacs side, one would do:


     (pymacs-load "manglers")
     (global-set-key [f7] 'manglers-break-on-whitespace)
Something went wrong with that request. Please try again.