msgpack_rpc.txt

*msgpack_rpc.txt*    For Nvim.						 {Nvim}


		 NVIM REFERENCE MANUAL    by Thiago de Arruda


The Msgpack-RPC Interface to Nvim				  *msgpack-rpc*

1. Introduction			|msgpack-rpc-intro|
2. API mapping			|msgpack-rpc-api|
3. Connecting			|msgpack-rpc-connecting|
4. Clients			|msgpack-rpc-clients|
5. Types			|msgpack-rpc-types|
6. Wrapping methods		|msgpack-rpc-wrap-methods|
7. Vimscript functions		|msgpack-rpc-vim-functions|

==============================================================================
1. Introduction						    *msgpack-rpc-intro*

The primary way to control a running Nvim instance is through
MessagePack-RPC, a messaging protocol that uses the MessagePack serialization
format: https://github.com/msgpack/msgpack/blob/7498cf3/spec.md.
From now on, we refer to the protocol as msgpack-rpc.

At this point, only plugins use msgpack-rpc, but eventually even user
interaction will happen through it, since user interfaces will be separate
programs that control a headless Nvim instance.

By connecting to the msgpack-rpc interface, programs can:

- Call any Nvim API function
- Listen for Nvim events
- Receive remote calls from Nvim

Nvim's msgpack-rpc interface is like a more powerful version of Vim's
`clientserver` feature.

==============================================================================
2. API mapping						      *msgpack-rpc-api*

The Nvim C API, see |nvim-api|, is automatically exposed to the msgpack-rpc
interface by the build system, which parses headers at src/nvim/api from the
project root. A dispatch function is generated, which matches msgpack-rpc method
names with non-static API functions, converting/validating arguments and return
values back to msgpack.

Client libraries will normally provide wrappers that hide msgpack-rpc details
from programmers. The wrappers can be automatically generated by reading
bundled API metadata from a compiled Nvim instance.

There are two ways to obtain API metadata:

1. By connecting to a running Nvim instance and calling `vim_get_api_info`
   via msgpack-rpc. This is best for clients written in dynamically-typed
   languages, which can define functions at runtime.

2. By starting Nvim with the `--api-info` command-line option, which makes Nvim
   dump a blob of msgpack metadata to standard output and exit. This is best
   for clients written in statically-typed languages, which require a separate
   compilation step.

Here's a simple way to get human-readable description of the API (requires
Python and the `pyyaml`/`msgpack-python` pip packages):
>
    nvim --api-info | python -c 'import msgpack, sys, yaml; print yaml.dump(msgpack.unpackb(sys.stdin.read()))' > api.yaml
<
==============================================================================
3. Connecting					       *msgpack-rpc-connecting*

There are four ways to open msgpack-rpc streams to Nvim:

1. Through Nvim's stdin/stdout when it's started with the `--embed` option.
   This is how other programs can embed Nvim.

2. Through the stdin/stdout of a program spawned by the |rpcstart()| function.

							*$NVIM_LISTEN_ADDRESS*
3. Through the socket automatically created with each instance. To get the
   socket location for a running Nvim instance (which is random by default),
   see the |$NVIM_LISTEN_ADDRESS| environment variable:
>
       :echo $NVIM_LISTEN_ADDRESS
<
   See also |v:servername|.

4. Through a TCP/IP socket. To make Nvim listen on a TCP/IP socket, set the
   |$NVIM_LISTEN_ADDRESS| environment variable in a shell before starting Nvim:
>
        NVIM_LISTEN_ADDRESS=127.0.0.1:6666 nvim
<
Connecting to the socket is the easiest way a programmer can test the API, which
can be done through any msgpack-rpc client library or fully-featured Nvim client
(which we'll see in the next section). Here's a Ruby script that prints 'hello
world!' in the current Nvim instance:
>
    #!/usr/bin/env ruby
    # Requires msgpack-rpc: gem install msgpack-rpc
    #
    # To run this script, execute it from a running Nvim instance (notice the
    # trailing '&' which is required since Nvim won't process events while
    # running a blocking command):
    #
    #	:!./hello.rb &
    #
    # Or from another shell by setting NVIM_LISTEN_ADDRESS:
    # $ NVIM_LISTEN_ADDRESS=[address] ./hello.rb

    require 'msgpack/rpc'
    require 'msgpack/rpc/transport/unix'

    nvim = MessagePack::RPC::Client.new(MessagePack::RPC::UNIXTransport.new, ENV['NVIM_LISTEN_ADDRESS'])
    result = nvim.call(:vim_command, 'echo "hello world!"')
<
A better way is to use the Python REPL with the `neovim` package, where API
functions can be called interactively:
>
    >>> from neovim import attach
    >>> nvim = attach('socket', path='[address]')
    >>> nvim.command('echo "hello world!"')
<
One can also spawn and connect to an embedded Nvim instance via |rpcstart()|
>
    let vim = rpcstart('nvim', ['--embed'])
    echo rpcrequest(vim, 'vim_eval', '"Hello " . "world!"')
    call rpcstop(vim)
<
==============================================================================
4. Implementing new clients				  *msgpack-rpc-clients*

Nvim is still in alpha, so there's no in-depth documentation explaining how to
properly implement a client library yet. The Python client (the pip package
"neovim") will always be up-to-date with the latest API changes, so its source
code is the best documentation currently available. There are some guidelines
however:

- Separate the transport layer from the rest of the library. See
  |msgpack-rpc-connecting| for details on how clients can connect to Nvim.
- Use a MessagePack library that implements at least version 5 of the
  MessagePack spec, which supports the `bin` and `ext` types used by Nvim.
- Read API metadata in order to create client-side wrappers for all
  msgpack-rpc methods.
- Use a single-threaded event loop library/pattern.
- Use a fiber/coroutine library for the language being used for implementing a
  client. These greatly simplify concurrency and allow the library to expose a
  blocking API on top of a non-blocking event loop without the complexity that
  comes with preemptive multitasking.
- Don't assume anything about the order that responses to msgpack-rpc requests
  will arrive.
- Clients should expect msgpack-rpc requests, which need to be handled
  immediately because Nvim is blocked while waiting for the client response.
- Clients should expect to receive msgpack-rpc notifications, but these don't
  need to be handled immediately because they won't block Nvim (although they
  should probably be handled immediately anyway).

Most of the complexity could be handled by a msgpack-rpc library that supports
server to client requests and notifications, but it's not clear if this is part
of the msgpack-rpc spec. At least the Ruby msgpack-rpc library does not seem
to support it:

https://github.com/msgpack-rpc/msgpack-rpc-ruby/blob/master/lib/msgpack/rpc/transport/tcp.rb#L150-L158

==============================================================================
5. Types						    *msgpack-rpc-types*

Nvim's C API uses custom types for all functions, se |nvim-api-types|.
For the purpose of mapping to msgpack, he types can be split into two groups:

- Basic types that map natively to msgpack (and probably have a default
  representation in msgpack-supported programming languages)
- Special Nvim types that map to msgpack EXT with custom type codes.

Basic type mapping:

Nil				  -> msgpack nil
Boolean				  -> msgpack boolean
Integer (signed 64-bit integer)	  -> msgpack integer
Float (IEEE 754 double precision) -> msgpack float
String				  -> msgpack string
Array				  -> msgpack array
Dictionary			  -> msgpack map

Special Nvim types that use msgpack EXT:

Buffer				  -> enum value kObjectTypeBuffer
Window				  -> enum value kObjectTypeWindow
Tabpage				  -> enum value kObjectTypeTabpage

An API method expecting one of these types may be passed an integer instead,
although they are not interchangeable. For example, a Buffer may be passed as
an integer, but not a Window or Tabpage.

The most reliable way of determining the type codes for the special Nvim types
is to inspect the `types` key of metadata dictionary returned by the
`vim_get_api_info` method at runtime. Here's an example JSON representation of
the `types` object:
>
  "types": {
    "Buffer": {
      "id": 0
    },
    "Window": {
      "id": 1
    },
    "Tabpage": {
      "id": 2
    }
  }
<
Even for statically compiled clients, it's a good practice to avoid hardcoding
the type codes, because a client may be built against one Nvim version but connect
to another with different type codes.

==============================================================================
6. Wrapping methods				     *msgpack-rpc-wrap-methods*

As mentioned before, clients should provide an API that hides msgpack-rpc
details from programmers, and the API metadata object contains information
that makes this task easier:

- The "functions" key contains a list of metadata objects for individual
  functions.
- Each function metadata object has type information about the return value
  and parameters. These can be used for generating strongly-typed APIs in
  static languages.
- Container types may be decorated with type/size constraints, e.g.
  ArrayOf(Buffer) or ArrayOf(Integer, 2). This can be useful to generate even
  more strongly-typed APIs.
- Methods that operate instances of Nvim's types are prefixed with the type
  name in lower case, e.g. `buffer_get_line` represents the `get_line` method
  of a Buffer instance.
- Global methods are prefixed with `vim`, e.g. `vim_get_buffers`.

So, for an object-oriented language, a client library would have the classes
that represent Nvim's types, and the methods of each class could be defined
by inspecting the method name prefix. There could also be a singleton Vim
class with methods mapped to functions prefixed with `vim_`

==============================================================================
7. Vimscript functions				   *msgpack-rpc-vim-functions*

Four msgpack-rpc functions are available in Vimscript:

1. |rpcstart()|: Similarly to |jobstart()|, this will spawn a co-process with
   its standard handles connected to Nvim. The difference is that it's not
   possible to process raw data to or from the process's stdin, stdout, or
   stderr.  This is because the job's stdin and stdout are used as a single
   msgpack channel that is processed directly by Nvim.

2. |rpcstop()|: Same as |jobstop()|, but operates on handles returned by
   |rpcstart()|.

3. |rpcrequest()|: Sends a msgpack-rpc request to the process.

4. |rpcnotify()|: Sends a msgpack-rpc notification to the process.

The last two functions may also be used with channels created from
connections to |$NVIM_LISTEN_ADDRESS|.

==============================================================================
 vim:tw=78:ts=8:noet:ft=help:norl: