Skip to content
A code-completion & code-comprehension server
C++ Python C CMake Shell M4 Other
Branch: master
Clone or download
mergify Merge pull request #1382 from bstaletic/no_py2
[READY] Drop python2 support
Latest commit 38bb02f Jan 16, 2020
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
azure Remove frozendict Nov 24, 2019
cpp Drop python2 support Jan 8, 2020
docs Document API changes Nov 7, 2019
examples Drop python2 support Jan 8, 2020
third_party Drop python2 support Jan 8, 2020
ycmd Report running in python2 through an exit code Jan 8, 2020
.clang-tidy Pass pybind11 types by value Jul 12, 2019
.coveragerc Enable Python subprocesses coverage Jul 22, 2016
.gitignore Ignore clangd index output Sep 18, 2019
.gitmodules Drop python2 support Jan 8, 2020
.mergify.yml Use ycm-core for azure build Jun 25, 2019
.vimspector.json Vimspector: Rmemove duplication of python path Jul 27, 2019
.ycm_extra_conf.py Add numpydoc to Jedi dependencies Apr 8, 2019
CODE_OF_CONDUCT.md Adding an explicit Contributor Code of Conduct Jan 26, 2016
CONTRIBUTING.md Drop python2 support Jan 8, 2020
COPYING.txt Adding the license file May 17, 2014
CORE_VERSION Fix system headers search on macOS Jan 26, 2019
DEBUG.md Drop Vagrant setup Feb 26, 2018
JAVA_SUPPORT.md PR description as a md file for posterity Jan 21, 2018
README.md Report running in python2 through an exit code Jan 8, 2020
TESTS.md Drop Vagrant setup Feb 26, 2018
azure-pipelines.yml Drop python2 support Jan 8, 2020
benchmark.py Drop python2 support Jan 8, 2020
build.py Drop python2 support Jan 8, 2020
codecov.yml Update coverage.py Apr 17, 2019
run_tests.py Drop python2 support Jan 8, 2020
style_format.sh Removing ycm_client_support Feb 23, 2016
test_requirements.txt Vimspector: Rmemove duplication of python path Jul 27, 2019
tox.ini Fix Flake8 errors Oct 26, 2018
update_api_docs.py Drop python2 support Jan 8, 2020
update_boost.py Drop python2 support Jan 8, 2020
update_clang.py Drop python2 support Jan 8, 2020
update_omnisharp.py Drop python2 support Jan 8, 2020
update_unicode.py Drop python2 support Jan 8, 2020

README.md

ycmd: a code-completion & comprehension server

Build status Coverage status

ycmd is a server that provides APIs for code-completion and other code-comprehension use-cases like semantic GoTo commands (and others). For certain filetypes, ycmd can also provide diagnostic errors and warnings.

ycmd was originally part of YouCompleteMe's codebase, but has been split out into a separate project so that it can be used in editors other than Vim.

Check the API documentation if you want to implement a client. A good way to learn how to interact with ycmd is by reading through (and running) the example_client.py file. See the README for the examples folder for details on how to run the example client.

Known ycmd clients:

Feel free to send a pull request adding a link to your client here if you've built one.

Building

If you're looking to develop ycmd, see the instructions for running the tests.

This is all for Ubuntu Linux. Details on getting ycmd running on other OS's can be found in YCM's instructions (ignore the Vim-specific parts). Note that ycmd runs on Python 3.5.1+.

First, install the minimal dependencies:

sudo apt install build-essential cmake python3-dev

Next, install the language specific dependencies you need:

  • sudo apt install golang-go for Go.
  • sudo apt install npm for JavaScript and TypeScript.
  • sudo apt install mono-devel for C#.
  • sudo apt install openjdk-8-jre for Java.

When you first clone the repository you'll need to update the submodules:

git submodule update --init --recursive

Then run python3 build.py --all or any of the specific completers listed by python3 build.py --help. This should get you going.

For more detailed instructions on building ycmd, see YCM's instructions (ignore the Vim-specific parts).

Supported compilers

  • GCC 4.8 and later
  • Clang 3.3 and later
  • Microsoft Visual Studio 2015 Update 3 and later

API notes

  • All strings going into and out of the server are UTF-8 encoded.
  • All lines end with \n.
  • All line and column numbers are 1-based, not 0-based. They are also byte offsets, not Unicode codepoint offsets.
  • All file paths are full, absolute paths.
  • All requests to the server must include an HMAC in the x-ycm-hmac HTTP header. The HMAC is computed from the shared secret passed to the server on startup and the request/response body. The digest algorithm is SHA-256. The server will also include the HMAC in its responses; you must verify it before using the response. See example_client.py to see how it's done.
  • API is documented in swagger and published on the website.

How ycmd works

There are several completion engines in ycmd. The most basic one is an identifier-based completer that collects all of the identifiers in the file provided in the completion request, other files of the same filetype that were provided previously and any tags files produced by ctags. This engine is non-semantic.

There are also several semantic engines in YCM. There's a libclang-based completer and clangd-based completer that both provide semantic completion for C-family languages. clangd support is currently experimental and changes in the near future might break backwards compatibility. There's also a Jedi-based completer for semantic completion for Python, an OmniSharp-based completer for C#, a Gocode-based completer for Go (using Godef for jumping to definitions), a TSServer-based completer for JavaScript and TypeScript, a jdt.ls-based server for Java, and a RLS-based completer for Rust. More will be added with time.

There are also other completion engines, like the filepath completer (part of the identifier completer).

The server will automatically detect which completion engine would be the best in any situation. On occasion, it queries several of them at once, merges the outputs and presents the results.

Semantic engines are triggered only after semantic "triggers" are inserted in the code. If the request received shows that the user's cursor is after the last character in string foo; foo. in a C# file, this would trigger the semantic engine to examine members of foo because . is a default semantic trigger for C# (triggers can be changed dynamically). If the text were string foo; foo.zoo, semantic completion would still be triggered (the trigger is behind the zoo word the user is typing) and the results would be filtered with the zoo query.

Semantic completion can also be forced by setting force_semantic: true in the JSON data for the completion request, but you should only do this if the user explicitly requested semantic completion with a keyboard shortcut; otherwise, leave it up to ycmd to decide when to use which engine.

The reason why semantic completion isn't always used even when available is because the semantic engines can be slow and because most of the time, the user doesn't actually need semantic completion.

There are two main use-cases for code-completion:

  1. The user knows which name they're looking for, they just don't want to type the whole name.
  2. The user either doesn't know the name they need or isn't sure what the name is. This is also known as the "API exploration" use-case.

The first use case is the most common one and is trivially addressed with the identifier completion engine (which BTW is blazing fast). The second one needs semantic completion.

Completion string filtering

A critical thing to note is that the completion filtering is NOT based on the input being a string prefix of the completion (but that works too). The input needs to be a subsequence match of a completion. This is a fancy way of saying that any input characters need to be present in a completion string in the order in which they appear in the input. So abc is a subsequence of xaybgc, but not of xbyxaxxc.

Completion string ranking

The subsequence filter removes any completions that do not match the input, but then the sorting system kicks in. It's a bit involved, but roughly speaking "word boundary" (WB) subsequence character matches are "worth" more than non-WB matches. In effect, this means given an input of "gua", the completion "getUserAccount" would be ranked higher in the list than the "Fooguxa" completion (both of which are subsequence matches). A word-boundary character are all capital characters, characters preceded by an underscore and the first letter character in the completion string.

Auto-shutdown if no requests for a while

If the server hasn't received any requests for a while (controlled by the --idle_suicide_seconds ycmd flag), it will shut itself down. This is useful for cases where the process that started ycmd dies without telling ycmd to die too or if ycmd hangs (this should be extremely rare).

If you're implementing a client for ycmd, ensure that you have some sort of keep-alive background thread that periodically pings ycmd (just call the /healthy handler, although any handler will do).

You can also turn this off by passing --idle_suicide_seconds=0, although that isn't recommended.

Exit codes

During startup, ycmd attempts to load the ycm_core library and exits with one of the following return codes if unsuccessful:

  • 3: unexpected error while loading the library;
  • 4: the ycm_core library is missing;
  • 7: the version of the ycm_core library is outdated.
  • 8: server is started with python; recompile with python3.

User-level customization

You can provide settings to ycmd on server startup. There's a default_settings.json file that you can tweak. See the Options section in YCM's User Guide for a description on what each option does. Pass the path to the modified settings file to ycmd as an --options_file=/path/to/file flag. Note that you must set the hmac_secret setting (encode the value with base64). Because the file you are passing contains a secret token, ensure that you are creating the temporary file in a secure way (the mkstemp() Linux system call is a good idea; use something similar for other OS's).

After it starts up, ycmd will delete the settings file you provided after it reads it.

The settings file is something your editor should produce based on values your user has configured. There's also an extra file (.ycm_extra_conf.py) your user is supposed to provide to configure certain semantic completers. More information on it can also be found in the corresponding section of YCM's User Guide.

.ycm_extra_conf.py specification

The .ycm_extra_conf.py module may define the following functions:

Settings( **kwargs )

This function allows users to configure the language completers on a per project basis or globally. Currently, it is required by the libclang-based completer and optional for other completers. The following arguments can be retrieved from the kwargs dictionary and are common to all completers:

  • language: an identifier of the completer that called the function. Its value is python for the Python completer and cfamily for the C-family completer. This argument is useful to configure several completers at once. For instance:

    def Settings( **kwargs ):
      language = kwargs[ 'language' ]
      if language == 'cfamily':
        return {
          # Settings for the libclang and clangd-based completer.
        }
      if language == 'python':
        return {
          # Settings for the Python completer.
        }
      return {}
  • filename: absolute path of the file currently edited.

  • client_data: any additional data supplied by the client application. See the YouCompleteMe documentation for an example.

The return value is a dictionary whose content depends on the completer.

LSP based completers

LSP servers often support user configuration via the initialise request. These are usually presented as options in the UI. Ycmd supports this using the .ycm_extra_conf.py by allowing the user to specify the exact dictionary of settings that are passed in the server initialise message. These options are returned from Settings under the ls key. The python dictionary is converted to json and included verbatim in the LSP initialize request. In order to determine the set of options for a server, consult the server's documentation or package.json file.

Example of LSP configuration:

def Settings( **kwargs ):
  if kwargs[ 'language' ] == 'java':
    return { 'ls': { 'java.rename.enabled' : False } }

In addition, ycmd can use any language server, given a file type and a command line. A user option language_server can be used to plug in a LSP server ycmd wouldn't usually know about. The value is a list of dictionaries containing:

  • name: the string representing the name of the server
  • cmdline: the list representing the command line to execute the server
  • filetypes: list of supported filetypes.
  • project_root_files: Tells ycmd which files indicate project root.
{
  "language_server": [ {
    "name": "gopls",
    "cmdline": [ "/path/to/gopls", "-rpc.trace" ],
    "filetypes": [ "go" ],
    "project_root_files": [ "go.mod" ]
  } ]
}

When plugging in a completer in this way, the kwargs[ 'language' ] will be set to the value of the name key, i.e. gopls in the above example.

LSP completers currently supported without language_server:

  • Java
  • Rust
  • Go
  • C-family
C-family settings

The Settings function is called by the libclang and clangd-based completers to get the compiler flags to use when compiling the current file. The absolute path of this file is accessible under the filename key of the kwargs dictionary.

The return value expected by both completers is a dictionary containing the following items:

  • flags: (mandatory for libclang, optional for clangd) a list of compiler flags.

  • include_paths_relative_to_dir: (optional) the directory to which the include paths in the list of flags are relative. Defaults to ycmd working directory for the libclang completer and .ycm_extra_conf.py's directory for the clangd completer.

  • do_cache: (optional) a boolean indicating whether or not the result of this call (i.e. the list of flags) should be cached for this file name. Defaults to True. If unsure, the default is almost always correct.

The libclang-based completer also supports the following items:

  • override_filename: (optional) a string indicating the name of the file to parse as the translation unit for the supplied file name. This fairly advanced feature allows for projects that use a 'unity'-style build, or for header files which depend on other includes in other files.

  • flags_ready: (optional) a boolean indicating that the flags should be used. Defaults to True. If unsure, the default is almost always correct.

A minimal example which simply returns a list of flags is:

def Settings( **kwargs ):
  return {
    'flags': [ '-x', 'c++' ]
  }
Java settings

The java subserver allows formatter configuration, but it expects the configuration to be supplied in a different way than the rest. For this purpose the Settings function can return a formatter property.

An example of the formatter configuration would be:

def Settings( **kwargs ):
  return {
    'formatting_options': {
       'org.eclipse.jdt.core.formatter.lineSplit': 30, 
    }
  }
Python settings

The Settings function allows users to specify the Python interpreter and the sys.path used by the completer to provide completion and code comprehension. No additional arguments are passed.

The return value expected by the completer is a dictionary containing the following items:

  • interpreter_path: (optional) path to the Python interpreter. ~ and environment variables in the path are expanded. If not an absolute path, it will be searched through the PATH.

  • sys_path: (optional) list of paths prepended to sys.path.

Usage example:

def Settings( **kwargs ):
  return {
    'interpreter_path': '~/project/virtual_env/bin/python',
    'sys_path': [ '~/project/third_party/module' ]
  }

PythonSysPath( **kwargs )

Optional for Python support.

This function allows further customization of the Python path sys.path. Its parameters are the possible items returned by the Settings function for the Python completer:

  • interpreter_path: path to the Python interpreter.

  • sys_path: list of Python paths from sys.path.

The return value should be the modified list of Python paths.

See ycmd's own .ycm_extra_conf.py for an example.

Global extra conf file specification

The global extra module must expose the same functions as the .ycm_extra_conf.py module with the following additions:

YcmCorePreLoad()

Optional.

This method, if defined, is called by the server prior to importing the c++ python plugin. It is not usually required and its use is for advanced users only.

Shutdown()

Optional.

Called prior to the server exiting cleanly. It is not usually required and its use is for advanced users only.

Backwards compatibility

ycmd's HTTP+JSON interface follows SemVer. While ycmd has seen extensive use over the last several years as part of YCM, the version number is below 1.0 because some parts of the API might change slightly as people discover possible problems integrating ycmd with other editors. In other words, the current API might unintentionally be Vim-specific. We don't want that.

Note that ycmd's internal API's (i.e. anything other than HTTP+JSON) are NOT covered by SemVer and will randomly change underneath you. DON'T interact with the Python/C++/etc code directly!

FAQ

Is HMAC auth for requests/responses really necessary?

Without the HMAC auth, it's possible for a malicious website to impersonate the user. Don't forget that evil.com can send requests to servers listening on localhost if the user visits evil.com in a browser.

This is not merely a theoretical concern; a working proof-of-concept remote code execution exploit was created for ycmd running on localhost. The HMAC auth was added to block this attack vector.

Contributor Code of Conduct

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

Contact

If you have questions about the plugin or need help, please use the ycmd-users mailing list.

The author's homepage is http://val.markovic.io.

License

This software is licensed under the GPL v3 license. © 2015-2019 ycmd contributors

You can’t perform that action at this time.