Please note that this project has been merged into CEDET proper. This repo will be maintained only until the next major release of CEDET, and then it will be deprecated. No new features will be delivered to this repo, instead please consider tracking CEDET master.
EDE-compdb is a library that enables the Emacs Development Environment (EDE), which is part of CEDET, to be used with a compilation database, as provided by build tools such as CMake or Ninja. This enables CEDET to be automatically configured for use to support parsing, navigation, completion, and so on. This is especially useful for C and C++ projects which are otherwise quite tricky to configure for use with CEDET and other libraries.
Features:
- Reads compilation database on-demand and provides include paths and preprocessor symbols to the Semantic parser
- For GCC-compatible compilers, reads the internal include paths (no need to use
semantic-gcc-setup) - Most EDE commands supported, such as C-c . c to build the current target (source file)
- Autoloads based on the presence of
compile_commands.jsonfile, with no additional configuration typically required - Supports multiple build configurations (eg Debug, Release) and corresponding build directories
- Supports Flymake and Flycheck, allowing source files to be checked on-the-fly with no additional setup required
- Easy integration with auto-complete-c-headers and company-c-headers for include file completion
- (For Ninja projects only) Automatically generates the compilation database as required without needing to generate a
compile_commands.jsonfile - (For Ninja projects only) Loads and caches available phony build targets (eg “test”) for convenient builds
License: GPLv2
The easiest method to install EDE-compdb is via the MELPA package repository. No further configuration is needed with this method.
Alternatively you can download/clone the ede-compdb repo to a suitable directory and add it to your emacs load-path. You should then it to your .emacs file after loading CEDET, using:
(require 'ede-compdb)In order to use EDE-compdb you need to ensure your project has a compilation database. If you are using CMake you can do something like the following:
$ cd .../yourproj $ cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE .
This will generate build files in the source directory, as well as a compile_commands.json file, which is the compilation database.
Note that if you are using the Ninja build tool, you don’t need to generate the compile_commands.json file, because ninja can generate the compilation database dynamically.
Now open any of your source files in emacs, and you should see all the includes correctly parsed. Use M-x semantic-c-describe-environment to check that all include paths are recognised correctly. If not, see Troubleshooting.
By default, EDE-compdb projects are automatically loaded when a compile_commands.json file or build.ninja file is found in the current directory, or in a parent directory. If you build your projects in the same directory as your source, no additional configuration is required.
EDE-compdb projects can also be created manually by invoking either the ede-compdb-project or ede-ninja-project functions.
In the most basic case, you can simply set the :file property to the relevant compilation database file, and the project will be loaded from the information in that file.
(ede-add-project-to-global-list
(ede-compdb-project "Myproj" :file "/path/to/src/compile_commands.json"))The examples above assume that you are building your binaries into the same directory as the project source code. If you want to use a different directory for the compilation database, then you can use a :compdb-file property in conjunction with a :file property, as follows:
(ede-add-project-to-global-list
(ede-compdb-project "Myproj"
:file "/path/to/src/CMakeLists.txt"
:compdb-file "/path/to/build/compile_commands.json"))EDE-compdb can be configured to use multiple build directories. This is desirable when developing for multiple configurations (eg a debug build for feature development, and a release build for performance/integration testing). Furthermore, the build directories can be either “in-source” or “out-of-source”, meaning they can be subdirectories of your project source tree, or elsewhere on the filesystem.
In EDE terminology, each of these different types of builds is a project configuration. The current configuration is referred to as the configuration default. In EDE-compdb, each project configuration is associated with a build directory, and the current configuration directory is used to locate the compilation database.
Multiple configurations and associated directories can be specified when an EDE-compdb project is created, using the :configurations and :configuration-directories properites. The :configuration-default property can be used to specify the current configuration, but if not present the first one in the list of configurations will be chosen.
Once the project is created, you can switch to a different configuration using the ede-project-configurations-set function, bound to C-c . b by default. You can also change directory for the current configuration by using ede-compdb-set-configuration-directory, which is bound to C-c . B by default.
Here we are creating an EDE-compdb project for a CMake-based source tree in ~/src/myproj. It can be built as either “debug” or “release”, with build.dbg and build.rel as the corresponding directories.
(ede-add-project-to-global-list
(ede-compdb-project "Myproj"
:file (expand-file-name "~/src/myproj/CMakeLists.txt")
:configurations '("debug" "release")
:configuration-directories '("build.dbg" "build.rel")
:compdb-file "compile_commands.json"
:build-command "cmake --build .."
))Note that we need to provide a :file property which corresponds to a file in the root of the source tree.
In this (admittedly complex) example, we have a possible four different types of build for each project. Each build type is assigned a separate directory, relative to the project root. At load time, we examine the project to see which, if any, of the build directories is present. This directory is selected as the build directory, and additionally we set the :configuration-default to the corresponding value.
Furthermore we’re using the EDE autoload mechanism to automatically create and load the project as required.
(defvar my-cmake-build-directories
'(("None" . "build")
("Debug" . "build.dbg")
("Release" . "build.rel")
("RelWithDebInfo" . "build.r+d")))
(defun my-load-cmake-project (dir)
"Creates a project for the given directory sourced at dir"
(let ((default-directory dir)
(config-and-dir (car (cl-member-if (lambda (c)
(file-readable-p
(expand-file-name "compile_commands.json" (concat dir (cdr c)))))
my-cmake-build-directories))))
(unless config-and-dir
(error "Couldn't determine build directory for project at %s" dir))
(ede-add-project-to-global-list
(ede-compdb-project
(file-name-nondirectory (directory-file-name dir))
:file (expand-file-name "CMakeLists.txt" dir)
:compdb-file (expand-file-name "compile_commands.json" (cdr config-and-dir))
:configuration-default (car config-and-dir)
:configuration-directories (mapcar #'cdr my-cmake-build-directories)
:configurations (mapcar #'car my-cmake-build-directories)
:build-command "cmake --build .."
))))
(defun vc-project-root (dir)
(require 'vc)
(let* ((default-directory dir)
(backend (vc-responsible-backend dir)))
(and backend (vc-call-backend backend 'root default-directory))))
(ede-add-project-autoload
(ede-project-autoload "CMake"
:file 'ede-compdb
:proj-file "CMakeLists.txt"
:proj-root 'vc-project-root
:load-type 'my-load-cmake-project
:class-sym 'ede-compdb-project))The current buffer can be compiled using the ede-compile-target function, which is bound to C-c . c by default.
When creating an EDE-compdb project, the :build-command attribute can be set to the command to be used to build the entire project. This is invoked with ede-compile-project, which is bound to C-c . C by default. This command is run from the current configuration directory.
When ede-ninja-project is used, some additional features are supported. EDE-compdb supports automatically loading the list of top-level phony projects, like “all” and “test”. These are often useful during development, and EDE-compdb makes these available for use via the ede-compile-selected command. This is bound to the “Build Other Target…” menu item and C-c . C-c by default. These phony targets are queried using ninja -t targets and cached per-project.
One of the limitations of using the compilation database is that it only contains the compilation commands for source files. However related source files such as header files are not generally compiled independently, hence are not inserted into the compilation database.
EDE-compdb works around this limitation using some heuristics to locate a compilation database entry for each buffer file. This is the process that is followed when a new file is opened within an existing EDE-compdb project.
- If the current buffer file is in the compilation database, that is used.
- If there is an “other” file associated with the current buffer which is also in the compilation database, that is used. The definition of an “other” file is almost exactly the same as that used by the the built-in emacs function
ff-get-other-file. By default,ff-get-other-filewill search the current directory for an equivalent.cppfile, so if the current buffer is visiting an.hppfile and the equivalent.cppfile is in the compilation database, that is used. Other directories can be searched, and indeed custom functions can be provided to search for arbitary files. - Otherwise the compilation database is searched, and the entry which has the longest common prefix with the current buffer file is used. So for example if you are visiting
src/bar.hpp, and there is an entry forsrc/foo.cpp, this will be used in preference tomain.cpp.
This technique ensures that every header file should be matched to a compilation database entry. To see the compilation database entry for a given header file, just compile it! (See Building above).
A compilation database provides a way for tools to get access to the compilation commands that are to be executed for a given source file. The following is an example of a compilation database entry:
{
"directory": "/home/user/llvm/build",
"command": "/usr/bin/clang++ -Irelative -DSOMEDEF=\"With spaces, quotes and \\-es.\" -c -o file.o file.cc",
"file": "file.cc"
},This information is very useful for tools like CEDET, as it enables the tool to unambiguously determine the include paths and preprocessor definitions for C and C++ source files. This information is otherwise quite difficult to determine automatically, and most current tools typically require it to be provided redundantly (eg once in the build tool input file and again in an EDE project).
When CEDET is able to use the information in a compilation database, it significantly simplifies the configuration and setup of a typical C/C++ project, and possibly helps with other languages/projects. Furthermore it helps to improve the accuracy of the parser and provide many other benefits besides.
So how is the compilation database generated? Several methods are possible:
- For CMake-based projects using the GNU Make build tool, there is the
CMAKE_EXPORT_COMPILE_COMMANDSoption (described above) which tells CMake to write out acompile_commands.jsonfile along with the generated Makefiles in the build directory. This file contains the entire compilation database for the project. - For projects using the Ninja build tool, the compilation database can be generated on-demand using the
-t compdbcommand. - The Build EAR (Bear) tool can generate a compilation database from any build system by sniffing the compiler commands as they are executed.
Use of the compilation database is becoming more and more common, particularly for those projects using the clang toolset.
EDE-compdb will rescan the compilation database when the ede-rescan-toplevel function (bound to C-c . g by default) is invoked.
Typically this should not be needed, because EDE-compdb detects when the compilation database has changed, and rescans it. Changes are detected by examining the size and modification date/time for the relevant file in the current build directory, which is the one specified by the :compdb-file slot. Generally this is set to compile_commands.json for regular EDE-compdb projects, and build.ninja for Ninja projects.
Note that changing build directories will often cause the compilation database to be rescanned, as it generally represents a detected change in size or modification date/time of the :compdb-file.
Each time the compilation database is rescanned, open buffers are updated to reference the corresponding compilation database entry, as described in the process above.
The hook ede-compdb-project-rescan-hook is called for every open buffer after the compilation database is rescanned.
With a small amount of customization, EDE-compdb can integrate with other packages to provide many additional benefits.
The ede-compdb-flymake-init function is suitable for use with flymake-mode, which enables on-the-fly compilation checking of the current buffer. To configure it, simply add the following to your emacs init file:
(require 'flymake)
(setq flymake-allowed-file-name-masks
(cons '("\\.[ch]pp$" ede-compdb-flymake-init)
flymake-allowed-file-name-masks))
(add-hook 'find-file-hook 'flymake-find-file-hook)This will enable the use of flymake for all .cpp and .hpp files. Header files are supported, as long as a matching source file can be located, as described above.
The standard clang and gcc checkers can be automatically configured using EDE-compdb. Currently there is no init function but the following example should be sufficient for most needs.
(require 'flycheck)
;; TODO: load lazily...
(require 'ede-compdb)
(defun flycheck-compdb-setup ()
(when (and ede-object (oref ede-object compilation))
(let* ((comp (oref ede-object compilation))
(cmd (get-command-line comp)))
;; Configure flycheck clang checker.
;; TODO: configure gcc checker also
(when (string-match " -std=\\([^ ]+\\)" cmd)
(setq-local flycheck-clang-language-standard (match-string 1 cmd)))
(when (string-match " -stdlib=\\([^ ]+\\)" cmd)
(setq-local flycheck-clang-standard-library (match-string 1 cmd)))
(when (string-match " -fms-extensions " cmd)
(setq-local flycheck-clang-ms-extensions t))
(when (string-match " -fno-exceptions " cmd)
(setq-local flycheck-clang-no-exceptions t))
(when (string-match " -fno-rtti " cmd)
(setq-local flycheck-clang-no-rtti t))
(when (string-match " -fblocks " cmd)
(setq-local flycheck-clang-blocks t))
(setq-local flycheck-clang-includes (get-includes comp))
(setq-local flycheck-clang-definitions (get-defines comp))
(setq-local flycheck-clang-include-path (get-include-path comp t))
)))
(add-hook 'ede-compdb-project-rescan-hook #'flycheck-compdb-setup)
(add-hook 'ede-minor-mode-hook #'flycheck-compdb-setup)
(add-hook 'after-init-hook #'global-flycheck-mode)Basically the idea is to populate the relevant flycheck variables from the current compilation database entry. The example above will work for the clang checker, and it can be trivially extended to work with the gcc and other checkers which rely on similar information.
The auto-complete-c-headers package provides auto-completion for C and C++ header files using the auto-complete library. To do this successfully, it needs to know the current include directories. EDE-compdb can be configured to provide this information, as in the following example:
(add-hook 'ede-minor-mode-hook (lambda ()
(setq achead:get-include-directories-function 'ede-object-system-include-path)))The company-c-headers package provides auto-completion for C and C++ header files using the company-mode library. It can be configured similarly to the above package:
(add-hook 'ede-minor-mode-hook (lambda ()
(setq company-c-headers-path-system 'ede-object-system-include-path)))Unfortunately there is too much variation between build systems to accomodate all of them with sensible defaults, and so you may find that EDE-compdb doesn’t work as intended. Here are a few steps you can take to investigate problems.
- When opening a source file, you should first check that the include paths are set correctly for your project. To do this, you can use the “Show include summary” mouse menu item, or M-x semantic-decoration-all-include-summary. This should tell you whether or not the include paths are being correctly read from the compilation database.
- Check the contents of your
*compdb: projname*buffer (whereprojnameis the name of your project). This should contain the compilation database for your project, in JSON format. If the buffer is not strictly JSON formatted, then it cannot be parsed by EDE-compdb. In particular, look for spurious output at the start or end of the buffer. - For Ninja projects, you may need to customise the project for certain build rules. These are generally specific to the generator, and hence there are no good defaults. Generally you should ensure that the output of
ninja -t compdb RULENAMEproduces JSON-formatted compilation database, whereRULENAMEis a compilation database rule. Check your generatedbuild.ninjafile for rule names if you don’t know what to use here. Once you have determined the correct build rule names, you can use them in your project by providing the:build-rulesargument, for example:
(ede-ninja-project projname
:file projfile
:compdb-file compdb-file
:build-rules '("icc14.0.3_cxx" "gcc4.8.3_cxx")
)There is an ert test suite which uses a sample CMake project, with a temporary directory as a build directory. CMake, Ninja and a C++ compiler are required to run these tests successfully.
To run the tests, you need to install Cask, and use make test.
- If you use “Summarize includes current buffer” you will NOT see the system include path for the buffer. The reason is that the include path is set on the target, and not on the project. However, the summarize function only prints out the system include path for the project, and not the target. Use M-x semantic-c-describe-environment instead.
- EDE-compdb only does very basic parsing of the GCC (or compatible) command line options, and doesn’t support any of the more esoteric GCC-specfic ones such as “-isysroot”, “-idirafter”, etc.
- Currently uses the json module for loading the compilation database. This can be slow for large projects. We should either speed it up, or somehow defer the work so that it is scheduled it idle time.
- Full Ninja target heirarchy parsing. Basically we can use the ninja -t targets tool to query the target heirarchy. We would need to insert the source targets into the heirarchy at the right locations.
- Support Debug/Run target. This doesn’t really make sense for an individual source file, but we should be able to prompt for or guess (as per the previous point) the appropriate executable.
- Automated setup of build directory. Given a compdb generator (eg cmake) we should be able to automate the setup of a new build directory. Ideally this would work for a new source tree.
Some comparable projects to ede-compdb: