DXR, A Code Search and Cross-Reference Tool
DXR is a source code browser and search engine, building on the work of LXR and MXR. It supports full-text and regex searches as well as structural queries like "Find all the callers of this function." Behind the scenes, it uses trigram indices, the re2 library, and static analysis data collected by instrumented compilers. DXR runs only on Linux for the moment.
DXR is fairly well documented, so whether you're looking to deploy
DXR, write plugins or templates for DXR have a look at the
docs/ folder. The documents in this folder is a great starting point
for anybody who wants to deploy and/or develop DXR.
To try out DXR on a HelloWorld-size project...
git clone --recursive git://github.com/mozilla/dxr.git cd dxr # (the top level of the repository, not the dxr folder within it) # If you want to customize VM configuration for your environment, run the # line below, and change the settings you need, e.g. turning off NFS: cp vagrantconfig_local.yaml-dist vagrantconfig_local.yaml vagrant up vagrant ssh
On the VM...
cd ~/dxr make cd ~/dxr/tests/test_basic make dxr-serve.py --all target
You need to bind to all interfaces (--all) or else you will be unable to get to the Vagrant box's test server from the host machine. Then, on your host box, surf to http://22.214.171.124:8000/, and poke around to your heart's content. You may have to substitute the IP address of your Vagrant machine.
Please note that the vagrant image is built against virtualbox 4.2.0. If your version is older, the image might not work as expected.
If, after trying a search, you see the error...
Server Error ..... Database error: no such module: trilite
SSH into the VM and, run
ldconfig as root to sort out the shared
library linking problem. Then restart
dxr-serve.py, and all should
work as expected.
Installing on a Real Machine
To install DXR on a machine of your choice, do the following
- Install the dependencies in
- You can either install dxr at the system level (requires root) or inside a virtualenv which is the recommended way.
For a system-level install, run
python2.6 setup develop(to hack on DXR itself) or
python2.6 setup.py installto install it.
The recommended way is to install it as an unprivileged (non-root) user. For this, use virtualenv:
virtualenv my-dxr-env source my-dxr-env/bin/activate # You'll have to repeat this line each time # you want to use DXR in a new shell. pip install -r requirements.txt python setup.py develop # Magic path munging will make this the # copy of Python in the virtualenv.
- Next, you need to build the various binaries that
makeat the top level should do the needful.
- It should build the
libtrilite.solibrary inside the
- It should build the
dxr/plugins/clangdirectory. You might have to edit the
makefilein that directory to make the locations of
tests/test_basicdirectory contains a small tree which you can build an index for to test your build.
cdinto this directory and run
make. If it successfully runs, your build is fine.
- Then run
LD_LIBRARY_PATH=/path/to/dxr/trilite dxr-serve.py tests/test_basic/targetto start the server and browse the code. Without the
LD_LIBRARY_PATH, it will complain about not being able to find the
trilitemodule when you do a search on the web interface. You can dispense with this requirement by installing the library globally on your machine: on Linux, move
/usr/local/lib, then run
Things To Do
This is a short list of major things that could be done to improve DXR.
- Refactor the database schema
- Remove plugin specific tables
- Provide data collection API for plugins (ie. no direct database access for plugins)
- Optimize schema for search queries
- Store extents as a blob in *_refs, types, functions, etc.
- Use trilite for functions, types, macros, etc.
- Store the transitive closure of indirect calls
- Port search logic to new schema
- Minor refactoring of template interface to minimize size
- Reuse annotations (i.e., don't store the same annotation for each line)
- Reuse menus (i.e., don't store the same menu for each occurrence)
- Minor UI fixes and cleanup
- Colorize highlighted matches (highlight with classes, .m1, .m2, ..., use merge-extents)
- Make the UI pretty
- Minimize template size (at 20 MiB for 20k lines, this is a real issue)