Skip to content
This repository

An intelligent source code browser

branch: master
Octocat-spinner-32 bin Add --all option to dxr-serve.py as a shorter way of saying --host 0.… December 12, 2013
Octocat-spinner-32 docs closes bugs #991990 & #991991 - url hash is consistent with ranges, c… April 07, 2014
Octocat-spinner-32 dxr Merge highlighting fixes. Fix bug 991991. Fix bug 991990. Close #274. April 07, 2014
Octocat-spinner-32 node_modules Add npm-lockdown and hashes of the npm packages. February 05, 2014
Octocat-spinner-32 tests Try to fix tests under clang 3.3. March 25, 2014
Octocat-spinner-32 trilite @ e64a2a1 Revert accidental change to the trilite pinning. February 03, 2014
Octocat-spinner-32 .gitignore Convert docs from markdown to reStructuredText with Sphinx. Close #260.… March 17, 2014
Octocat-spinner-32 .gitmodules vagrant is up and running December 10, 2012
Octocat-spinner-32 .hgignore Warning parser, fixes for tool compat and python versioning. July 21, 2010
Octocat-spinner-32 Gruntfile.js Fix Bug 964091, precompile nunjucks templates always, remove pointers… January 31, 2014
Octocat-spinner-32 LICENSE humph's code drop June 27, 2009
Octocat-spinner-32 MANIFEST.in Include the recently moved templates folder when installing DXR. Stop… January 27, 2014
Octocat-spinner-32 README.rst Convert docs from markdown to reStructuredText with Sphinx. Close #260.… March 17, 2014
Octocat-spinner-32 Vagrantfile Only !merge local vagrant params when present March 31, 2014
Octocat-spinner-32 deploy.py Deploy more quietly. March 14, 2014
Octocat-spinner-32 format Whoops. Increment format version since espressive changed layout.html… April 07, 2014
Octocat-spinner-32 jenkins.sh Add submodule syncing to the Jenkins script so we get our new, x86_64… April 22, 2013
Octocat-spinner-32 lockdown.json Add grunt-cli, and redo lockdown. February 05, 2014
Octocat-spinner-32 makefile Update default make target to compile nunjucks templates. March 14, 2014
Octocat-spinner-32 package.json Add grunt-cli, and redo lockdown. February 05, 2014
Octocat-spinner-32 peep.py Update to peep pre-1.0. March 27, 2014
Octocat-spinner-32 requirements.txt Upgrade Jinja to get the filesizeformat fix. Fix bug 965773. February 12, 2014
Octocat-spinner-32 setup.py Convert docs from markdown to reStructuredText with Sphinx. Close #260.… March 17, 2014
Octocat-spinner-32 vagrant_provision.sh Convert docs from markdown to reStructuredText with Sphinx. Close #260.… March 17, 2014
Octocat-spinner-32 vagrantconfig.yaml vagrant is up and running December 10, 2012
Octocat-spinner-32 vagrantconfig_local.yaml-dist Remove no-longer-used options from vagrantconfig. March 14, 2014
README.rst

DXR, A Code Search and Cross-Reference Tool

Introduction

DXR is a source code browser and search engine, building on the work of LXR and MXR. It supports full-text and regex searches as well as structural queries like "Find all the callers of this function." Behind the scenes, it uses trigram indices, the re2 library, and static analysis data collected by instrumented compilers. DXR runs only on Linux for the moment.

Documentation

DXR is fairly well documented, so whether you're looking to deploy DXR, write plugins or templates for DXR have a look at the docs/ folder. The documents in this folder is a great starting point for anybody who wants to deploy and/or develop DXR.

Quick Start

To try out DXR on a HelloWorld-size project...

git clone --recursive git://github.com/mozilla/dxr.git
cd dxr  # (the top level of the repository, not the dxr folder within it)

# If you want to customize VM configuration for your environment, run the
# line below, and change the settings you need, e.g. turning off NFS:
cp vagrantconfig_local.yaml-dist vagrantconfig_local.yaml

vagrant up
vagrant ssh

On the VM...

cd ~/dxr
make
cd ~/dxr/tests/test_basic
make
dxr-serve.py --all target

You need to bind to all interfaces (--all) or else you will be unable to get to the Vagrant box's test server from the host machine. Then, on your host box, surf to http://33.33.33.77:8000/, and poke around to your heart's content. You may have to substitute the IP address of your Vagrant machine.

Please note that the vagrant image is built against virtualbox 4.2.0. If your version is older, the image might not work as expected.

Troubleshooting

If, after trying a search, you see the error...

Server Error
.....
Database error: no such module: trilite

SSH into the VM and, run ldconfig as root to sort out the shared library linking problem. Then restart dxr-serve.py, and all should work as expected.

Installing on a Real Machine

To install DXR on a machine of your choice, do the following

  1. Install the dependencies in docs/deployment.mkd.
  2. You can either install dxr at the system level (requires root) or inside a virtualenv which is the recommended way.
  • For a system-level install, run python2.6 setup develop (to hack on DXR itself) or python2.6 setup.py install to install it.

  • The recommended way is to install it as an unprivileged (non-root) user. For this, use virtualenv:

    virtualenv my-dxr-env
    source my-dxr-env/bin/activate  # You'll have to repeat this line each time
                                    # you want to use DXR in a new shell.
    pip install -r requirements.txt
    python setup.py develop         # Magic path munging will make this the
                                    # copy of Python in the virtualenv.
    
  1. Next, you need to build the various binaries that dxr needs. Running make at the top level should do the needful.
  • It should build the libtrilite.so library inside the trilite directory.
  • It should build the libclang-index-plugin.so inside the dxr/plugins/clang directory. You might have to edit the makefile in that directory to make the locations of llvm-config, clang and clang++ accurate.
  1. The tests/test_basic directory contains a small tree which you can build an index for to test your build. cd into this directory and run make. If it successfully runs, your build is fine.
  2. Then run LD_LIBRARY_PATH=/path/to/dxr/trilite dxr-serve.py    tests/test_basic/target to start the server and browse the code. Without the LD_LIBRARY_PATH, it will complain about not being able to find the trilite module when you do a search on the web interface. You can dispense with this requirement by installing the library globally on your machine: on Linux, move libtrilite.so to /usr/local/lib, then run ldconfig.

Things To Do

This is a short list of major things that could be done to improve DXR.

  • Refactor the database schema
    • Remove plugin specific tables
    • Provide data collection API for plugins (ie. no direct database access for plugins)
    • Optimize schema for search queries
      • Store extents as a blob in *_refs, types, functions, etc.
      • Use trilite for functions, types, macros, etc.
      • Store the transitive closure of indirect calls
    • Port search logic to new schema
  • Minor refactoring of template interface to minimize size
    • Reuse annotations (i.e., don't store the same annotation for each line)
    • Reuse menus (i.e., don't store the same menu for each occurrence)
  • Minor UI fixes and cleanup
    • Colorize highlighted matches (highlight with classes, .m1, .m2, ..., use merge-extents)
    • Make the UI pretty
    • Add option in "Advanced" search to disable redirect (use cookies to remember setting)
    • Minimize template size (at 20 MiB for 20k lines, this is a real issue)
Something went wrong with that request. Please try again.