Skip to content
This repository has been archived by the owner on Oct 13, 2021. It is now read-only.

Commit

Permalink
Add a persistent /code dir for your own indexable code to live in.
Browse files Browse the repository at this point in the history
Rename the now-too-generic "data" container to "venv".
  • Loading branch information
erikrose committed Jan 29, 2016
1 parent ad06591 commit f3c5db8
Show file tree
Hide file tree
Showing 6 changed files with 48 additions and 27 deletions.
2 changes: 1 addition & 1 deletion docs/source/example-configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@
# Some global options here, if you like

[yourproject]
source_folder = /path/to/your/code
source_folder = /code/my-checkout
build_command = make clean; make -j {workers}
19 changes: 10 additions & 9 deletions docs/source/getting-started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ Getting Started
The easiest way to get DXR working on your own machine is...

1. Get the source code you want to index.
2. If it's C or C++, tell DXR how to build it.
2. If it a language analyzed at build time (like C++ or Rust), tell DXR how to
build it.
3. Run :program:`dxr index` to index your code.
4. Run :program:`dxr serve` to present a web-based search interface.

Expand All @@ -25,9 +26,9 @@ Configuration
=============

Before DXR can index your code, it needs to know where it is and, if you want
to be able to do structural queries (like find-all-the-callers) for C or C++,
how to kick off a build. (Analysis of more dynamic languages like Python does
not require a build step.) If you have a simple build process powered by
to be able to do structural queries (like find-all-the-callers) for C, C++, or
Rust, how to kick off a build. (Analysis of more dynamic languages like Python
does not require a build step.) If you have a simple build process powered by
:command:`make`, a configuration like this might suffice. Place the following
in a file called :file:`dxr.config`. The location of the file doesn't matter,
but the usual place is adjacent to your source directory.
Expand All @@ -38,11 +39,11 @@ but the usual place is adjacent to your source directory.

Be sure to replace the placeholder paths in the above config. You'll need to
move your code to be indexed into the VM, either by downloading it from
within the VM, or by copying it in through your DXR repository folder, where
it will be visible from within the VM in the shared ``~/dxr`` folder. It's
possible to index your code from a folder within ``~/dxr``, but for
performance and sanity we recommend keeping your personal code to be indexed
outside of ``~/dxr``.
within the VM, or by moving it into your DXR repository folder, where
it will be visible from within the VM in the shared ``~/dxr`` folder. It's
possible to index your code from a folder within ``~/dxr``, but, if you are
using a non-Linux host machine, moving it to :file:`/code` will give you
much faster IO by taking VirtualBox's shared-folder machinery out of the mix.

By building your project with clang and under the control of
:program:`dxr index`, DXR gets a chance to interpose a custom compiler
Expand Down
31 changes: 17 additions & 14 deletions docs/source/indexing-firefox.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,11 @@ from which Firefox is built.
Increase Your RAM
=================

Stop your containers, and increase the RAM and disk on your VM. The compilation
needs around 7GB. The temp files are 15GB, and the ES index and generated HTML
are also on that order. It's also a good idea to add more virtual CPUs, up to
the limit of your physical ones. On your host machine... ::
Stop your containers, and increase the RAM and disk on your docker-machine VM
(if using docker-machine). The compilation needs around 7GB. The temp files are
15GB, and the ES index and generated HTML are also on that order. It's also a
good idea to add more virtual CPUs, up to the limit of your physical ones. On
your host machine... ::

make docker_stop
docker-machine rm default
Expand All @@ -27,24 +28,26 @@ the limit of your physical ones. On your host machine... ::
Configure The Source Tree
=========================

1. Put mozilla-central checkout in a folder somewhere in the VM, let's say
:file:`~/src`. (If you put it somewhere else be sure your choice is reflected
in :file:`dxr.config` in Step 4.) You can use ``hg clone`` as documented at
https://developer.mozilla.org/en-US/docs/Simple_Firefox_build.
1. Put a mozilla-central checkout in :file:`/code` on the VM. This is a
special, blessed folder that will not evaporate when the docker container
exits. (If you decide to put it somewhere else, be sure your choice is
reflected in :file:`dxr.config` in Step 4.) You can use ``hg clone`` as
documented at https://developer.mozilla.org/en-US/docs/Simple_Firefox_build.

.. note::

Don't put :file:`src` in the mounted DXR repo itself, or IO bridging between
host and guest will kill your performance.
If using docker-machine and VirtualBox, keep your source code out of
:file:`/home/dxr/dxr`; VirtualBox's sharing of that folder between host and
guest will kill your performance.

2. Have the compiler include the debug code so it can be analyzed. Put this in
:file:`src/mozilla-central/mozconfig`::
:file:`/code/mozilla-central/mozconfig`::

ac_add_options --enable-debug

3. Get it ready to build::

cd src/mozilla-central
cd /code/mozilla-central
./mach bootstrap
./mach mercurial-setup

Expand All @@ -55,8 +58,8 @@ Configure The Source Tree
enabled_plugins=clang pygmentize

[mozilla-central]
source_folder=/home/dxr/src/mozilla-central
object_folder=/home/dxr/src/mozilla-central/obj-x86_64-unknown-linux-gnu
source_folder=/code/mozilla-central
object_folder=/code/mozilla-central/obj-x86_64-unknown-linux-gnu
build_command=cd $source_folder && ./mach clobber && make -f client.mk build MOZ_OBJDIR=$object_folder MOZ_MAKE_FLAGS="-s -j$jobs"

Bump Up Elasticsearch's RAM
Expand Down
9 changes: 9 additions & 0 deletions tooling/docker/code/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# A Data Volume Container to persist some source code to index across rebuilds
# of the "dev" image

# Same base as dev image, to save space. (It's copy-on-write.)
FROM ubuntu@sha256:0ca448cb174259ddb2ae6e213ebebe7590862d522fe38971e1175faedf0b6823

# Match UID with dev image:
RUN mkdir /code && chown -R 1000:1000 /code
VOLUME /code
14 changes: 11 additions & 3 deletions tooling/docker/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,17 @@ dev:
volumes:
- ../..:/home/dxr/dxr
volumes_from:
- data
- code
- venv

# A Data Volume Container to persist any code you might care to index. On a
# Linux host, you can squirrel away code in the shared "dxr" source checkout,
# but, on a Mac, /code will be more attractive because it offers better IO
# performance than VirtualBox's shared folder implementation.
code:
build: ./code

# A Data Volume Container so we don't lose our whole virtualenv and need to
# re-download all the Python packages every time the dev image is rebuilt
data:
build: ./data
venv:
build: ./venv
File renamed without changes.

0 comments on commit f3c5db8

Please sign in to comment.