How to install OpenGrok

Kryštof Tulinger edited this page Feb 21, 2017 · 38 revisions

{OpenGrok can be installed and used under different use cases, we will just show how to use the wrapper script. Advanced usage depends on your knowledge of running java applications and command line options of {OpenGrok. Note, that you need the index setup no matter what use case you plan to use. Without indexes {Opengrok will be simply useless. This document is a live document, we are updating it as the new versions/changes come out (note the date at the right bottom of the page)

Table of Contents

Requirements

You need the following:

  • JDK 1.8 or higher
  • {OpenGrok binaries from https://github.com/OpenGrok/OpenGrok/releases (either the package for Solaris, or .tar.gz with binaries, NOT the src !)
  • Exuberant Ctags for analysis (https://github.com/universal-ctags is recommended)
  • A servlet container like GlassFish or Tomcat (8.x or later) also running with java at least 1.8
  • If history is needed, appropriate binary (in some cases also cvs/svn repository) must be present on the system (e.g. Subversion or Mercurial or SCCS or ... )
  • 2GB of memory for indexing process using OpenGrok script (can use less, this is scaled for bigger deployments)
  • a recent browser for clients - IE, Firefox, recent Chrome or Safari
  • sufficient ULIMIT settings (refer to README)

Recommended links to read for advanced users

INDEX and web application setup

After installing the package (e.g. pkgadd -d ./OSOLopengrok-0.?.pkg) or untgzing the binaries in your target directory (e.g. cd your_target_dir ; gzcat opengrok-0.?.tar.gz | tar xf - ), you just need to create the index and then you can use the command line interface to grok your sources.

It is also necessary to install SCM environment accessible by OpenGrok for your repositories for the history indexing to work.

Creating the index

OpenGrok's command line use is a typical case where OpenGrok is deployed as indexing service on a webserver for an active source repository, which is updated automatically using scheduled cron jobs.

Project concept - one project is one directory underneath SRC_ROOT and usually contains a checkout of a project (or it's branch, version, ...) sources, it can have several attributes (in its XML description), note that interface of projects is being stabilized so it can change. Projects effectively replace need for more web applications with opengrok .war and leave you with one indexer and one web application serving MORE source code repositories - projects. A nice concept is to have directories underneath SRC_ROOT with a naming convention, thereby creating a good overview of projects (e.g. name-version-branch). Then you have a simple update script & simple index refresher script in place, which simplifies management of more repositories.

Step.0 - Setting up the Sources. Having the web application container ready.

Source base must be available locally for OpenGrok to work efficiently. No changes are required to your source tree. If the code is under CVS or SVN, OpenGrok requires the checked out source tree under SRC_ROOT.

Note: A local CVSROOT (or SVN if opengrok version less than 0.7) repository must be available. File history will not be fetched from a remote server for CVS (& SVN if opengrok version less than 0.7) !. For newer opengrok versions you must use option -r on for remote repositories history to be indexed, also note this can be more resource (time,cpu,network) demanding. Note also that OpenGrok ignores symbolic links.

Please install web application container of your choice (e.g. Tomcat,GlassFish or Sun Java Application Server) before going to the next step and make sure it's started.

Step.1 - Deploy the web application

We provided you with OpenGrok wrapper script, which should aid in deploying the web application. Please change to opengrok directory (can vary on your system)

 # cd /usr/opengrok/bin

and run

 # ./OpenGrok deploy

This command will do some sanity checks and will deploy the source.war in its directory to one of detected web application containers. Please follow the error message it provides. If it fails to discover your container, please refer to optional steps on changing web application properties, which has manual steps on how to do this. Alternatively use OPENGROK_TOMCAT_BASE environment variable, e.g.

 # OPENGROK_TOMCAT_BASE=/path/to/my/tomcat/install ./OpenGrok deploy

Note that OpenGrok script expects the directory /var/opengrok to be available to user running opengrok with all permissions. In root user case it will create all the directories needed, otherwise you have to manually create the directory and grant all permissions to the user used.

Step.2 - Populate DATA_ROOT Directory, let the indexer generate the project XML config file, update configuration.xml to your web app

Second step is to just run the indexing (can take a lot of time). After this is done, indexer automatically attempts to upload newly generated configuration to the web application. Most probably you will not be able to use {Opengrok before this is done.

Please change to opengrok directory (can vary on your system)

 $ cd /usr/opengrok/bin

and run, if your SRC_ROOT is prepared under /var/opengrok/src

 $ ./OpenGrok index

otherwise (if SRC_ROOT is in different directory) run:

 $ ./OpenGrok index <absolute_path_to_your_SRC_ROOT>

NOTE: Please DON'T use symlinks to /var/opengrok/src - rather use above command with parameter.

Above command should try to upload latest index status reflected into configuration.xml to a running source web application. Once above command finishes without errors(e.g. SEVERE: Failed to send configuration to localhost:2424 ), you should be able to enjoy your opengrok and search your sources using latest indexes and setup.

It is assumed that any SCM commands are reachable in one of the components of the PATH environment variable (e.g. 'git' command for Git repositories). Likewise, this should be maintained in the environment of the user which runs the web server instance.

Congratulations, you should now be able to point your browser to http://YOUR_WEBAPP_SERVER:WEBAPPSRV_PORT/source to work with your fresh opengrok installation! :-)

At this time we'd like to point out some customization to OpenGrok script for advanced users. A common case would be, that you want the data in some other directory than /var/opengrok. This can be easily achieved by using environment variable OPENGROK_INSTANCE_BASE . E.g. if my opengrok data directory is /tank/opengrok and my source root is in /tank/source and I'd like to get more verbosity I'd run the indexer as:

 $ OPENGROK_VERBOSE=true OPENGROK_INSTANCE_BASE=/tank/opengrok ./OpenGrok index /tank/source

Since above will also change default location of config file, beforehands(or restart your web container after creating this symlink) I suggest doing below for our case of having opengrok instance in /tank/opengrok :

 # ln -s /tank/opengrok/etc/configuration.xml /var/opengrok/etc/configuration.xml

A lot more customizations can be found inside the script, you just need to have a look at it, eventually create a configuration out of it and use OPENGROK_CONFIGURATION environment variable to point to it. Obviously such setups can be used for nightly cron job updates of index or other automated purposes.

Large number of projects on Tomcat

If you have a large number of projects the http headers start to get big and you need to increase this in the settings.xml file in the http connector add:

 maxHttpHeaderSize="65536"

Failure to do so will result in 400 errors after first query - with the error "Error parsing HTTP request header"

Optional info

Additional information can be found in the EXAMPLE.txt in the doc folder

Windows installation

http://algopadawan.blogspot.co.uk/2012/07/installing-opengrok-on-windows.html

CLI - Command Line Interface Usage

You need to pass location of project file + the query to Search class, e.g. for fulltext search for project with above generated configuration.xml you'd do:

 $ java -cp ./opengrok.jar org.opensolaris.opengrok.search.Search -R /var/opengrok/etc/configuration.xml -f fulltext_search_string

For quick help run:

 $ java -cp ./opengrok.jar org.opensolaris.opengrok.search.Search

Sample search:

Optional need to change web application properties or name

You might need to modify the web application if you don't store the configuration file in the default location (/var/opengrok/etc/configuration.xml).

To configure the webapp source.war, look into the parameters defined in WEB-INF/web.xml of source.war (use jar or zip/unzip or your preffered zip tool to get into it - e.g. extract the web.xml file from source.war ($ unzip source.war WEB-INF/web.xml) file, edit web.xml and re-package the jar file (zip -u source.war WEB-INF/web.xml) ) file and change those web.xml parameters appropriately. These sample parameters need modifying.

  • CONFIGURATION - the absolute path to XML file containing project configuration (e.g. /var/opengrok/etc/configuration.xml)
  • ConfigAddress - port for remote updates to configuration, optional, but advised (since there is no authentification) to be set to localhost:<some_port></some_port> (e.g. localhost:2424), if you choose some_port below 1024 you have to have root privileges.

If you need to change name of the web application from source to something else you need to use special option &#45;w &lt;new_name&gt; for indexer to create proper xrefs, besides changing the .war file name. Examples below show just deploying source.war, but you can use it to deploy your new_name.war too.

Deploy the modified .war file in glassfish/Sun Java App Server:

  • Option 1: Use browser and log into glassfish web administration interface
Common Tasks / Applications / Web Applications , button Deploy and point it to your source.war webarchive
  • Option 2: Copy the source.war file to //GLASSFISH///domains///YOURDOMAIN///autodeploy directory, glassfish will try to deploy it "automagically".
  • Option 3: Use cli from //GLASSFISH// directory:
 # ./bin/asadmin deploy /path/to/source.war

Deploy the modified .war file in tomcat:

  • just copy the source.war file to //TOMCAT_INSTALL///webapps directory.

Optional setup of security manager for tomcat

On some linux distribution you need to setup permissions for SRC_ROOT and DATA_ROOT. Please check your Tomcat documentation on this, or simply disable (your risk!) security manager for Tomcat (e.g. in debian/ubuntu : in file /etc/default/tomcat5.5 set TOMCAT5_SECURITY&#61;no). A sample approach is to edit /etc/tomcat5.5/04webapps.policy (or /var/apache/tomcat/conf/catalina.policy) and set this (it will give opengrok all permissions) for your opengrok webapp instance:

grant codeBase &quot;file&#58;$&#123;catalina.home&#125;/webapps/source/&#45;&quot; &#123;     
permission java.security.AllPermission&#59;&#125;&#59;
grant codeBase &quot;file&#58;$&#123;catalina.home&#125;/webapps/source/WEB&#45;INF/lib/&#45;&quot; &#123;     
permission java.security.AllPermission&#59;&#125;&#59;

Alternatively you can be more restrictive (haven't tested below with a complex setup(e.g. some versioning system which needs local access as cvs), if it will not work, please report through Discussions.

grant codeBase &quot;file&#58;$&#123;catalina.home&#125;/webapps/source/&#45;&quot; &#123;  
permission java.util.PropertyPermission &quot;subversion.native.library&quot;, &quot;read&quot;&#59;  
permission java.lang.RuntimePermission &quot;loadLibrary.svnjavahl&#45;1?&#59;  
permission java.lang.RuntimePermission &quot;loadLibrary.libsvnjavahl&#45;1?&#59;  
permission java.lang.RuntimePermission &quot;loadLibrary.svnjavahl&quot;&#59;  
permission java.util.PropertyPermission &quot;disableLuceneLocks&quot;, &quot;read&quot;&#59;  
permission java.util.PropertyPermission &quot;catalina.home&quot;, &quot;read&quot;&#59;  
permission java.util.PropertyPermission &quot;java.io.tmpdir&quot;, &quot;read&quot;&#59;  
permission java.util.PropertyPermission &quot;org.apache.lucene.lockdir&quot;, &quot;read&quot;&#59;  
permission java.util.PropertyPermission &quot;org.apache.lucene.writeLockTimeout&quot;, &quot;read&quot;&#59;  
permission java.util.PropertyPermission &quot;org.apache.lucene.commitLockTimeout&quot;, &quot;read&quot;&#59;  
permission java.util.PropertyPermission &quot;org.apache.lucene.mergeFactor&quot;, &quot;read&quot;&#59;  
permission java.util.PropertyPermission &quot;org.apache.lucene.minMergeDocs&quot;, &quot;read&quot;&#59;  
permission java.util.PropertyPermission &quot;org.apache.lucene.&#42;&quot;, &quot;read&quot;&#59;  
permission java.io.FilePermission &quot;/var/lib/tomcat5/temp&quot;, &quot;read&quot;&#59;  
permission java.io.FilePermission &quot;/var/lib/tomcat5/temp/&#42;&quot;, &quot;write&quot;&#59;  
permission java.io.FilePermission &quot;/var/lib/tomcat5/temp/&#42;&quot;, &quot;delete&quot;&#59;&#125;&#59;
grant codeBase &quot;file&#58;$&#123;catalina.home&#125;/webapps/source/WEB&#45;INF/lib/&#45;&quot; &#123;  
permission java.util.PropertyPermission &quot;subversion.native.library&quot;, &quot;read&quot;&#59;  
permission java.lang.RuntimePermission &quot;loadLibrary.svnjavahl&#45;1?&#59;  
permission java.util.PropertyPermission &quot;disableLuceneLocks&quot;, &quot;read&quot;&#59;  
permission java.util.PropertyPermission &quot;catalina.home&quot;, &quot;read&quot;&#59;  
permission java.util.PropertyPermission &quot;java.io.tmpdir&quot;, &quot;read&quot;&#59;&#125;&#59;
grant codeBase &quot;file&#58;$&#123;catalina.home&#125;/webapps/source/WEB&#45;INF/classes/&#45;&quot; &#123;  
permission java.util.PropertyPermission &quot;subversion.native.library&quot;, &quot;read&quot;&#59;  
permission java.lang.RuntimePermission &quot;loadLibrary.svnjavahl&#45;1?&#59;  
permission java.util.PropertyPermission &quot;disableLuceneLocks&quot;, &quot;read&quot;&#59;  
permission java.util.PropertyPermission &quot;catalina.home&quot;, &quot;read&quot;&#59;  
permission java.util.PropertyPermission &quot;java.io.tmpdir&quot;, &quot;read&quot;&#59;&#125;&#59;

Thanks to Vincent Liu & Flo.