Skip to content

Commit

Permalink
Revert to 0.3.1 on master to avoid complaints until dev is released.
Browse files Browse the repository at this point in the history
  • Loading branch information
Fabian Yamaguchi committed Dec 26, 2016
1 parent d928ff1 commit 642e2c1
Show file tree
Hide file tree
Showing 1,610 changed files with 28,913 additions and 76,746 deletions.
15 changes: 15 additions & 0 deletions .classpath
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
<?xml version="1.0" encoding="UTF-8"?>
<classpath>
<classpathentry kind="src" path="src"/>
<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
<classpathentry kind="con" path="org.eclipse.jdt.junit.JUNIT_CONTAINER/4"/>
<classpathentry kind="lib" path="lib/commons-cli-1.2.jar"/>
<classpathentry kind="lib" path="lib/lucene-core-3.6.2.jar"/>
<classpathentry kind="lib" path="lib/geronimo-jta_1.1_spec-1.1.1.jar"/>
<classpathentry kind="lib" path="lib/antlr4-annotations-4.2.1-SNAPSHOT.jar"/>
<classpathentry kind="lib" path="lib/antlr4-runtime-4.2.1-SNAPSHOT.jar"/>
<classpathentry kind="lib" path="lib/neo4j-kernel-2.1.5.jar"/>
<classpathentry kind="lib" path="lib/neo4j-lucene-index-2.1.5.jar"/>
<classpathentry kind="lib" path="lib/neo4j-primitive-collections-2.1.5.jar"/>
<classpathentry kind="output" path="bin"/>
</classpath>
Binary file removed .gitignore
Binary file not shown.
17 changes: 17 additions & 0 deletions .project
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>joern</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
<buildCommand>
<name>org.eclipse.jdt.core.javabuilder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>org.eclipse.jdt.core.javanature</nature>
</natures>
</projectDescription>
4 changes: 2 additions & 2 deletions AUTHORS
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Fabian Yamaguchi <fabian.yamaguchi@cs.uni-goettingen.de>

Contributors:

Fabian Yamaguchi <fabian.yamaguchi@cs.uni-goettingen.de> (main developer)
Alwin Maier <alwin.maier@stud.uni-goettingen.de>
Malte Skoruppa <skoruppa@cs.uni-saarland.de> (php support)
823 changes: 666 additions & 157 deletions LICENSE

Large diffs are not rendered by default.

70 changes: 0 additions & 70 deletions build.gradle

This file was deleted.

3 changes: 0 additions & 3 deletions build.sh

This file was deleted.

86 changes: 86 additions & 0 deletions build.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
<?xml version="1.0"?>
<project name="Joern" default="jar" basedir=".">

<property name="src.dir" location="src" />
<property name="build.dir" location="bin" />
<property name="lib.dir" location="lib" />
<property name="test.dir" location="src" />
<property name="test.report.dir" location="testreport" />

<!-- Junit class path -->
<path id="junit.class.path">
<pathelement location="lib/junit.jar" />
<pathelement location="${build.dir}" />
</path>

<!-- Add jars to the classpath -->
<path id="joern.classpath">
<pathelement location="bin"/>
<path refid="junit.class.path"/>
<pathelement location="${lib.dir}/antlr4-runtime-4.2.1-SNAPSHOT.jar"/>
<pathelement location="${lib.dir}/antlr4-annotations-4.2.1-SNAPSHOT.jar"/>
<pathelement location="${lib.dir}/commons-cli-1.2.jar"/>
<pathelement location="${lib.dir}/lucene-core-3.6.2.jar"/>
<pathelement location="${lib.dir}/neo4j-kernel-2.1.5.jar"/>
<pathelement location="${lib.dir}/neo4j-lucene-index-2.1.5.jar"/>
<pathelement location="${lib.dir}/neo4j-primitive-collections-2.1.5.jar"/>
</path>

<target name="clean">
<delete dir="${build.dir}" />
<delete dir="${test.report.dir}" />
</target>

<target name="makedir">
<mkdir dir="${build.dir}" />
<mkdir dir="${test.report.dir}" />
</target>

<target name="compile">
<javac srcdir="${src.dir}" destdir="${build.dir}" includeantruntime="false">
<src path="src"/>
<classpath refid="joern.classpath"/>
</javac>
</target>

<target name="jar" depends="clean,makedir,compile">
<jar destfile="${build.dir}/joern.jar"
basedir="${build.dir}"
includes="**/*.class">
<manifest>
<attribute name="Main-Class" value="tools.index.IndexMain" />
</manifest>
<fileset dir="${build.dir}" includes="**/*.class" />
<zipgroupfileset dir="${lib.dir}" includes="**/*.jar" />
</jar>
</target>

<target name="tools" depends="icfg,argumentTainter">
</target>

<target name="icfg" depends="compile">
<jar destfile="${build.dir}/icfg.jar"
basedir="${build.dir}"
includes="**/*.class">
<manifest>
<attribute name="Main-Class" value="tools.icfg.ICFGMain" />
</manifest>
<fileset dir="${build.dir}" includes="**/*.class" />
<zipgroupfileset dir="${lib.dir}" includes="**/*.jar" />
</jar>
</target>

<target name="argumentTainter" depends="compile">
<jar destfile="${build.dir}/argumentTainter.jar"
basedir="${build.dir}"
includes="**/*.class">
<manifest>
<attribute name="Main-Class" value="tools.argumentTainter.ArgumentTainterMain" />
</manifest>
<fileset dir="${build.dir}" includes="**/*.class" />
<zipgroupfileset dir="${lib.dir}" includes="**/*.jar" />
</jar>
</target>


</project>
2 changes: 1 addition & 1 deletion docs/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXBUILD = sphinx-build2
PAPER =
BUILDDIR = build

Expand Down
92 changes: 92 additions & 0 deletions docs/source/access.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
Accessing Code with python-joern
=================================

Once code has been imported into a Neo4j database, it can be accessed
using a number of different interfaces and programming languages. One
of the simplest possibilities is to create a standalone Neo4J server
instance as described in the previous section and connect to this
server using ``python-joern``, the python interface to joern.

**General Note:** It is highly recommended to test your installation on a
small code base first. The same is true for early attempts of creating
search queries, as erroneous queries will often run for a very long
time on large code bases, making a trial-and-error approach
unfeasible.

Basic Usage
-----------

Python-joern currently provides a single class, JoernSteps, that
allows to connect to the database server and run queries. The
following is a simple sample script that employs JoernSteps to
configure the database connection, connect to the server and run a
Gremlin query.

::

from joern.all import JoernSteps

j = JoernSteps()

j.setGraphDbURL('http://localhost:7474/db/data/')

# j.addStepsDir('Use this to inject utility traversals')

j.connectToDatabase()

res = j.runGremlinQuery('getFunctionsByName("main")')
# res = j.runCypherQuery('...')

for r in res: print r


python-joern API
----------------

The sample script described in the previous section employs all
methods offered by JoernSteps. We now discuss each of these methods in
detail.

setGraphDbURL(url)
"""""""""""""""""""

**Sets the URL of the graph database server.** The REST API of the
Neo4J Database server is exposed on port 7474 by default. If your
server runs on a different port or server, you can use setGraphDbURL
to specify the alternate URL.

addStepsDir(dirname)
""""""""""""""""""""

**Add a source directory for utility traversals.** By default,
python-joern will inject all utility traversals contained in any of
the source files in joern/joernsteps into the database before running
scripts. Additional traversals specific to your application or
analysis are best placed in a separate directory. python-joern can be
instructed to honor this additional directory using addStepsDir.

connectToDatabase()
"""""""""""""""""""

**Connect to the database.** Call this method once the connection has
been configured to connect to the database server. A connection is
required before queries can be executed.

runGremlinQuery(query)
""""""""""""""""""""""

**Run the specified Gremlin query.** The supplied query is executed
and the result is returned. Depending on the query, the result may
have a different data type, however, it is typically an iterable
containing nodes that match the query.


runCypherQuery(query)
"""""""""""""""""""""

**Run the specified Cypher query.** The supplied query is executed
and the result is returned. Depending on the query, the result may
have a different data type, however, it is typically an iterable
containing nodes that match the query.


8 changes: 8 additions & 0 deletions docs/source/databaseOverview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -84,3 +84,11 @@ of global variables are saved in declaration statement nodes and
connected to the source file they are contained in using
``IS_FILE_OF`` edges.

The Node Index
---------------

In addition to the graphs stored in the Neo4J database, Joern makes an
Apache Lucene Index available that allows nodes to be quickly
retrieved based on their properties. This is particularly useful to
select start nodes for graph database traversals. For examples of node
index usage, refer to :doc:`querying`.
61 changes: 44 additions & 17 deletions docs/source/development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,28 +10,55 @@ clone the git repository by issuing the following command:

.. code-block:: none
git clone https://github.com/ocotopus-platform/joern.git
git clone https://github.com/fabsx00/joern.git
Optionally, change to the branch of interest. For example, to test the
development version, issue the following:

.. code-block:: none
git checkout dev
If you want to report issues or suggest new features, please do so via
https://github.com/octopus-platform/joern . For fixes, please fork the
repository and issue a pull request.
https://github.com/fabsx00/joern . For fixes, please fork the
repository and issue a pull request or alternatively send a diff to
the developers by mail.

Build system and IDEs
Modifying Grammar Definitions
------------------------------

Joern is a multi-module Gradle project where Java/Groovy sub-projects
are located in the directory 'projects', while python projects reside
in the directory 'python'.
When building Joern, pre-generated versions of the parsers will be
used by default. This is fine in most cases, however, if you want to
make changes to the grammar definition files, you will need to
regenerate parsers using the antlr4 tool. For this purpose, it is
highly recommended to use the optimized version of ANTLR4 to gain
maximum performance.

To build the optimized version of ANTLR4, do the following:

.. code-block:: none
git clone https://github.com/sharwell/antlr4/
cd antlr4
mvn -N install
mvn -DskipTests=true -Dgpg.skip=true -Psonatype-oss-release -Djava6.home=$PATH_TO_JRE install
If the last step gives you an error, try building without
``-Psonatype-oss-release``.

.. code-block:: none
mvn -DskipTests=true -Dgpg.skip=true -Djava6.home=$PATH_TO_JRE install
Next, copy the antlr4 tool and runtime to the following locations:

.. code-block:: none
To start hacking on Joern, make sure you can build it using the
supplied build script `build.sh`. For small modifications, it may be
sufficient to edit the source files using a simple text editor, and
subsequently invoking this script.
cp tool/target/antlr4-$VERSION-complete.jar $JOERN/
cp runtime/Java/target/antlr4-runtime-$VERSION-SNAPSHOT.jar $JOERN/lib
For larger changes, please consider using a JAVA IDE such as IntelliJ
or Eclipse. We use both of these IDEs for Joern-development on a
regular basis, so the import should hopefully be possible without
trouble using the corresponding Gradle plugin. IntelliJ typically
comes with a gradle plugin pre-installed, and Eclipse offers a plugin
in its "Marketplace".
where $JOERN is the directory containing the $JOERN installation.

Parsers can then be regenerated by executing the script
``$JOERN/genParsers.sh``.

0 comments on commit 642e2c1

Please sign in to comment.