Building

LaCuneta edited this page Jul 24, 2018 · 84 revisions

To hack on NetLogo, you'll need to know the following.

Quick start

git clone https://github.com/NetLogo/NetLogo.git
cd NetLogo
git submodule update --init
./sbt all

If this works, it will download stuff and build jars and other build products.

To run our automated test suites (takes about 20 minutes):

./nightly.sh

If it fails with a bunch of errors about missing extensions and/or models, you probably forgot to do git submodule update --init.

To run the GUI (on branches where there is one):

./sbt netlogo/run

Which branch?

The major branch is hexy.

For details on all our development branches, see Branches.

Prerequisites

Operating systems

You must have JDK 1.8 (JDK 9 doesn't work). We test and build releases using only Oracle JDK. The build is not known to have any problems running under OpenJDK, but we do not guarantee that the build runs on OpenJDK either.

Windows

While not currently supported for 5.x, we hope to add Windows sbt support in hexy. The 5.x build runs on Windows using Cygwin, but you will need to ensure that Cygwin installs the necessary dependencies.

Windows sbt support is available for NetLogo 6.0 and later. See our detailed Windows Setup instructions for configuring a Windows development environment.

Mac

On a Mac, Mac OS X 10.8 or 10.9 is required (probably; at least, no effort is going into testing or maintaining the build on older OS versions). You must install Xcode, a free download from Apple through the App Store. Once Xcode is installed, you must also run Xcode and install the optional Command Line Tools component.

Linux

On Linux, the build has only been tested at Northwestern using a Sun JDK, not OpenJDK. OpenJDK works well too.

Some extension builds may also require curl, which might not be installed by default (on Ubuntu, you can sudo apt-get install curl; on Mac OS X, the same if you use Fink, or brew install curl if you use Homebrew).

The following are needed to build hexy in Ubuntu:

  • git
  • java (and friends) installed with update-alternatives
  • curl
  • pandoc
  • nodejs (if building parserJS for tortoise)

Tools

You'll need a Java 8 JDK and a Git client.

You don't need to install Scala or SBT. The build will pull them down for you and use them, without needing to have them installed systemwide.

Versions

  • Java 8
  • Scala 2.12.4 (and will continue to upgrade within the 2.12 series)
    • If you want to double-check, look for the scalaVersion property in build.sbt.

About the build

Our main build tool is SBT (Scala Build Tool) version 1.x. SBT itself is written in Scala and is popular in the Scala community. SBT build files are themselves written in Scala (in build.sbt and project/*.scala).

You should run git submodule update --init and ./sbt all once to get started. Running ./nightly.sh (or ./nightly.sh --clean) runs ./sbt all and also runs our automated tests.

After having run ./sbt all once, often you can just work in SBT. ./sbt to start it up, then test, test-fast, package (builds JARs), run, etc.

hexy has several projects. See the wiki page on Hexy build architecture. In sbt you can run a particular command on a particular project by prefixing the command with the name of the project and a slash. So compiling the netlogo project (on the hexy branch) becomes ./sbt netlogo/compile.

Submodules

Some pieces of the full NetLogo package are kept in separate repositories, using git's submodules feature. The Models Library, as well as each bundled extension live in individual repos, to make it easier for extension developers to build and fork them individually. Each extension submodule should have a 5.x and a hexy branch, each of their respective HEADs tracked by the HEAD of the respective NetLogo branch.

Incremental build

SBT supports incremental building. Run ./sbt to enter the SBT shell, then issue commands like compile, which compiles the main tree. A table of some common tasks and the appropriate sbt command for each of them on each branch is below.

Task Description command
Compile netlogo/compile
Run netlogo/run
Run NetLogo Shell netlogo/runMain org.nlogo.headless.Shell
run quick tests netlogo/test:fast
run fast and medium-fast tests netlogo/test:medium
run slow tests only netlogo/test:slow
run a specific test netlogo/testOnly <test class> -- -z <test name>

(If you're coming from 5.x development, the commands are generally the same as in 5.x but are prefixed with netlogo/)

Tests

All of our automated tests use the ScalaTest test framework.

./nightly.sh runs the whole test suite in several different ways (2D and 3D, bytecode generator on and off). Note that running all of the tests takes a long time (20-30 minutes).

If you want to be more sure everything is built from scratch, you can do ./nightly.sh --clean instead.

Alternatively, pushing your commits will let travis build and test your code, instead of running the tests on your local machine.

Which tests do I run?

How can you be confident that your changes haven't broken anything? Our first line of defense is the compiler. A build that fails to compile is a broken build. Our last line of defense is Travis, which does some fairly exhaustive checking of the NetLogo internals to ensure that the build is good in corner cases that your tests may not cover. However, you can save a lot of time by testing the appropriate components before pushing to Travis.

Ideally, the right way to get confidence in your changes is by writing a test that fails before you write any code, then write and/or change the code so that the test passes. This ensures that the test is correct and any future changes which break your code will result in a failing test. In general, when cleaning up or making small changes to existing code, you should run locally test:fast and all appropriate tests for the particular unit of code you changed with testOnly.

The exceptions to this rule are changes which alter NetLogo primitives or the operation of the NetLogo engine. These have a bit more testing around them to ensure they work correctly. They are tested in three primary ways:

  1. They are tested by a suite of language tests (tc, tr, etc.)
  2. They are tested by model checksums, which ensure that each model produces the same output. These can be updated by running netlogo/dump all. Note that (unless the models library has been updated) changes to the benchmarks file usually mean that you have broken something. There are exceptions, but you should understand why the benchmarks are changing and have confidence about why they're being changed.
  3. They are tested to make sure the generated source for them is suitable for use in automatically generated primitives. This is done by dumping the JVM-generated source of a few benchmark files into test/benchdumps. These benchdumps can be updated with netlogo/dump bench. Only changes to primitives or the code generator should result in changes to the benchdump files, and these changes should be checked with caution to ensure correctness and performance.

IDEs

Even if you setup an IDE with your project, you still need to run ./sbt all in order to do the initial build. No IDE files are checked into the source tree. If you use Eclipse, you may be able to use the sbteclipse plugin to generate an eclipse build configuration. If you use IntelliJ, you may be able use the sbt project import tool in IntelliJ (sbt plugin) to create a Project. Please note that both Eclipse and IntelliJ are likely to miss sbt tasks used to build the project. If you're just looking to build the project without the command line, you might also check out Lightbend Activator which wraps sbt in a nice web UI. Note that you will still need to provide Activator with the correct java home depending on which branch of NetLogo you check out.

Version control

Pro Git (Apress, available online) is a good intro book. The O'Reilly book is more in-depth, but works less well as a practical tutorial.

Debugging

You should be able to debug using the IntelliJ debugger. It understands Scala, sort of.

When not using a debugger, the following three calls are your best friends:

  • println(...)
  • throwable.printStackTrace()
  • Thread.currentThread.dumpStack()

JVisualVM is also really useful. It comes with the Mac OS X Developer Tools; type jvisualvm to run it. It has a profiler and heap analyzer, can grab thread dumps, etc.

Building release bundles

See Releasing for details.

Source tree

See Build-Architecture-(hexy) for details. Each project contains src which in turns contains main and test. These folders contain the sources (Java & Scala combined) in packages (the org.nlogo. prefix is not reflected in the source tree). For example, FileDialog.java can be found in netlogo-gui/src/main/swing/FileDialog.java.

A few Java sources are automatically generated using JFlex, a lexer generator, or using custom Scala code. The input files for this are in project/build/autogen. The generated code is not kept in version control.

Version-controlled resources are in the resources directory. (Some internationalization-related resources are auto-generated and stored in target/resource_managed.)

Other directories

The following directories are accessed when you run the application:

  • netlogo-gui/docs -- User Manual in HTML. See #616
  • models -- Models Library
  • extensions -- submodules (see above) of the bundled extensions
  • netlogo-gui/natives -- native libraries for those libraries that use JNI

These files and directories are not accessed by the application:

  • build.sbt (file) and project (directory) -- SBT build
  • dist -- files associated with distributing NetLogo to end users

Plus there are some extra files and directories at the root:

  • bin, models/bin -- scripts used during development. most of them are written in Scala and run inside a shell script wrapper.
  • target -- SBT puts build products here
  • tmp -- some of our tests put temporary stuff here; bin/release.sh uses it too

More SBT tasks

  • tc: runs org.nlogo.headless.TestCommands (to run one file do e.g. tc Generator)
  • tr: runs org.nlogo.headless.TestReporters (to run one file do e.g. tr Lists)
  • te: runs org.nlogo.headless.TestExtensions (to run just one file do e.g. te array)
  • tm: runs org.nlogo.headless.TestModels (to run just one file do e.g. tm Fire)
  • testOnly: runs a specific test given by a class name (e.g. testOnly org.nlogo.generator.TestHelperMethodSafe)
  • depend: looks for dependencies between packages that aren't supposed to depend on each other; see Architecture for a list of what dependencies are and aren't allowed. included in nightly.sh
  • pmd: run PMD to check Java source files for flawed/suspicious code
  • bench: runs the engine benchmarks
  • dump

this list isn't necessarily complete or up-to-date

Extra scripts

There are also some scripts in the bin directory that do useful things.

  • bin/shell.sh: does run-main org.nlogo.headless.Shell inside rlwrap, so you get a headless "command center" with GNU Readline niceness
  • bin/benches.scala: repeatedly runs our entire benchmark suite. suitable for overnight runs. See Benchmarking
  • bin/profile.scala: runs benchmark models under the HProf profiler, dumping results to tmp/profiles, suitable for viewing with PerfAnal

this list isn't necessarily complete or up-to-date

Miscellaneous

How to fix bintray warnings

If you don't have bintray credentials setup, you will see errors like: "Missing bintray credentials /Users/yourname/.bintray/.credentials. Some bintray features depend on this." You can fix these errors by creating the file suggested and adding the following text to it:

realm = Bintray API Realm
host = api.bintray.com
user =
password =

Git warnings on windows

If building on Windows, you may see an error similar to the following:

[error] fatal: 'submodule' appears to be a git command, but we were not
[error] able to execute it. Maybe git-submodule is broken?

As part of building extensions, NetLogo tries to checkout all git submodules to make sure you have the latest of each extension before building. On Windows, this sometimes fails with the error above. When the Windows git installer (not cygwin git) is used and configured to have git work in the Windows command prompt, this error should not appear.

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.