Build Architecture (hexy)

Description of builds

This diagram presents a wide-angle view of the NetLogo project. Each circle represents an sbt subproject. The circles with dashed borders indicate projects which aren't build on their own but are rather shared sources for each of their dependencies. Following the image, you will find a description of each build.

Dist (dist in sbt)

The dist project packages NetLogo for release. It depends on NetLogo and Mac App. Note that certain tasks in dist may not work on all platforms. For instance, the Windows packaging task won't work on mac or vice-versa. You should only need to interact with this project when preparing to release NetLogo or changing something about the release.

NetLogo GUI (netlogo in sbt)

The NetLogo project is the desktop NetLogo application. It depends on the JVM Parser and NetLogo Core projects. In general, if you want to run or test the latest version of NetLogo, this is the build you want to use. This project includes the code for NetLogo, as well as HubNet and 3D.

NetLogo Headless (headless in sbt)

The NetLogo Headless project is a stripped-down version of NetLogo. It has a similar engine to NetLogo GUI and they both share an api exposed by the NetLogo Core project. NetLogo Headless doesn't include 3D, nor does it include hubnet. While not currently part of the desktop package, it is used to verify that the results of NetLogo Web match those of NetLogo Desktop.

NetLogo Core (netlogoCore in sbt)

The NetLogo Core project is a shared-source project which includes files shared between NetLogo Desktop and NetLogo Headless. Files added to this project are compiled alongside each project and included with the jar produced by each of them. More packages are likely to be moved into this package as similarities are identified between NetLogo GUI and NetLogo Headless.

Parser JVM (parserJVM in sbt)

The Parser JVM project builds the NetLogo parser for use in the JVM environment. It shares the source found in Parser Core with the Parser JS project. As such, the sources in the parser-jvm directory are only used in the JVM version of the parser. The parser itself consists of three packages: core, lex, and parse. The core package contains classes used in lex and parse as well as throughout the rest of NetLogo. lex and parse are the NetLogo lexer and parser, respectively.

Parser JS (parserJS in sbt)

The Parser JS project builds the NetLogo parser for use in scala.js. At the moment, this project is used by the Tortoise project to generate scala.js for use in the Galapagos web app. It shares source from parser-core with Parser JVM. The code in the parser-js directory is only used in the JS version of the parser.

Parser Core (parserCore in sbt)

The Parser Core shared-source project is where the majority of code for the NetLogo parser lives. Note that since this code is compiled for both the JVM and JS environments it should respect the semantic differences of scala.js. The biggest limitation is that code using reflection must live in either Parser JVM or Parser JS because scala.js does not yet support reflection.

Macros (macros in sbt)

The Macros project is used by the parser JS project to provide reflection-like capabilities. It's only a few files which read the class data available at compile-time and use it to generate code that doesn't require runtime reflection. Note that only the js parser should depend on this project.

Shared Resources (sharedResources in sbt)

This contains resources needed by the various parser projects. Note that the Macros project depends on this one for the data needed for the I18n methods.

Mac App (macApp in sbt)

This project contains mac-specific code. It interacts with the NetLogo project to enable mac-specific behavior. It is separate from the NetLogo build because it compiles against the Oracle apple SDK's (com.apple.eawt), which are available only on Mac builds of the jdk. In the future, this would ideally be moved to a completely separate repository and be depended upon either as a jar or a submodule.

Why is the build like this?

This is almost certainly the most difficult information on this page to derive from the code that makes the build run. This section will first discuss the history of the NetLogo build and then discuss why the build's current form serves a valuable function for the NetLogo team.

Historically, NetLogo lived quite happily in one repository. There were various submodules (mostly for extensions), but it was essentially a monolithic desktop application. Sometime around 2012 the CCL began to develop NetLogo for the web. This led to a fork of NetLogo known as "NetLogo Headless" which had the goal of simplifying and improving the NetLogo code base. The idea behind NetLogo Headless was to keep NetLogo powerful enough to compile for the web and to verify a javascript implementation while removing any NetLogo components that were not needed. As this effort developed, the headless project was further split into a NetLogo parser and a barebones implementation of NetLogo used primarily (if not exclusively) to test the output of the NetLogo to JS compiler. With the introduction of scala.js in 2015, the parser underwent some fairly radical structural changes which would allow it to work in the browser as opposed to only on the server side. Once NetLogo Web was released the CCL development team wanted to reduce the number of different projects across which work needed to be ported (currently work was split between netlogo and netlogo-headless) as well as to get many of the enhancements and simplifications that headless had introduced to the NetLogo parser. The decision was made to combine the headless codebase with the rest of NetLogo in a single repository. At the moment, these two projects share very little code (just the api and shape packages) but the hope is that over time a great deal of code will be shared between these implementations. Additionally, the CCL made an effort at the end of 2015 to package NetLogo for Java 8, which led to an overhaul of the build process and produced both the Dist and Mac App projects.

In the current build, each project serves a distinct purpose. This is easy to see with the Dist project, which manages packaging NetLogo releases across multiple platforms. The Mac App project also serves to bridge the gap between the mac-native java experience and the rest of the NetLogo codebase, which is general across OS platforms. The Headless project exists as a way to test NetLogo Web while NetLogo GUI negotiates the components necessary to make NetLogo run in a java GUI environment. The parser projects are similarly constrained by the various requirements of NetLogo Web and scala.js.

None of this is to say that the component structure of NetLogo couldn't be improved. Ideally, NetLogo could be built as independently deployable jars for different functions, all of which are packaged together at runtime for easy use. While that goal is still a long way off, it's facilitated by our project dependency tests as well as good hygeine in the sbt structure of the build.

How Do I...

Following are some common tasks one might have related to the build with some solutions. This list is woefully incomplete and please feel free to add questions if you need one answered.

...improve the codebase and reduce duplication?

First, locate a package that is similar between netlogo-headless and netlogo-gui. The goal is to move the package into netlogo-core, while ensuring that the build doesn't break. You may need to modify the api package in order to abstract over differences or to modify various files to get them working in both builds. Verify that all tests pass in headless and netlogo-gui (GitHub Actions will do this for you when you push up, but it can be helpful to do it locally).

...report a bug?

If you find a bug in the build process, please open a GitHub issue and tag it with the "devel" tag. If it's an OS-Specific bug, please attach the "mac" or "windows" tags accordingly.

...find the source file I'm looking for?

This diagram should offer a good start:

...change something about the build?

First, consider the size of the change. Updating a dependency or adding a resource generator can be done with relative ease and little oversight, while adding a new project should prompt some serious discussion and testing. Before changing anything major in the build, ask around in the Gitter room to determine whether the change is necessary and, if so, what testing needs to be done after it's made.

File-by-file Project Breakdown

Files in project directory:

• AggregateLinuxBuild.scala: Builds linux application for distribution.
• AggregateMacBuild.scala: Builds Mac application for distribution.
• AggregateWindowsBuild.scala: Builds Windows application for distribution.
• Build.scala: Defines various version and build-date related settings
• BuildJDK.scala: Utility class for packaging system.
• BundledDirectory.scala: Classes for packaging system to abstract over the various subdirectories included in the build. If you want to customize how models, natives, docs, or Behaviorsearch are packaged, look here first.
• ChecksumsAndPreviews.scala: tasks for computing checksums and previews.
• CommonConfiguration.scala: Utility class for packaging system which specifies inputs given to javapackager.
• Depend.scala: Defines the build's depend task. If you have questions about which dependencies are legal (or ever want to change that) look here.
• Docs.scala: Provides high-level documentation tasks used in packaging NetLogo.
• Dump.scala: Provides tools for doing benchdumps (dumping the bytecode generated from a given NetLogo model).
• EventsGenerator.scala: Generates Swing events from the events.txt file.
• ExtensionDocs.scala: Generates documentation pages for the NetLogo extensions.
• Extensions.scala: Builds the NetLogo extensions
• FileActions.scala: Abstracts over common file operations. This was created initially because we were sometimes using sbt.io and sometimes using java.nio and it made sense to abstract over common tasks in a consistent location.
• GUISettings.scala: Sets properties to make NetLogo behave like a mac-app when being run on a Mac.
• HarvestResources.scala: Replaces the WiX Harvest Tool with a scala-native solution. This allows us to generate consistently-named components (I believe it also offers a slight speed improvement).
• I18n.scala: Translates the UTF-8 properties files into ISO-8859-1 encoding used by Java. When the project is upgraded to Java 9 (which reads properties files as UTF-8 by default), the files should be moved to the correct location and this should be removed.
• InfoTab.scala: Builds the info tab section of the user manual from the info tab model.
• InfoTabGenerator.scala: Fetches the info section out of a given model (only used on InfoTab model at the moment).
• JFlexRunner.scala: Runs jflex to generate the ImportLexer. Note that this lexer differs slightly from the lexer used by the NetLogo parser and has to be able to handle certain items which the parser-lexer isn't able to handle.
• JavaPackager.scala: Abstraction over the JavaPackager tool. Used to build platform-specific binaries and bundle the JRE with them.
• Markdown.scala: Renders the markdown pages of the NetLogo user manual.
• ModelsLibrary.scala: Contains various models-library grooming tasks, such as index generation (rendering info tab markdown into a HOCON file for display in the models library dialog), and resaving models (done before each release to bring them up-to-date).
• Mustache.scala: Renders mustache used in the NetLogo user manual.
• NativeLibs.scala: Fetches the native libraries required by JOGL.
• NetLogoDocs.scala: Actually does most of the work of documentation
• NetLogoDocsTest.scala: Tests for broken links in the NetLogo docs
• NetLogoPackaging.scala: Contains a number of tasks used to build NetLogo release packages.
• NetLogoWebExport.scala: Updates the template file used to export NetLogo web based on the latest available online release.
• Packaging.scala: Customizes the class-path set in generated jars
• PlatformBuild.scala: Utility classes used to abstract over the differences between packaging for different platforms
• PrimIndex.scala: Builds per-primitive "quick-help" files.
• Running.scala: Customizes the options used to start NetLogo
• SbtSubdirectory.scala: Code for running sbt tasks in subdirectories without listing them as subprojects. Why not list them as subprojects? That involves upgrading in lockstep, which is often very difficult.
• Scaladoc.scala: Generates scaladoc for NetLogo.
• SubApplication.scala: Utility classes representing the various sub-applications (Logging, 3D) included in the NetLogo package.
• Testing.scala: Defines the build-specific testing tasks (test:fast, test:medium, test:slow)
• build.properties: Defines the version of sbt.
• build.sbt: Turns on deprecation warnings for our sbt code
• documentation.conf: Used to format extension primitives
• plugins.sbt: Configures build plugins