Skip to content

Experimental: Extensions

jeremy.baker@northwestern.edu edited this page Aug 10, 2021 · 20 revisions

Extensions provide a way for NetLogo code to use functionality not provided by the core NetLogo primitives. Very often this involves interacting with external software and services, or performing specialized calculations that are difficult or tedious to implement in NetLogo. More information can be found in the NetLogo desktop documentation for extensions and the list of extensions for desktop. Very few extensions are supported in NetLogo Web at this time.

In NetLogo Web the target for an extension must be Javascript, and at the moment the only way to include an extension is by bundling it with the NetLogo Web code itself. This is all experimental, and we may change the definition file format or the way extensions are loaded as needed for future work. Longer-term the NetLogo team has plans to provide a mechanism to allow inclusion of third party extensions that are external to the NetLogo Web site, but that work is not yet complete.

Creating an extension requires providing at least two items to the Tortoise compiler and engine. First is a definition file that explains the syntax of the commands the extension provides. This must be read at NetLogo code compile time, since extensions involve statements in NetLogo code that will need to be parsed by the compiler. Second is the Javascript code that will actually run the commands during execution by the engine. See the definition and the code for the logging extension as examples.

Format for Defining Extensions

The following describes the grammar for defining an extension's .json defs file.

type ArgType = "unit" /* 'void' type; only appears in return type; indicates that this is a command, not a reporter */
  | "agentset"
  | "agent"
  | "booleanblock"
  | "boolean"
  | "bracketed"
  | "codeblock"
  | "commandblock"
  | "command"
  | "linkset"
  | "link"
  | "list"
  | "nobody"
  | "numberblock"
  | "number"
  | "optional"
  | "otherblock"
  | "patchset"
  | "patch"
  | "readable"
  | "reference"
  | "reporterblock"
  | "reporter"
  | "string"
  | "symbol"
  | "turtleset"
  | "turtle"
  | "wildcard"

// isRepeatable and isOptional default to false
// isOptional is for "commandblock" ArgType only
type fancyArgType = { type: ArgType, isRepeatable: boolean, isOptional: boolean }
type multiArgType = { types: Array[ArgType], isRepeatable: boolean }

type PrimDef = {
  name: String // name in NetLogo, used to access the Javascript implementation as a field of the `prims` object given by the extension
, argTypes: Array[ArgType | fancyArgType | multiArgType] // default: []
, returnType: ArgType // default: "unit", defines a reporter if not "unit"
, isInfix: Boolean // default: false, reporter only
, precedenceOffset: Number // default: 0, reporter only
, defaultArgCount: Number // default: 0
, minimumArgCount: Number // default: 0
, agentClassString: AgentClassString // string like "OTPL", "O---", "-TP-", and so on, default "OTPL"
, blockAgentClassString: Option[AgentClassString]
}

type ExtDef = {
  name: String
, prims: Array[PrimDef]
}

Import and Export of Extension State

If your extensions adds a porter field with an object that implements the ExtensionPorter interface, it will be called to handle exporting objects your extension made as well as for handling any global state. The intermediate types for exporting your extension objects and extension global state are the ExportedExtensionObject and the ExportedExtension.

For an example, see the SingleObjectExtensionPorter which handles extensions whose only state is a single type of extension object, such as Array, Matrix, and Table.

If your extension implements only global state (no extension objects), you can have canHandle() return false and ignore the object import/export functions entirely.

type ExtensionPorter[T] = {
  extensionName: String,

  canHandle: (Any) => Boolean,

  dump: (T, (Any) => String) => String,

  // turn an object into an ExportedExtensionObject for reloading later or serializing
  exportObject: (T, (Any) => Any) => ExportedExtensionObject,
  // turn your extension state into an `ExportedExtension` for reloading later or serializing
  export:       (Array[ExportedExtensionObject]) => ExportedExtension

  // turn an ExportedExtensionObject into a string for storing
  formatObject: (ExportedExtensionObject, (Any) => String) => String
  // turn your extension state into a string for storing
  format:       (ExportedExtension, (Any) => String) => String

  // read a string produced by `formatObject()` back into an ExportedExtensionObject for reloading
  readObject: (String, String, (String) => Any) => ExportedExtensionObject
  // read a string produce by `format()` back into an ExportedExtension for reloading
  read:       (String, (String) => Any) => ExportedExtension

  // reload an extension object's state
  importObject: (ExportedExtension, ExtensionPlaceholder, (Any) => Any)) => T
  // reload your extensions global state
  import:       (ExportedExtension, (Any) => Any) => Unit
}

Building, Testing, and Packaging

There is a Scala macro for the extension definition reader in the macros project. We have added an sbt task that should handle automatically re-compiling these macros each time there is a change in an extension's JSON file. This means when you run compilerJVM/compile the extensions macros will be recompiled automatically only if there is a change. This does not affect how the engine is compiled when you change the extension's code, only the JSON definitions that are provided to the compiler.

If for some reason you need to re-run the macros compilation (perhaps you believe it's not regenerating the extensions list for the compiler correctly), in an sbt console you can do this with the clean command for the JS and JVM versions of the project:

; macrosJS/clean ; macrosJVM/clean

You can test your extension with NetLogo language tests. It's often faster to develop your extension using language tests (if possible), then complete interactive tests after. Just add a text file with the tests in resources/main/test/commands then you can run in an sbt console (where Extension would match the name of the text file):

netLogoWeb/testOnly *TestCommands -- -z Extension

Once you're ready to publish your changes to test them out in a real environment, like Galapagos or your own project, you can locally publish the NetLogo JVM compiler (publishLocalVersioned compilerJVM) or the Javascript compiler and engine (publishLocalVersioned netLogoWeb) for use. More info on that in the Galapagos wiki.

Bundled Extensions

Some extensions are already bundled with NetLogo Web. These extensions are all experiemental and subject to change:

  • HTTP Req Extension - for sending HTTP requests.
  • CODAP Extension - for interacting with the CODAP service.
  • NLMAP Extension - a basic map or dictionary data type.
  • Experimental: Logging Extension - for logging model data to strings.
  • Experimental: CSV Extension* - for creating or splitting CSV strings.
  • Array* - A simple array data structure for models
  • Fetch* - for getting data into NetLogo Web from external sources, like files and URLs.
  • Dialog* - various dialogs to interact with the user of a model.
  • ExportThe* - for converting NetLogo data into strings.
  • ImportA* - for reading imported data back into NetLogo.
  • Matrix* - matrix data type and related mathematical operations - note some primitives are still unimplemented around Eigen values and vectors.
  • SendTo* - for sending data from NetLogo Web to a local file of the user's choosing.
  • Store* - for a persistent data store between model loads.
  • Table* - a key/value data structure for models

Info and links to docs for many of these extensions is in the NetLogo Web release notes.

A * indicates the extension has a compatible NetLogo desktop version.

Unimplemented Extensions

The following extensions come bundled with NetLogo desktop, but are not yet supported by NetLogo Web.

  • Arduino
  • Bitmap
  • GIS
  • Gogo
  • LevelSpace
  • Network
  • Palette
  • Profiler
  • Python
  • R
  • Rnd
  • Sound
  • Time
  • Vid
  • View2.5D