Writing custom scripts

The previous section described how scripts can be used to automate analysis within QuPath, and apply it in a high-throughput manner across many images within a project.

Note on versions

This documentation is for QuPath v0.1.2 (the latest stable release at the time of writing).

It has not yet been updated for v0.2.0 (still in development).

In principle, scripts in this case aren't actually needed to get the final result - it would be possible to run all the commands for every image manually, and end up at the same place. The benefit of scripting is that it makes the process a lot less manual, a lot faster, and a lot more reproducible - no longer undermined by a lapse in concentration leading to an errant mouse-click.

However, this isn't the only purpose of scripting within QuPath. Scripts can also be used to access far more features, and to be able to customize the analysis in far greater ways to ensure that it answers the right questions. Often, a tiny script - just two or three lines long - can already be enormously useful in getting the results just right.

This section describes a bit more of the technical background needed to make sense of QuPath's scripts.

QuPath scripts are Groovy

The scripts produced by QuPath are created for a particular programming language: Groovy. This was chosen because Groovy has a huge range of features, while closely matching the Java programming language in which the majority of QuPath itself is written. Programmers familiar with Java (or similar) languages should have little trouble learning Groovy quickly, while someone new to programming can also pick it up due to its support for lots of shortcuts and user-friendly features.

Other scripting languages

It's also possible to scripting languages other than Groovy. To switch to Javascript, simply open the Script Editor and select Language → Javascript. The information here might be useful to understand how Javascript works with Java.

Alternatively, the Data exchange contains information on how to use QuPath with Python, MATLAB or the ImageJ macro language.

Simple scripts with the Script Editor

This section describes with examples how some simple scripts can be created directly using QuPath's Script Editor.

To follow along, ideally you should open an image and create several annotation and detect and classify some cells. The Detecting objects and Classifying objects sections give more details on how to do this.

Counting all objects

The first, manually-written script here simply counts up the number of objects - of any type - within an image (or, more exactly, inside the Object hierarchy associated with an image).

To run it, simply open the Script Editor (Automate → Show script editor), type in the following code, and choose Run → Run.

n = nObjects()
print("I have " + n + " objects in total")

This section does not try to explain all the principles of scripting, or programming in general - a bit of googling can find many tutorials for this. So here we focus on the specifics of scripting in QuPath, after the following three points of introduction:

• nObjects() and print() are methods (sometimes called functions). Characteristics of methods are that they:
  • do something
  • sometimes require extra information (provided between the parentheses), and
  • sometimes return a result.
• n is a variable. It represents the result of whatever nObjects() returns, i.e. the number of objects in the image. = is the assignment operator that makes this happen.
• Text is referred to as a string. Strings are created by putting text inside double quotes (or, optionally, in single quotes). It is possible to make a longer string by concatenating parts using the + operator.

Finding methods

It can be troublesome to know what methods are available, or how exactly they should be typed.

To help, QuPath's Script Editor has a very rudimentary auto-complete function. To use it, type the first letter (or first few letters) of a possible method name, and press Ctrl + space. QuPath will then fill in the rest of any method that it knows about. Pressing Ctrl + space repeatedly will cause QuPath to cycle through any alternatives that also start with the same letters.

Counting objects of different types

It might be desirable to be a bit more specific than simply counting all objects. If so, the following script can count up the number of detection and annotation objects separately:

detections = getDetectionObjects()
print("I have " + detections.size() + " detections")

annotations = getAnnotationObjects()
print("I have " + annotations.size() + " annotations")

Loops

Things get more interesting whenever we start to 'loop' through objects of particular types, and then to do things with them.

For example, this script loops through all the annotation objects and prints out a representation of the object.

To learn more about for loops in Groovy, see the documentation here.

for (annotation in getAnnotationObjects()) {
    print(annotation)
}

Warning! Printing all detection objects is generally not a good idea, since there may be many very detections - and printing many thousands of lines can be very slow.

Alternatively, a representation of some particular property of the annotation could be printed instead. The following script prints the ROI, rather than the object itself:

for (annotation in getAnnotationObjects()) {
    roi = annotation.getROI()
    print(roi)
}

And this script prints the classification:

for (annotation in getAnnotationObjects()) {
    pathClass = annotation.getPathClass()
    print(pathClass)
}

Counting classifications

One application building on the above scripts would be to count up the number of detections with a specific classification. To do so, we first need to get a reference to the classification, and then check if this matches with the classification of the detection.

The following script shows one way to do this, for 'Tumor' classifications.

tumorClass = getPathClass("Tumor")
nTumor = 0
for (detection in getDetectionObjects()) {
    pathClass = detection.getPathClass()
    if (pathClass == tumorClass)
      nTumor++
}
print("Number of tumor detections: " + nTumor)

However, depending upon what the outcome should be, there might be a subtle problem with this script. This is because it will only count up detections that have exactly the classification 'Tumor' - but not the derived classifications 'Tumor: Positive' or 'Tumor: Negative', for example.

See Object classifications for more detail on classifications and derived classifications.

If the script should count up all detections classified as 'Tumor' - including those that also have a more specific derived classification - then it could be modified as follows:

tumorClass = getPathClass("Tumor")
nTumor = 0
for (detection in getDetectionObjects()) {
    pathClass = detection.getPathClass()
    if (tumorClass.isAncestorOf(pathClass))
      nTumor++
}
print("Number of tumor detections: " + nTumor)

By using the isAncestorOf method built-in to each classification, a check is performed to see whether the classification of the object is either equal to or derived from the tumor classification. The method returns true if either of these is the case.

If derived classifications were not used, the two scripts give the same results. Therefore for most applications it is safest to use the more exact second script, and test for ancestry - since that is likely to be what is expected.

The way classifications are represented is very QuPath-specific, and it is not immediately obvious why isAncestorOf should work - or even that the method exists.

However, please don't let this put you off scripting. It is important to be aware that such gotchas exist, however the situation is not hopeless. The Advanced scripting with IntelliJ section shows you how you can set things up to be able to quickly check the QuPath source code to find out how things are implemented and what methods are available. It should also be evident when looking carefully at the output of a script when something has gone wrong.

Therefore, the tools exist to help find out this kind of thing. And there are also a variety of ways to get help and ask questions, as described in the Getting help section.

Calculating percentages

Using a small modification of the last script, it is possible to then also count up the non-tumor classifications and determine proportions or percentages, e.g.

tumorClass = getPathClass("Tumor")
nTumor = 0
nNonTumor = 0
for (detection in getDetectionObjects()) {
    pathClass = detection.getPathClass()
    if (tumorClass.isAncestorOf(pathClass))
      nTumor++
    else
      nNonTumor++
}
print("Number of tumor detections: " + nTumor)
print("Number of non-tumor detections: " + nNonTumor)
percentageTumor = nTumor / (nTumor + nNonTumor) * 100
print("Percentage of tumor detections: " + percentageTumor)

Adding areas

In some cases, it might be useful to add up the areas of the detections for each class. The following code does this:

tumorClass = getPathClass("Tumor")
nTumor = 0
nNonTumor = 0
areaTumor = 0
areaNonTumor = 0
for (detection in getDetectionObjects()) {
    roi = detection.getROI()
    pathClass = detection.getPathClass()
    if (tumorClass.isAncestorOf(pathClass)) {
      nTumor++
      areaTumor += roi.getArea()
    } else {
      nNonTumor++
      areaNonTumor += roi.getArea()
    }
}
print("Number of tumor detections: " + nTumor)
print("Number of non-tumor detections: " + nNonTumor)
percentageTumor = nTumor / (nTumor + nNonTumor) * 100
print("Percentage of tumor detections: " + percentageTumor)
percentageAreaTumor = areaTumor / (areaTumor + areaNonTumor) * 100
print("Percentage of tumor area: " + percentageAreaTumor)

Note that this script does not output the absolute values of the areas. This is because, by default, the area values will be given in pixels. Therefore percentages can still be meaningful, but absolute values may be misleading unless they are scaled by the pixel sizes.

Of course, it's possible to get the pixel sizes in a script as well... but that's for another tutorial, or a discussion forum question.

---

Technical notes

The following notes are not essential for anyone new to scripting, or just wanting to use it within QuPath. Rather, they are intended for anyone who wants to know more about how it is designed and works under the hood.

Default methods & imports

In the tutorial above, several methods were used that are QuPath-specific and not normally available in Groovy (or other scripting languages) - yet there was no need to declare or import them anywhere.

This makes writing scripts in QuPath somewhat like using a customized macro language, rather than simply using 'plain' Groovy. This purpose of this is to try to help users unfamiliar with scripting to be able to solve useful problems with a minimum of fuss.

However, a natural question would be: where do these default methods come from, and how does QuPath know they are there?

Static methods in QP & QPEx

To answer the first part of the question, the default methods are implemented in one of two Java classes:

• QP.java
• QPEx.java

Clicking on the links above should open the source code, allowing the methods to be seen in more detail.

QP.java contains methods that do not depend upon the user-interface, while QPEx.java contains all the same methods - plus a few more that do depend on the user-interface.

Does this distinction matter when writing a simple script? Probably not, but it is made to try to help make most QuPath scripts still work in scenarios where the user interface may not be available. If you don't care about this, just use QPEx.java. It extends from QP.java, which means that it includes everything that QP.java does.

The use of static methods has some troublesome limitations in terms of maintainability (particular with regard to subclassing), so the implementation underpinning this approach may change in the future... although any changes that are made will hopefully remain backwards-compatible.

Importing the static methods

To answer the second part of the question above, QuPath knows about the methods in QP and QPEx because they are imported automatically for every script that is run through the Script Editor. Furthermore, the variables returned by getCurrentImageData() and getCurrentHierarchy() are set appropriately, depending upon what image is active at the time.

However, it may be important to note that this is only true so long as Run → Include default bindings is selected (it is by default). Turning this setting off means that QuPath will not import the methods are set the variables. This tends to result in scripts needing to be longer and more complicated, with more import statements - but may be useful if the default imports are found to somehow conflict with a script that is being written.

If you want to import QP or QPEx manually, then you need to add one or both of these lines to the top of your script:

import qupath.lib.scripting.QP
import qupath.lib.scripting.QPEx

From QuPath v0.2.0-1, QPEx is under qupath.lib.gui.scripting.QPEx.