Skip to content
A suite of tools to assist with reviewing Open Source Software dependencies.
Branch: master
Clone or download
sschuberth BundlerTest: Update an expected homepage URL
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Latest commit 75e7d13 May 24, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Do not require organization members to signoff the DCO Nov 7, 2018
analyzer BundlerTest: Update an expected homepage URL May 24, 2019
buildSrc Apply miscellaneous minor formatting fixes Apr 24, 2019
cli Also capture the Java version ORT is running on May 15, 2019
docs Support Ivy version matchers in curation data Apr 12, 2019
downloader Downloader: Fix an error message to make DownloaderTest pass again May 22, 2019
evaluator evaluator: Visit dependencies only once per scope in dependency rules May 14, 2019
gradle/wrapper Gradle: Upgrade to version 5.4.1 Apr 26, 2019
logos README: Add our new ORT logos Jan 25, 2019
model model: Use annotations to add custom (de-)serializers May 22, 2019
reporter-web-app evaluator: Switch from `OrtIssue` to `RuleViolation` May 14, 2019
reporter ReportTableModelMapper: Collect the license findings only once May 20, 2019
scanner ScanCode: Move the error mapping funtions to the companion object May 22, 2019
spdx-utils spdx-utils: Fix parsing license references as exceptions May 22, 2019
test-utils Also capture the Java version ORT is running on May 15, 2019
utils Improve code readability by consistently using orEmpty() calls May 6, 2019
.appveyor.yml AppVeyor: Remove an empty statement Apr 24, 2019
.codecov.yml Codecov: Add a path mapping for the separate spdx-utils module Jan 28, 2019
.detekt.yml Detekt: Allow wildcard imports of SPDX enums Feb 20, 2019
.editorconfig Add a basic .editorconfig file Dec 19, 2017
.gitattributes spdx-utils: Add a Gradle task generating LicenseRef text resources Jan 10, 2019
.gitignore Gradle: Gather common Kotlin DSL extension functions in the buildSrc … Apr 17, 2019
.gitmodules scanner: Really bootstrap ScanCode instead of using it from a submodule Nov 1, 2018
.mailmap Update mail mappings Apr 4, 2019
.ort.yml Add an .ort.yml file for ORT Mar 18, 2019
.travis.yml Travis: Accept patch-level PHP version changes, again Apr 11, 2019
CONTRIBUTING.md Fix sentence issue CONTRIBUTING.md Mar 13, 2019
LICENSE Add an Apache 2.0 LICENSE file Nov 6, 2017
README.md README: Add a brief mention of the evaluator May 2, 2019
build.gradle.kts Gradle: Do not refer to a (fun)testRuntime configuration May 20, 2019
gradle.properties Gradle: Upgrade OkHttp to version 3.14.2 May 22, 2019
gradlew Gradle: Upgrade to version 5.3.1 Mar 28, 2019
gradlew.bat Gradle: Upgrade to version 5.3.1 Mar 28, 2019
settings.gradle.kts Gradle: Migrate the settings file to Kotlin script Apr 17, 2019

README.md

OSS Review Toolkit Logo

 

Linux (OpenJDK 10) Windows (Oracle JDK 9)
Linux build status Windows build status
Linux code coverage
Interact with us!
ort-talk

Introduction

The goal of the OSS Review Toolkit (ORT) is to verify Free and Open Source Software license compliance by checking project source code and dependencies.

At a high level, it works by analyzing the source code for dependencies, downloading the source code of the dependencies, scanning all source code for license information, and summarizing the results.

The different tools that make up ORT are designed as libraries (for programmatic use) with a minimal command line interface (for scripted use).

The toolkit is envisioned to consist of the following libraries:

  • Analyzer - determines dependencies of a software project even if multiple package managers are used. No changes to the software project are required.
  • Downloader - fetches the source code based on the Analyzer's output.
  • Scanner - wraps existing copyright / license scanners to detect findings in local source code directories.
  • Evaluator - evaluates results as OK or NOT OK against user-specified rules.
  • Advisor * - retrieves security advisories based on Analyzer results.
  • Reporter - presents results in various formats (incl. NOTICE files), making it easy to identify dependencies, licenses, copyrights and policy violations.
  • Documenter * - generates the final outcome of the review process, e.g. annotated SPDX files that can be included into your distribution.

* Libraries to be implemented, see our roadmap for details.

Installation

Follow these steps to run the OSS Review Toolkit from source code:

  1. Install the following basic prerequisites:

    • Git (any recent version will do).
    • OpenJDK 8 or Oracle JDK 8u161 or later (not the JRE as you need the javac compiler); also remember to the JAVA_HOME environment variable accordingly.
    • Node.js 8.* (for the Web App reporter).
    • Yarn 1.9.* - 1.12.* (for the Web App reporter).
  2. Clone this repository with submodules by running git clone --recurse-submodules. If you have already cloned non-recursively, you can initialize submodules afterwards by running git submodule update --init --recursive. Note that submodules are only required if you intend to run tests, though.

  3. Change into the created directory and run ./gradlew installDist to build / install the start script for ORT. On the first run, this will also bootstrap Gradle and download required dependencies. The start script can then be run as:

    • ./cli/build/install/ort/bin/ort --help

    Alternatively, ORT can be directly run by Gradle like:

    • ./gradlew cli:run --args="--help"

    Note that in this case the working directory used by ORT is that of the cli project, not directory gradlew is located in (see https://github.com/gradle/gradle/issues/6074).

  4. Make sure that the locale of your system is set to en_US.UTF-8, using other locales might lead to issues with parsing the output of external tools.

  5. Install any missing external command line tools as listed by

    • ./cli/build/install/ort/bin/ort requirements

    or

    • ./gradlew cli:run --args="requirements"

Alternatively, you can also run the OSS Review Toolkit by building its Docker image:

  1. Ensure you have Docker installed and its daemon running.

  2. Clone this repository with submodules by running git clone --recurse-submodules. If you have already cloned non-recursively, you can initialize submodules afterwards by running git submodule update --init --recursive. Note that submodules are only required if you intend to run tests, though.

  3. Change into the created directory and run ./gradlew cli:dockerBuildImage to build the Docker image and send it to the locally running daemon.

  4. Execute docker run ort requirements to verify all required command line tools are available in the container.

Tools

Analyzer

The Analyzer determines the dependencies of software projects inside the specified input directory (-i). It does so by querying whatever supported package manager is found. No modifications to your existing project source code, or especially to the build system, are necessary for that to work. The tree of transitive dependencies per project is written out as part of an OrtResult in YAML (or JSON, see -f) format to a file named analyzer-result.yml to the specified output directory (-o). The output file exactly documents the status quo of all package-related meta-data. It can be further processed or manually edited before passing it to one of the other tools.

 

Downloader

Taking an ORT result file with an analyzer result as the input (-a), the Downloader retrieves the source code of all contained packages to the specified output directory (-o). The Downloader takes care of things like normalizing URLs and using the appropriate VCS tool to checkout source code from version control.

 

Scanner

This tool wraps underlying license / copyright scanners with a common API so all supported scanners can be used in the same way to easily run them and compare their results. If passed an ORT result file with an analyzer result (-a), the Scanner will automatically download the sources of the dependencies via the Downloader and scan them afterwards. In order to not download or scan any previously scanned sources, the Scanner can be configured (-c) to use a remote storage hosted e.g. on Artifactory or S3 (not yet implemented, see #752). Using the example of configuring an Artifactory storage, the YAML-based configuration file would look like:

artifactory_storage:
  url: "https://artifactory.domain.com/artifactory"
  repository: "generic-repository-name"
  apiToken: $ARTIFACTORY_API_KEY

 

Evaluator

The evalutor is used to perform custom license policy checks on scan results. The rules to check against are implemented via scripting. Currently, Kotlin script with a dedicated DSL is used for that, but support for other scripting languages can be added as well. See no_gpl_declared.kts for a very simple example of a rule written in Kotlin script which verifies that no dependencies that declare the GPL are used.

 

Reporter

The reporter generates human-readable reports from the scan result file generated by the scanner (-s). It is designed to support multiple output formats. Currently the following report formats are supported:

  • Excel sheet (-f Excel)
  • NOTICE file (-f Notice)
  • Static HTML (-f StaticHtml)
  • Web App (-f WebApp)

Getting Started

Please see GettingStarted.md for an introduction to the individual tools.

Configuration

Please see Configuration.md for details about the ORT configuration.

Supported package managers

Currently, the following package managers / build systems can be detected and queried for their managed dependencies:

Supported license scanners

ORT comes with some example implementations for wrappers around license / copyright scanners:

Supported remote storages

For reusing already known scan results, ORT can currently use one of the following backends as a remote storage:

Development

The toolkit is written in Kotlin and uses Gradle as the build system. We recommend the IntelliJ IDEA Community Edition as the IDE which can directly import the Gradle build files.

The most important root project Gradle tasks are listed in the table below.

Task Purpose
assemble Build the JAR artifacts for all projects
detekt Run static code analysis on all projects
test Run unit tests for all projects
funTest Run functional tests for all projects
installDist Build all projects and install the start scripts for distribution

License

Copyright (C) 2017-2019 HERE Europe B.V.

See the LICENSE file in the root of this project for license details.

You can’t perform that action at this time.