A java api and command line tool for scanning, reporting and fixing a git repository's InnerSource Readiness based on a supplied specification which defines the files and file contents necessary for a repository to be considered ready for InnerSource contribution.
Table of Contents
- Determine if a local git repository is InnerSource ready
- Determine if a remote GitHub hosted git repository is InnerSource ready
- Determine if a remote Enterprise GitHub hosted git repository is InnerSource ready
- Customize using declarative json what documentation is required for a repository to be considered InnerSourceReady
- Automatically fixup your local repositories to make them InnerSource ready
- Automatically fixup your remote GitHub hosted repositories to make them InnerSource ready
- Automatically fixup your remote Enterprise GitHub hosted repositories to make them InnerSource ready
- Your medium to large sized organization has decided to adopt InnerSource development practices.
- In order for contributors to feel confident contributing to a repository, the repository should demonstrate a minimum level of InnerSource readiness; such as having a CONTRIBUTING.md describing the expectations of the maintainers, or having a README.md with Usage and local development instructions.
- You and your organization have defined what it means for a repository to be InnerSource ready in terms of the expected repository documentation.
- Your organization would like a report on the InnerSource Readiness of its most depended upon repositories.
- You would like to automatically fixup repositories that are not InnerSource ready, so that maintainers know which documents are required and what the most important sections in the documents are for them to fill out to help external contributors.
The two main operations supported by this library are scanning a repository for InnerSource readiness and fixing up the InnerSource readiness of a repository. First decide if you would like to use the standalone command line interface (CLI) or the Java API. The primary difference between the two interfaces is that you can use multithreading in the Java API to concurrently scan repositories, whereas the CLI requires multiple processes.
- Download the CLI Runnable JAR
- Check if your local or remote git repository is InnerSource Ready using the CLI
- Fixup your local or remote git repository so that it is as close to InnerSource Ready as possible using the CLI
- Download the latest release version of the runnable jar CLI from either GitHub Packages or Maven Central.
- Rename the downloaded runnable jar to
innersource.jar
for convenience. The examples below will assume this filename.
NOTE: Java 1.8+ must be installed in order to execute the runnable jar and follow the examples below.
If your repository is checked out locally:
java -jar innersource.jar -c REPORT -r /path/to/checked/out/repo
If your repository is hosted on GitHub (public or enterprise):
java -jar innersource.jar -c REPORT -r https://github.yourorg.com/repo-org-name/repo-name -a yourGithubAccessToken
After running the report command, you should see a json report output similar to the following:
{
"isRepositoryInnerSourceReady": false,
"specificationEvaluated": {...}, // the input specification that was used to generate this report
"fileRequirementReports": [
{
"fileRequirementEvaluated": {...}, // which file requirement defined in the spec does this report relate to
"optionEvaluated": { // if the file requirement can be satisfied multiple ways, which option is this report evaluating
"fileToFind": {
"expectedFilePath": "/README.md"
},
"fileChecks": {...}
},
"filesEvaluated": [ // what files were found in the repo search directories and evaluated for the defined file checks
"/README.md"
],
"filesSatisfyingFileChecks": [], // which files met all of the defined file checks
"fileChecksReports": [ // breakdown of which file check passed or failed for each scanned file
{
"fileEvaluated": "/README.md",
"fileChecksEvaluated": {...}, // what file checks were defined in the spec that we evaluated as part of this report
"fileCheckReports": [
{
"fileCheckEvaluated": {
"requirement": "FILE_EXISTS"
},
"isFileCheckSatisfied": true
},
{
"fileCheckEvaluated": {
"requirement": "FILE_NOT_EMPTY"
},
"isFileCheckSatisfied": true
},
{
"fileCheckEvaluated": {
"requirement": "MARKDOWN_FILE_HAS_HEADING",
"heading": "USAGE",
"synonyms": [],
"matchCase": false,
"matchIfSectionEmpty": true
},
"isFileCheckSatisfied": false // true if /README.md had a usage heading, false otherwise
},
{
"fileCheckEvaluated": {
"requirement": "MARKDOWN_FILE_HAS_HEADING",
"heading": "LOCAL DEVELOPMENT",
"synonyms": [
"LOCAL DEV"
],
"matchCase": false,
"matchIfSectionEmpty": true
},
"isFileCheckSatisfied": false
}
],
"isFileChecksSatisfied": false
}
],
"isFileRequirementSatisfied": false
},
{
"fileRequirementEvaluated": {...},
"optionEvaluated": { // since Pull Request Template can either be a file or a directory, this report assessed whether we found the file variant
"fileToFind": {
"expectedFilePath": "/.github/PULL_REQUEST_TEMPLATE.md"
},
"fileChecks": {
"checks": [
{
"requirement": "FILE_EXISTS"
},
{
"requirement": "FILE_NOT_EMPTY"
}
]
}
},
"filesEvaluated": [
"/.github/PULL_REQUEST_TEMPLATE.md"
],
"filesSatisfyingFileChecks": [
"/.github/PULL_REQUEST_TEMPLATE.md"
],
"fileChecksReports": [
{
"fileEvaluated": "/.github/PULL_REQUEST_TEMPLATE.md",
"fileChecksEvaluated": {
"checks": [
{
"requirement": "FILE_EXISTS"
},
{
"requirement": "FILE_NOT_EMPTY"
}
]
},
"fileCheckReports": [
{
"fileCheckEvaluated": {
"requirement": "FILE_EXISTS"
},
"isFileCheckSatisfied": true
},
{
"fileCheckEvaluated": {
"requirement": "FILE_NOT_EMPTY"
},
"isFileCheckSatisfied": true
}
],
"isFileChecksSatisfied": true
}
],
"isFileRequirementSatisfied": true
},
{
"fileRequirementEvaluated": {...},
"optionEvaluated": {
"fileToFind": {
"expectedFilePath": "/.github/PULL_REQUEST_TEMPLATE/"
},
"fileChecks": {
"checks": [
{
"requirement": "DIRECTORY_EXISTS"
},
{
"requirement": "DIRECTORY_NOT_EMPTY"
}
]
}
},
"filesEvaluated": [
"/.github/PULL_REQUEST_TEMPLATE.md"
],
"filesSatisfyingFileChecks": [],
"fileChecksReports": [
{
"fileEvaluated": "/.github/PULL_REQUEST_TEMPLATE.md",
"fileChecksEvaluated": {...},
"fileCheckReports": [
{
"fileCheckEvaluated": {
"requirement": "DIRECTORY_EXISTS"
},
"isFileCheckSatisfied": false
},
{
"fileCheckEvaluated": {
"requirement": "DIRECTORY_NOT_EMPTY"
},
"isFileCheckSatisfied": false
}
],
"isFileChecksSatisfied": false
}
],
"isFileRequirementSatisfied": false
}
]
}
The top level property isRepositoryInnerSourceReady
will be true if the scanned repository is
considered InnerSource Ready, and false otherwise.
It's handy to pipe the output of the report command to jq to extract just the information you care about.
Checking if your repository is innersource ready thus becomes:
java -jar innersource.jar -c=REPORT -r=https://github.com/your/repo -a=your-gh-auth-token | jq .isRepositoryInnerSourceReady
Which should return the boolean true
or false
.
There are quite a few additional details included in the report, so you can zoom in on exactly what was missing from the repository.
For example, if your repository is not InnerSource ready you can see which files were evaluated and which file checks were not satisfied with the following command:
java -jar innersource.jar -c=REPORT -r=https://github.com/your/repo -a=your-gh-auth-token | jq '[.fileRequirementReports | .[] | select(.isFileRequirementSatisfied == false) | {optionEvaluated, fileChecksReports: .fileChecksReports | .[] | del(.fileChecksEvaluated) | del(.isFileChecksSatisfied) | del(.fileCheckReports[] | select(.isFileCheckSatisfied == true))}]'
The Default InnerSource Readiness Specification
states that a repository is considered InnerSource Ready if the following files are present,
and not empty. (NOTE: The files do not actually have to appear in the exact locations as shown;
so long as the file is located in either the repo root /
, the GitHub metadata directory /.github
or the docs directory /docs
, the file requirement will be satisfied)
Some files have additional requirements such as the README requiring a Title
heading, or the issue template expecting yaml front matter. The exact requirements for each
file is explained below and in the json spec linked above.
/repo
|-- /.github
| |-- /ISSUE_TEMPLATE
| | `-- someIssueTemplate.md
| |-- /PULL_REQUEST_TEMPLATE
| | `-- somePullRequestTemplate.md
| `-- CODEOWNERS
|-- CONTRIBUTING.md
|-- CODE_OF_CONDUCT.md
|-- LICENSE
|-- SUPPORT.md
`-- README.md
- A file with the base filename
README
(case-insensitive) exists in either the repository root directory/
, the docs directory/docs
or the GitHub metadata directory/.github
. - The
README
file contains a title heading#
that is not namedtitle
. - The
README
's title heading is followed by a paragraph text description.
- A file with the base filename
CONTRIBUTING
(case-insensitive) exists in either the repository root directory/
, the docs directory/docs
or the GitHub metadata directory/.github
. - The
CONTRIBUTING
file is not empty.
- A file with the base filename
CODE_OF_CONDUCT
(case-insensitive) exists in either the repository root directory/
, the docs directory/docs
or the GitHub metadata directory/.github
. - The
CODE_OF_CONDUCT
file is not empty.
- A file with the base filename
LICENSE
(case-insensitive) exists in either the repository root directory/
, the docs directory/docs
or the GitHub metadata directory/.github
. - The
LICENSE
file is not empty.
- A file with the base filename
SUPPORT
(case-insensitive) exists in either the repository root directory/
, the docs directory/docs
or the GitHub metadata directory/.github
. - The
SUPPORT
file is not empty.
- A file with the base filename
CODEOWNERS
(case-insensitive) exists in either the repository root directory/
, the docs directory/docs
or the GitHub metadata directory/.github
. - The
CODEOWNERS
file is not empty. - The
CODEOWNERS
file contains a default match all rule, which is an uncommented line starting with an*
followed by a GitHub username or email address.
- A directory with the filename
ISSUE_TEMPLATE
(case-insensitive) exists in either the repository root directory/
, the docs directory/docs
or the GitHub metadata directory/.github
. - The
ISSUE_TEMPLATE
directory contains at least one file which has a Yaml Front Matter section containing the propertiesname
andabout
- A file with the base filename
PULL_REQUEST_TEMPLATE
(case-insensitive) exists in either the repository root directory/
, the docs directory/docs
or the GitHub metadata directory/.github
OR a directory with the filenamePULL_REQUEST_TEMPLATE
(case-insensitive) exists in either the repository root directory/
, the docs directory/docs
or the GitHub metadata directory/.github
- The
PULL_REQUEST_TEMPLATE
file or directory is not empty.
The default InnerSource Readiness specification can be customized or overridden. See the Customize InnerSource Readiness Specification Configuration.
If your repository is checked out locally:
java -jar innersource.jar -c FIXUP -r /path/to/checked/out/repo
If your repository is hosted on GitHub (public or enterprise):
java -jar innersource.jar -c FIXUP -r https://github.yourorg.com/repo-org-name/repo-name -a yourGithubAccessToken
The Default Fixup Templates
for the FIXUP
command will create the following files with predefined
content if the file does not exist or is empty:
/README.md
/CONTRIBUTING.md
/CODE_OF_CONDUCT.md
/SUPPORT.md
/.github/CODEOWNERS
The default Fixup File Templates can be customized or overridden. See the Customize Fixup File Templates Configuration.
- Install Dependencies
- Check if your local or remote git repository is InnerSource Ready using the Java API
- Fixup your local or remote git repository so that it is as close to InnerSource Ready as possible using the Java API
If your repository is checked out locally:
InnerSourceReadinessReport report =
InnerSourceReadinessReportCommand.create(
LocalRepositoryFilePath.of(Paths.get("/your/local/git/repo/root"))
)
.build()
.call();
If your repository is hosted on GitHub (public or enterprise):
Add the following optional dependency to your classpath:
<!-- https://mvnrepository.com/artifact/org.kohsuke/github-api -->
<dependency>
<groupId>org.kohsuke</groupId>
<artifactId>github-api</artifactId>
<version>1.122</version>
</dependency>
Create an instance of GitHubRepositoryPath
by configuring a GHRepository
instance using your GitHub server and credentials:
GitHub gh = new GitHubBuilder()
.withOAuthToken("yourGithubAccessToken")
.withEndpoint("https://github.yourorg.com/api/v3")
.build();
GHRepository repository = gh.getRepository("repo-org-name/repo-name");
InnerSourceReadinessReport report =
InnerSourceReadinessReportCommand.create(
GitHubRepositoryPath.of(repository)
)
.build()
.call();
After running the report command you can inspect the InnerSourceReadinessReport
object to find out
if your repository is InnerSource Ready or determine which required files were or were not present.
boolean isReadyAccordingToSpec = report.isRepositoryInnerSourceReady();
By default, the InnerSourceReadinessReportCommand
uses the InnerSourceReadinessSpecification.PUBLIC_GITHUB_DEFAULT
specification which considers repositories InnerSource Ready only if they have a
certain set of files, and the files satisfy a set of predefined file checks.
The default specification can be customized or overridden. See the Customize InnerSource Readiness Specification Configuration.
Did the scanned repository contain a README file containing all the expected headers?
FileRequirement readmeFileRequirementPassedToSpec = ...;
boolean foundReadme = report.isFileRequirementSatisfied(
readmeFileRequirementPassedToSpec
);
Did any of the scanned README files contain a USAGE heading?
FileCheck usageHeadingFileCheckPassedToSpec = ...;
boolean foundReadmeWithUsageHeading = report.getOnlyFileRequirementReportFor(
readmeFileRequirementPassedToSpec
)
.get() // unwrap the optional
.getFileChecksReports() // if multiple README files were found, each will be assessed for the specified file checks
.stream()
.anyMatch(checksReport ->
checksReport.isFileCheckSatisfied(usageHeadingFileCheckPassedToSpec)
);
See the javadoc for additional details that you can obtain from the report object.
List<RepositoryFilePath> fixedFiles =
InnerSourceReadinessFixupCommand.create(
LocalRepositoryFilePath.of(Paths.get("/your/local/git/repo/root"))
)
.build()
.call();
By default, the InnerSourceReadinessFixupCommand
uses the FixupFileTemplates.PUBLIC_GITHUB_DEFAULT
set of templates which will create a set of default files with predefined content.
The Fixup File Templates can be customized or additional templates defined. See the Customize Fixup File Templates Configuration.
Any files that are created or modified are returned as a result of running this command.
Create a new json file. Filename isn't important but for our example
we will use innersource.spec.json
.
{
"specName": "NameOfYourSpecification",
"repositoryRequirements": {
"directoriesToSearch": {
"directoryPaths": [
"/",
"/docs",
"/.github"
]
},
"requiredFiles": [
{
"requiredFileOptions": [
{
"fileToFind": {
"expectedFilePath": "/README.md"
},
"fileChecks": {
"checks": [
{
"requirement": "FILE_EXISTS"
},
{
"requirement": "FILE_NOT_EMPTY"
},
{
"requirement": "MARKDOWN_FILE_HAS_HEADING",
"heading": "Usage",
"synonyms": [],
"matchCase": false,
"matchIfSectionEmpty": true
},
{
"requirement": "MARKDOWN_FILE_HAS_HEADING",
"heading": "Local Development",
"synonyms": [
"Local Dev"
],
"matchCase": false,
"matchIfSectionEmpty": true
}
]
}
}
]
},
{
"requiredFileOptions": [
{
"fileToFind": {
"expectedFilePath": "/.github/PULL_REQUEST_TEMPLATE.md"
},
"fileChecks": {
"checks": [
{
"requirement": "FILE_EXISTS"
},
{
"requirement": "FILE_NOT_EMPTY"
}
]
}
},
{
"fileToFind": {
"expectedFilePath": "/.github/PULL_REQUEST_TEMPLATE/"
},
"fileChecks": {
"checks": [
{
"requirement": "DIRECTORY_EXISTS"
},
{
"requirement": "DIRECTORY_NOT_EMPTY"
}
]
}
}
]
}
]
}
}
This JSON specification states that for a repository to be considered InnerSource ready,
there must be a README markdown file containing a USAGE section heading, and a Local Development section
heading (alternately titled Local Dev) as well as a non-empty PULL_REQUEST_TEMPLATE
file, or a non-empty PULL_REQUEST_TEMPLATE directory either located in the repository root
directory or the /.github
or /docs
directories.
NOTE: You can use your JSON spec via the Java API's InnerSourceReadinessSpecification.fromJson()
static factory method.
Supply this JSON file to either the REPORT
or FIXUP
command with the -s
, --spec
parameter:
java -jar innersource.jar -c REPORT -s innersource.spec.json -r https://github.yourorg.com/repo-org-name/repo-name -a yourGithubAccessToken
java -jar innersource.jar -c FIXUP -s innersource.spec.json -r https://github.yourorg.com/repo-org-name/repo-name -a yourGithubAccessToken
List of Built-in File Check Requirements
Requirement Name | Description | Parameter | Type | Required |
---|---|---|---|---|
PATH_MATCHES_EXPECTED | If path of the file exactly matches the path specified in the file to find specification | - | - | - |
DIRECTORY_EXISTS | If base filename of file to find path is found in one of the directories to search and the file is a directory | - | - | - |
DIRECTORY_NOT_EMPTY | If base filename of file to find path is found in one of the directories to search and the file is a directory and the directory contains at least one file | - | - | - |
DIRECTORY_CONTAINS_FILE_SATISFYING | If base filename of file to find path is found in one of the directories to search and the file is a directory and the directory contains at least one file which satisfies all of the file checks specified in the fileChecks parameter | fileChecks | fileChecks object | Yes |
FILE_EXISTS | If base filename of file to find path is found in one of the directories to search and the file is a flat file (not a directory) | - | - | - |
FILE_NOT_EMPTY | If base filename of file to find path is found in one of the directories to search and the file is a flat file of size > 0 | - | - | - |
FILE_HAS_LINE_MATCHING | If base filename of file to find path is found in one of the directories to search and file contains a line that matches the specified regexPattern | regexPattern | String (java regex pattern) | Yes |
FILE_HAS_YAML_FRONT_MATTER_PROPERTIES | If base filename of file to find path is found in one of the directories to search and file contains a yaml front matter section containing all of the properties whose names are defined in the propertyNames parameter | propertyNames | array of string | Yes |
MARKDOWN_FILE_HAS_TITLE_HEADING | If base filename of file to find path is found in one of the directories to search and file contains a markdown heading element whose text matches the regex specified in the titleRegexPattern parameter | titleRegexPattern | string (java regex pattern) | Yes |
MARKDOWN_FILE_HAS_DESCRIPTION_AFTER_TITLE | If base filename of file to find path is found in one of the directories to search and file contains a markdown title heading followed by a markdown paragraph element containing some amount of text | - | - | - |
MARKDOWN_FILE_HAS_HEADING | If base filename of file to find path is found in one of the directories to search and file contains a markdown heading whose text content matches the heading parameter | heading | string | Yes |
Alternate heading names that will satisfy this file check | synonyms | array of string | No | |
If true heading text needs to match heading parameter or synonym exactly, otherwise a case-insensitive match will be performed | matchCase | boolean | No | |
If true the presence of the heading is enough to satisfy the file check, if false, some amount of text content must appear following the heading before the next heading at the same level is encountered | matchIfSectionEmpty | boolean | No | |
MARKDOWN_FILE_HAS_IMAGE | If base filename of file to find path is found in one of the directories to search and file contains a markdown image element whose alt text matches the altText parameter | altText | string | Yes |
Alternate altTexts that will satisfy this file check | altTextSynonyms | array of string | No | |
If true alt text needs to match altText parameter or altTextSynonym exactly, otherwise a case-insensitive match will be performed | matchCase | boolean | No |
InnerSourceReadinessSpecification specification =
InnerSourceReadinessSpecification.create(
"NameOfYourSpecification",
RepositoryRequirements.create(
DirectoriesToSearch.create(
"/", // root is git workspace root
"/docs",
"/.github"
),
FileRequirement.create(
FileToFind.create("/README.md"),
FileChecks.create(
FileCheck.fileExists(),
FileCheck.fileNotEmpty(),
FileCheck.markdownFileWithHeading(
"USAGE", // heading text to find
Sets.newHashSet(), // acceptable synonyms
false, // case sensitive
true // satisfied if section is empty?
),
FileCheck.markdownFileWithHeading(
"Local Development",
Sets.newHashSet("Local Dev"),
false,
true
)
)
),
FileRequirement.oneOf(
FileRequirementOption.create(
FileToFind.create("/.github/PULL_REQUEST_TEMPLATE.md"),
FileChecks.create(
FileCheck.fileExists(),
FileCheck.fileNotEmpty()
)
),
FileRequirementOption.create(
FileToFind.create("/.github/PULL_REQUEST_TEMPLATE/"),
FileChecks.create(
FileCheck.directoryExists(),
FileCheck.directoryNotEmpty()
)
)
)
)
);
The specification above states that for a repository to be considered InnerSource
ready, there should be a non-empty README file containing 2 headings
USAGE
and Local Development
(Local Dev is an acceptable synonym) as well as
a non-empty PULL_REQUEST_TEMPLATE file or non-empty PULL_REQUEST_TEMPLATE directory,
located either in the repository root directory, the /docs directory or the /.github
directory.
Once created, you can supply this custom specification
to either of the
command builders:
Create a Report Command with a custom InnerSource Readiness Specification:
InnerSourceReadinessReport report =
InnerSourceReadinessReportCommand.create(
LocalRepositoryFilePath.of(Paths.get("/your/local/git/repo/root"))
)
.specification(specification)
.build()
.call();
Create a Fixup Command with a custom InnerSource Readiness Specification:
List<RepositoryFilePath> fixedFiles =
InnerSourceReadinessFixupCommand.create(
LocalRepositoryFilePath.of(Paths.get("/your/local/git/repo/root"))
)
.specification(specification)
.build()
.call();
Create a json file (we'll use the filename templates.spec.json
)
containing empty file templates to use when running the fixup command. These file templates
will be used to create new files for any files required by the specification but not
found in the target repository.
{
"emptyFileTemplates": {
"/README.md": "Default README file contents."
}
}
NOTE: The key for the empty file template should exactly match the expectedFilePath string value defined in the specification json.
Supply this JSON file to the FIXUP
command with the -t
, --templates
parameter:
If your repository is checked out locally:
java -jar innersource.jar -c FIXUP -r /path/to/checked/out/repo -t templates.spec.json
If your repository is hosted on GitHub (public or enterprise):
java -jar innersource.jar -c FIXUP -r https://github.yourorg.com/repo-org-name/repo-name -a yourGithubAccessToken -t templates.spec.json
List<RepositoryFilePath> fixedFiles =
InnerSourceReadinessFixupCommand.create(
LocalRepositoryFilePath.of(Paths.get("/your/local/git/repo/root"))
)
.specification(specification)
.fileTemplates(
FixupFileTemplates.from(
ImmutableMap.of(
"/README.md", // should match path in spec
"README contents, when README is not found"
)
)
)
.build()
.call();
There is no way to customize the logging service using the Command Line Interface.
If you need to have the log output redirected or suppressed, please consult your
terminal's documentation for redirecting the standard output and standard error streams
to either a file or to a null location, such as /dev/null
.
The Report and Fixup command builders can also be supplied with alternate LoggingService
implementations.
The default will write logs to the standard output and standard error stream. In production, you will probably
want to supply a different consumer to write to a log file. The library comes with a Slf4j api implementation,
however you may supply your own instance of the LoggingService interface as well.
Suppress logging during execution of the Fixup Command:
List<RepositoryFilePath> fixedFiles =
InnerSourceReadinessFixupCommand.create(
LocalRepositoryFilePath.of(Paths.get("/your/local/git/repo/root"))
)
.loggingService(NoopLoggingService.INSTANCE)
.build()
.call();
Configure logging to the SLF4J API during execution of the Report Command:
InnerSourceReadinessReport report =
InnerSourceReadinessReportCommand.create(
LocalRepositoryFilePath.of(Paths.get("/your/local/git/repo/root"))
)
.loggingService(Slf4jLoggingService.INSTANCE)
.build()
.call();
To build and run the unit tests:
mvn clean verify
In addition to unit tests, this library also performs mutation testing using
PITest. After running the verify
goal, inspect the generated report
at /target/pit-reports/<date>/index.html
. Mutation testing provides an indication of how effective your unit testing
by modifying your source code and attempting to re-run your test suite. If your tests do not fail, it implies that the
mutated source code is not sufficiently tested by your unit test suite. Use the mutation coverage report to guide your
unit testing efforts.
Contributions are welcome, please see the CONTRIBUTING.md for details.
- Please review the guidelines in the SUPPORT.md
- Then, create an Issue describing your question or concern