pkgdiff-stability.Rmd

---
title: "Stability Score"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Stability Score}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

The function `pkg_stability()` provides a **Stability Score** for a selected 
package.  This function looks at the entire history of package releases,
and compares them to identify breaking changes.  It then uses the number
of breaking changes to calculate a **Stability Score**.  This vignette 
describes how the score is calculated.

### Breaking Changes

Before describing how the **Stability Score** is calculated, we first have to 
understand what is meant by a "breaking change".  In the context of **pkgdiff**,
a "breaking change" is a function that exists in one release, but is removed in the next.
It can also be a function parameter that exists in one release, but is removed
in the next.  There are, therefore, two kinds of breaking changes tracked
by **pkgdiff**: removed functions and removed parameters.  Either one of these
will count as a breaking change.

Note that there are other kinds of changes not tracked by **pkgdiff**.  Functions
that are changed, but not removed, are not tracked.  Also, functions that 
are deprecated, but not removed, are not tracked.  That means you can start 
getting a deprecation warning on a function, but as long as it is not actually
removed, **pkgdiff** does not count that as a breaking change.

### Percentage of Non-Breaking Changes

A package version with at least one breaking change from the previous version is called 
a "breaking release". A package version that is compatible with the previous 
version is called a "non-breaking release".  

The **Stability Score** is calculated as the percentage of non-breaking releases.
In other words, the **Stability Score** is the number of breaking releases 
divided by the total number of releases, subtracted from one.  The basic 
formula for the stability score is thus: 1 - (BR / TR), where 
BR is Breaking Releases and TR is Total Releases.

For example, if a package has 10 releases, and one of them is a breaking 
release, the **Stability Score** will be .9 or 90%.

### Weighting Over Time

Breaking releases are weighted over time.  This consideration reflects the 
perception that breaking releases many years ago matter less than recent breaking
releases.  Breaking releases therefore start with a value of 
one (1), and that value is reduced by .01 for each month away from the present
date.  That means a breaking release in the last month will count as 1, and
a breaking release two years ago will count as 1 - (.01 * 24) = 0.76.

Observe that this weighting over time will eventually cause the breaking release
to not be counted at all.  The discounting of a breaking release occurs at 8.333
years.  That means breaking releases older than 8.333 years will not be considered
in the **Stability Score**.

### Other Considerations

Also note that the number of releases eventually age out of the calculation. 
Any release older
than 10 years from the current date will be discarded from the Stability Score.
These releases will still be shown in the Stability Data. Keeping them in 
the data helps to give you an accurate understanding of the package history.

## Stability Assessment

In addition to the above Stability Score, the `pkg_stability()` function also provides
a **Stability Assessment**.  The **Stability Assessment** is a categorization of
the Stability Score.  The Stability Assessment is a way to give you an
English interpretation of the Stability Score.  The Stability Assessment is 
determined as follows:

  - 100%: Perfect
  - >= 95%: Very Stable
  - >= 90%: Stable
  - >= 80%: Somewhat Unstable
  - < 80%: Unstable
  
The above categorization sets a high bar. The purpose of setting a high bar is 
simply to bring the 
R Open Source language up to the stability level of proprietary 
languages.  It was designed to promote stability and encourage package
developers to reduce the number of breaking changes in their Open Source packages.
A high stability score is something that R package developers should be proud of, 
and R users should look for when selecting packages to include in their programs.

## CRAN Packages Only

The **Stability Score** and **Stability Assessment** only work with packages 
published on CRAN.
The reason is because CRAN preserves all releases of a package, and those archived
releases are easily available to **pkgdiff**.  Releases on Github are not reliable,
as they are subject to whether the package developer remembers to create 
a release version or not.