This is a Python package to compute basic signal detection theory measures and to fit theoretical recognition memory models to data using ROC curves.
Currently-supported models are those most frequently seen in the literature:
- High Threshold
- Equal-Variance Signal Detection
- Unequal-Variance Signal Detection
- Dual-Process Signal Detection
These models are fit to observed data by minimising the
Install with pip
:
$ pip install roc-face
It is generally recommend to use a virtual environment when installing python packages (see here).
With a single true positive and false positive rate, compute the common measures of sensitivity and bias:
>>> from roc_face import measures
>>> measures.d_prime(0.75, 0.21)
1.480910997214322
>>> measures.c_bias(0.75, 0.21)
0.06596574841107933
You can also return a dictionary of measures in a similar way:
>>> measures.compute_performance(tpr=0.75, fpr=0.21)
{
'FPR': 0.21,
'TPR': 0.75,
'dprime': 1.480910997214322,
'cbias': 0.06596574841107933,
'aprime': 0.850886075949367,
'beta': 1.1026202605581668,
}
Given a set of responses to a set of signal and noise trials, the ROC and z-ROC plots of the observed frequencies can be viewed as follows.
>>> import matplotlib.pyplot as plt
>>> from roc_face import utils
# Strongest "signal" <---> Strongest "noise"
# All responses to signal-present trials
>>> signal = [505,248,226,172,144,93]
# All responses to signal-absent (i.e. noise) trials
>>> noise = [115,185,304,523,551,397]
>>> fig, ax = plt.subplots(1, 2)
>>> utils.plot_roc(signal, noise, ax=ax[0])
>>> utils.plot_zroc(signal, noise, poly=1, ax=ax[1])
>>> ax[1].legend()
>>> plt.show()
The utils.plot_roc
and utils.plot_zroc
functions are for convenience as
they carry out some minor plotting customisations (square axes, chance-line,
etc). The fitted line on the z-ROC in this example is a simple linear model,
which is useful for interpreting the ROC data.
With the signal and noise data, the different models can be fitted.
>>> from roc_face.models import SignalDetection
# Create an equal- and unequal-variance signal detection models
>>> evsd = SignalDetection(signal, noise)
>>> uvsd = SignalDetection(signal, noise, equal_variance=False)
After creation, the models are fit as follows:
# Fit the models using the G-test fit function
>>> evsd.fit(verbose=True)
(
{
'model': 'Equal Variance Signal Detection',
'success': True,
'method': 'G',
'statistic': 86.65649645461215,
'log_likelihood': -13560.04271197083,
'AIC': 27132.08542394166,
'BIC': 27171.729820697474,
'SSE': 0.0038210383584672968
},
{
'd': 1.3706692903039621,
'criteria': array([ 0.87497843, 0.40584089, -0.01393461, -0.48919362, -1.0589949 ])
}
)
Using verbose=True
in the fit
method prints out the results of the fitting procedure when it ends, along with the parameter estimates. After calling fit
, they can also be accessed via .results
and .parameter_estimates
:
>>> uvsd.fit()
>>> uvsd.results
{
'model': 'Unequal Variance Signal Detection',
'success': True,
'method': 'G',
'statistic': 0.9957013337195149,
'log_likelihood': -13517.212314410383,
'AIC': 27048.424628820765,
'BIC': 27094.676425035883,
'SSE': 0.00014501577579512347
}
>>> uvsd.parameter_estimates
{
'd': 1.6023922394537697,
'scale': 1.32284210876227,
'criteria': array([ 0.95984187, 0.36806527, -0.12572417, -0.65421799, -1.2496735 ])
}
These models can also be compared to one another. Although it is common practice to compare with the AIC or BIC values (see the results), it can also be done with the .compare
method:
>>> evsd.compare(uvsd)
('G(EVSD - UVSD)', 85.66079512089179, 1, 2.1360619666273588e-20)
This shows that the UVSD provides a significantly better fit than the EVSD.
Finally, we can just view the ROC data and the two fitted models as follows:
>>> fig, ax = plt.subplots(dpi=120)
>>> utils.plot_roc(signal, noise, c='k', ax=ax, label='data')
>>> ax.plot(*evsd.curve, label='EVSD')
>>> ax.plot(*uvsd.curve, label='UVSD')
>>> ax.legend(loc='lower right')
>>> plt.show()