We define here a number of performance indicators for a value-at-risk (VaR) model, together with traffic lights (“Red”, “Yellow”, “Green”) indicators or zones, in the spirit of the backtesting framework introduced by Basel Committee.
For each of the performance indicators, the Green zone corresponds to results that do not suggest a problem with the quality or accuracy of the model, the Yellow zone corresponds to results that do or should raise questions in this regard but, where any conclusion is not definitive. Finally the Red zone indicates a result that almost certainly is problematic and potentially due to a problem with the model, the data or fundamental or is reflective of fundamental changes in market environment that render the model inapplicable.
Basel Committee introduced a framework for the supervisory interpretation of financial institutions’ backtesting results. It defines the three zones (Green, Yellow, and Red) by identifying the starting points of the Yellow and the Red zone. The Yellow Zone begins at the point such that the probability of obtaining that number or fewer exceptions equals or exceeds 95%, the Red Zone begins at the point such that the probability of obtaining that number or fewer of exceptions equals or exceeds 99.99%. With those definitions, it then proceeds to define the three zones for the 1-year worth of backtesting results (250-days) at the 99% confidence level (i.e. the level at which the VaR risk measure is normally calculated).
The Basel backtesting procedure tests the null hypothesis that the VaR model predicts losses accurately against the alternative hypothesis that the model predicts losses incorrectly (see next section):
H0: VaR model is correct vs. H1: VaR model is incorrect
The test statistic used is based on the number of exceptions generated by the VaR model. For a given trading day, an exception occurs when the loss exceeds the model-based VaR. The test postulates that the probability of an exception, p, is equal to 0.01 and tests it against the alternative hypothesis that the probability of an exception is greater than 0.01. The test is based on the number of exceptions in 250 trading days. Given the chosen test statistic, the test rejects the VaR model if the number of exceptions is greater than or equal to 10 and accepts the model otherwise (see technical details below).
When evaluating statistical tests, it is common practice to examine their type I and II error rates. Under the Basel backtesting procedure, the type I error rate is the probability of rejecting the VaR model when the model is correct, while the type II error rate is the probability of accepting the model when the model is incorrect.
The Basel backtesting procedure is designed for controlling the type I error rate (or α level), the probability of rejecting the VaR model when the model is correct. For this test, type I error rate is the probability that number of exceptions, out of 250 daily observations, is greater than or equal to 10, when the probability of an exception is p = 0.01. The type I error rate of the test is 0.0003 (or α = 0.03%), see next section for technical details. Although the test establishes a very conservative threshold for controlling the type I error rate, it is not designed to control the type II error rate. Therefore, it does not control the probability of accepting the VaR model when the model is incorrect.
Based on the number of exceptions of the VaR model, Basel defines three zones. The green zone consists of four or fewer exceptions, and in this case the VaR model is assessed as correct. The yellow zone includes five to nine exceptions, and the accuracy of the VaR model is questioned. The red zone corresponds to 10 or more exceptions, and the model is rejected.
Taking Basel zone definitions, we extend this framework to backtesting results over different time horizons (1 year (250 days) and 3 years (750 days)) and different confidence levels (99.5%, 99% and 95%)).
The points where the yellow and red zones start are calculated according to the methodology described above, with for 1 year time horizon and for 3 years’ time horizon and for 99% confidence level model, for 99.5% confidence level model and for 95% confidence level model.
For instance, when and , so time horizon is 3 years and confidence level is 99.5%, the yellow zone starts at 7 exceptions and the red zone starts at 13 exceptions.
VaR is used to measure market risk for securities, derivatives, and derivative securities, such as https://finpricing.com/lib/FiBond.html