Aberson, S.D., 1998: Five-day Tropical cyclone track forecasts in the North
Atlantic Basin. Weather & Forecasting, 13, 1005-1015.
Atlantic Basin. Weather & Forecasting, 13, 1005-1015.
Barker, T. W., 1991: The relationship between spread and forecast error in
extended-range forecasts. Journal of Climate, 4, 733–742.
extended-range forecasts. Journal of Climate, 4, 733–742.
Bradley, A.A., S.S. Schwartz, and T. Hashino, 2008: Sampling Uncertainty
and Confidence Intervals for the Brier Score and Brier Skill Score.
*Weather and Forecasting*, 23, 992-1006.
and Confidence Intervals for the Brier Score and Brier Skill Score.
*Weather and Forecasting*, 23, 992-1006.
Brill, K. F., and F. Mesinger, 2009: Applying a general analytic method
for assessing bias sensitivity to bias-adjusted threat and equitable
threat scores. Weather and Forecasting, 24, 1748–1754.
for assessing bias sensitivity to bias-adjusted threat and equitable
threat scores. Weather and Forecasting, 24, 1748–1754.
Brown, B.G., R. Bullock, J. Halley Gotway, D. Ahijevych, C. Davis,
E. Gilleland, and L. Holland, 2007: Application of the MODE object-based
verification tool for the evaluation of model precipitation fields.
*AMS 22nd Conference on Weather Analysis and Forecasting and 18th*
*Conference on Numerical Weather Prediction*, 25-29 June, Park City, Utah,
American Meteorological Society (Boston), Available at
http://ams.confex.com/ams/pdfpapers/124856.pdf.
E. Gilleland, and L. Holland, 2007: Application of the MODE object-based
verification tool for the evaluation of model precipitation fields.
*AMS 22nd Conference on Weather Analysis and Forecasting and 18th*
*Conference on Numerical Weather Prediction*, 25-29 June, Park City, Utah,
American Meteorological Society (Boston), Available at
http://ams.confex.com/ams/pdfpapers/124856.pdf.
Buizza, R., 1997: Potential forecast skill of ensemble prediction and spread
and skill distributions of the ECMWF ensemble prediction system. Monthly
*Weather Review*,125, 99–119.
and skill distributions of the ECMWF ensemble prediction system. Monthly
*Weather Review*,125, 99–119.
Bullock, R., T. Fowler, and B. Brown, 2016: Method for Object-Based
Diagnostic Evaluation. NCAR Technical Note NCAR/TN-532+STR, 66 pp.
Diagnostic Evaluation. NCAR Technical Note NCAR/TN-532+STR, 66 pp.
Candille, G., and O. Talagrand, 2008: Impact of observational error on the
validation of ensemble prediction systems. Quarterly Journal of the Royal
*Meteorological Society* 134: 959–971.
validation of ensemble prediction systems. Quarterly Journal of the Royal
*Meteorological Society* 134: 959–971.
Casati, B., G. Ross, and D. Stephenson, 2004: A new intensity-scale approach
for the verification of spatial precipitation forecasts. Meteorological
*Applications* 11, 141-154.
for the verification of spatial precipitation forecasts. Meteorological
*Applications* 11, 141-154.
Davis, C.A., B.G. Brown, and R.G. Bullock, 2006a: Object-based verification
of precipitation forecasts, Part I: Methodology and application to
mesoscale rain areas. Monthly Weather Review, 134, 1772-1784.
of precipitation forecasts, Part I: Methodology and application to
mesoscale rain areas. Monthly Weather Review, 134, 1772-1784.
Davis, C.A., B.G. Brown, and R.G. Bullock, 2006b: Object-based verification
of precipitation forecasts, Part II: Application to convective rain systems.
*Monthly Weather Review*, 134, 1785-1795.
of precipitation forecasts, Part II: Application to convective rain systems.
*Monthly Weather Review*, 134, 1785-1795.
Dawid, A.P., 1984: Statistical theory: The prequential approach. Journal of
*the Royal Statistical Society* A147, 278-292.
*the Royal Statistical Society* A147, 278-292.
Ebert, E.E., 2008: Fuzzy verification of high-resolution gridded forecasts:
a review and proposed framework. Meteorological Applications, 15, 51-64.
a review and proposed framework. Meteorological Applications, 15, 51-64.
Eckel, F. A., M.S. Allen, M. C. Sittel, 2012: Estimation of Ambiguity in
Ensemble Forecasts. Weather Forecasting, 27, 50-69.
doi: http://dx.doi.org/10.1175/WAF-D-11-00015.1
Ensemble Forecasts. Weather Forecasting, 27, 50-69.
doi: http://dx.doi.org/10.1175/WAF-D-11-00015.1
Efron, B. 2007: Correlation and large-scale significance testing. Journal
of the American Statistical Association,* 102(477), 93-103.
of the American Statistical Association,* 102(477), 93-103.
Gilleland, E., 2010: Confidence intervals for forecast verification.
*NCAR Technical Note* NCAR/TN-479+STR, 71pp.
*NCAR Technical Note* NCAR/TN-479+STR, 71pp.
Gilleland, E., 2019: Bootstrap methods for statistical inference. Part II:
Extreme-value analysis. Submitted to the Journal of Atmospheric and
Oceanic Technology on 2 December 2019. Re-submitted on 12 May 2020
Extreme-value analysis. Submitted to the Journal of Atmospheric and
Oceanic Technology on 2 December 2019. Re-submitted on 12 May 2020
Gneiting, T., A. Westveld, A. Raferty, and T. Goldman, 2004: Calibrated
*Probabilistic Forecasting Using Ensemble Model Output Statistics and*
*Minimum CRPS Estimation*. Technical Report no. 449, Department of
Statistics, University of Washington. Available at
http://www.stat.washington.edu/www/research/reports/
*Probabilistic Forecasting Using Ensemble Model Output Statistics and*
*Minimum CRPS Estimation*. Technical Report no. 449, Department of
Statistics, University of Washington. Available at
http://www.stat.washington.edu/www/research/reports/
Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble
forecasts. Monthly Weather Review, 129, 550-560.
forecasts. Monthly Weather Review, 129, 550-560.
Hersbach, H., 2000: Decomposition of the Continuous Ranked Probability Score
for Ensemble Prediction Systems. Weather and Forecasting, 15, 559-570.
for Ensemble Prediction Systems. Weather and Forecasting, 15, 559-570.
Jolliffe, I.T., and D.B. Stephenson, 2012: Forecast verification. A
*practitioner's guide in atmospheric science.* Wiley and Sons Ltd, 240 pp.
*practitioner's guide in atmospheric science.* Wiley and Sons Ltd, 240 pp.
Knaff, J.A., M. DeMaria, C.R. Sampson, and J.M. Gross, 2003: Statistical,
Five-Day Tropical Cyclone Intensity Forecasts Derived from Climatology
and Persistence. Weather & Forecasting, Vol. 18 Issue 2, p. 80-92.
Five-Day Tropical Cyclone Intensity Forecasts Derived from Climatology
and Persistence. Weather & Forecasting, Vol. 18 Issue 2, p. 80-92.
Mason, S. J., 2004: On Using “Climatology” as a Reference Strategy
in the Brier and Ranked Probability Skill Scores. Monthly Weather Review,
132, 1891–1895.
in the Brier and Ranked Probability Skill Scores. Monthly Weather Review,
132, 1891–1895.
Mittermaier, M., 2014: A strategy for verifying near-convection-resolving
model forecasts at observing sites. Weather Forecasting, 29, 185-204.
model forecasts at observing sites. Weather Forecasting, 29, 185-204.
Mood, A. M., F. A. Graybill and D. C. Boes, 1974: Introduction to the
*Theory of Statistics*, McGraw-Hill, 299-338.
*Theory of Statistics*, McGraw-Hill, 299-338.
Murphy, A.H., and R.L. Winkler, 1987: A general framework for forecast
verification. Monthly Weather Review, 115, 1330-1338.
verification. Monthly Weather Review, 115, 1330-1338.
Roberts, N.M., and H.W. Lean, 2008: Scale-selective verification of rainfall
accumulations from high-resolution forecasts of convective events.
*Monthly Weather Review,* 136, 78-97.
accumulations from high-resolution forecasts of convective events.
*Monthly Weather Review,* 136, 78-97.
Saetra O., H. Hersbach, J-R Bidlot, D. Richardson, 2004: Effects of
observation errors on the statistics for ensemble spread and
reliability. Monthly Weather Review 132: 1487–1501.
observation errors on the statistics for ensemble spread and
reliability. Monthly Weather Review 132: 1487–1501.
Santos C. and A. Ghelli, 2012: Observational probability method to assess
ensemble precipitation forecasts. Quarterly Journal of the Royal
*Meteorological Society* 138: 209–221.
ensemble precipitation forecasts. Quarterly Journal of the Royal
*Meteorological Society* 138: 209–221.
Schwartz C. and Sobash R., 2017: Generating Probabilistic Forecasts from
Convection-Allowing Ensembles Using Neighborhood Approaches: A Review
and Recommendations. Monthly Weather Review, 145, 3397-3418.
Convection-Allowing Ensembles Using Neighborhood Approaches: A Review
and Recommendations. Monthly Weather Review, 145, 3397-3418.
Stephenson, D.B., 2000: Use of the “Odds Ratio” for diagnosing
forecast skill. Weather and Forecasting, 15, 221-232.
forecast skill. Weather and Forecasting, 15, 221-232.
Stephenson, D.B., B. Casati, C.A.T. Ferro, and C.A. Wilson, 2008: The extreme
dependency score: A non-vanishing measure for forecasts of rare events.
*Meteorological Applications* 15, 41-50.
dependency score: A non-vanishing measure for forecasts of rare events.
*Meteorological Applications* 15, 41-50.
Weniger, M., F. Kapp, and P. Friederichs, 2016: Spatial Verification Using
Wavelet Transforms: A Review. Quarterly Journal of the Royal
*Meteorological Society,* 143, 120-136.
Wavelet Transforms: A Review. Quarterly Journal of the Royal
*Meteorological Society,* 143, 120-136.
Wilks, D.S. 2010: Sampling distributions of the Brier score and Brier skill
score under serial dependence. Quarterly Journal of the Royal
*Meteorological Society,*, 136, 2109–2118. doi:10.1002/qj.709
score under serial dependence. Quarterly Journal of the Royal
*Meteorological Society,*, 136, 2109–2118. doi:10.1002/qj.709
Wilks, D., 2011: Statistical methods in the atmospheric sciences.
Elsevier, San Diego.
Elsevier, San Diego.