Contributor(s): Nathaniel Li
-- Aaron Gertler, 09/26/2021Today we celebrate not destroying the world. We do so today because 38 years ago, Stanislav Petrov made a decision that averted tremendous calamity. It's possible that an all-out nuclear exchange between the US and USSR would not have actually destroyed the world, but there are few things with an equal chance of doing so.
As a Lieutenant Colonel of the Soviet Army, Petrov manned the system built to detect whether the US government had fired nuclear weapons on Russia. On September 26th, 1983, the system reported five incoming missiles. Petrov’s job was to report this as an attack to his superiors, who would launch a retaliative nuclear response. But instead, contrary to the evidence the systems were giving him, he called it in as a false alarm, for he did not wish to instigate nuclear armageddon.
For more information, see: 1983 Soviet nuclear false alarm incident
Petrov is not alone in having made decisions that averted destruction — presidents, generals, commanders of nuclear submarines, and similar also made brave and fortunate calls — but Petrov's story is salient, so today we celebrate him and all those who chose equally well.
As the world progresses, it's likely that many more people will face decisions like Petrov's. Let's hope they'll make good decisions! And if we expect to face decisions ourselves, let us resolve to decide wisely!
This lecture discusses the applications of machine learning to advance institutional decision-making. Human judgement is not always perfect, particularly in moments of crisis or disagreement. Thus, we aim to improve the decision-making of institutions and leaders by employing ML systems, which could have more objective, calibrated, and accurate judgements. Unlike other topics in this course, this note explores topics which can be applied to solve issues outside of AI safety. For example, ML systems can be applied to better decision-making in international relations, conflict de-escalation, or pandemic response.
Within AI safety, improved decision-making can avoid risks from highly persuasive AI systems and promote the responsible use of advanced AI capabilities, contributing to *systemic safety*.
Forecasting is the process of making predictions about events, and refining predictions as new information becomes available. Advanced ML systems could augment human forecasters by processing text more quickly, discerning patterns in high-dimensional spaces that humans cannot interpret, or considering large amounts of context across a variety of topics and time horizons. Specifically, highly capable ML systems could act as advisory systems, incorporating large quantities of historical context to help brainstorm ideas or risks. There are two main categories of forecasting:
- Statistical Forecasting - uses statistical tools such as moving average, autogression, other ML time-series models, etc. to make predictions.
- Judgemental Forecasting - uses forecasters’ own judgement integrated from statistical models, news, accumulated knowledge, a priori reasoning, etc.
Forecasters can retrieve materials, such as recent news articles or videos, to refine their predictions with additional context. Forecasting also involves calibration, providing confidence intervals to reflect the uncertainty of predictions.
Good forecasters should have:
- broad adaptation across topics, question schemes (multiple choice, true/false, numerical), time horizon, etc.
- granulated predictions with fine confidence intervals.
- high calibration of probabilities - tools for assessing calibration include the Brier score, RMS calibration error, and confidence interval calibration, which are explained at length in the interpretable uncertainty lecture.
Autocast is a benchmark for measuring the forecasting performance of language models (Zou et al. 2022). It includes a machine forecasting dataset with a diverse set of forecasting questions, spanning a variety of topics, time horizons, and forecasting websites. It also includes a corpus of news articles for information retrieval.
(Zou et al. 2022) trains four models on the Autocast dataset, and conclude that larger models which incorporate information retrieval perform best, although these ML methods still lag significantly behind human expert performance. A potential weakness of Autocast may be its contribution to safety-capabilities externalities; the authors suggest that if forecasting fits more generally into AI capabilities, they would discourage work in this area.
Zou, A., Xiao, T., Jia, R., Kwon, J., Mazeika, M., Li, R., ... & Hendrycks, D. (2022). Forecasting Future World Events with Neural Networks. arXiv preprint arXiv:2206.15474.