The stakes are higher at the level of public policy. Axios reported that the warnings have been a “gut-level question for lawmakers” who faced more pressing calls to consider AI safeguards, investigations, and efforts to engage in federal oversight. But probabilities of doom do not say where to allocate resources. For example, if the main avenue of threat is biological weapons, the policy response could consist of more access to biological information and public health defenses. But if, say, the concern is recursive self-improvement, then understanding the constraints on feedback loops is more important.
Calling everything existential may be provocative but obscures the more thoughtful policy and model development work that will be necessary to build the appropriate safeguards. Uncertainty is not the same as complacency, but rather an argument for understanding what experts can know and learn to design interventions that are likely to be effective.
The conclusion is not that policymakers should resolve all uncertainty before acting. Most decisions in life are made without complete information, but false precision is clouding rather than clarifying the tradeoffs. One can say that an outcome is plausible and consequential and therefore warrants safeguards without saying that the probability of apocalypse is greater than 10 percent. There is a lot that experts still don’t know, and uncovering those unknowns can help separate the pathways that are more likely and deserve guardrails from those where existing constraints will provide a natural barrier to extinction.
