Transform US AI Dominance in California by Prioritizing Post-Deployment Safety Strategies Globally
Article Rewrite:
Gov. Gavin Newsom's scrapping of California's AI safety bill (SB 1047) isn't all bad, but there's still an underlying issue that needs addressing. The bill's flawed approach to AI safety is rooted in its attempts to pinpoint risky systems via pre-deployment assessments. Even the most meticulous pre-deployment evaluations cannot fully anticipate the potential harms these AI models might cause, as their impact is heavily influenced by their real-world applications.
Politicians might be tempted to come up with new metrics to better foresee harmful models, but this won't be enough to create regulations that specifically target AI's pressing and ever-evolving risks. California may not be the AI regulation pioneer, but it might be the birthplace of a more effective approach. The upcoming International Network of AI Safety Institutes (AISIs) meeting in San Francisco provides an ideal opportunity to shift focus towards post-deployment evaluations, offering policymakers the insights they need to handle risks while nurturing innovation.
Post-deployment evaluations periodically assess systems already in use, focusing on evaluating their performance in real-world scenarios. For instance, the U.S. National Institute for Standards and Technology (NIST) has been routinely assessing the capabilities of commercial facial recognition algorithms since 2000. Recently, NIST evaluated how well these systems could identify masked faces owing to societal changes since the COVID-19 pandemic. The results showed that some algorithms that excelled at identifying unmasked faces struggled with masked faces, demonstrating that systems that perform well in controlled, pre-deployment tests can face challenges in real-life situations.
However, AISIs haven't focused on expanding these types of evaluations to gauge how well foundation models function once deployed in high-risk areas, despite this approach better capturing the nuances of real-world AI usage and risks. For example, AISIs could develop vendor tests to assess the summarization abilities of large language models (LLMs) when processing complex health information for public consumption-a project NIST has already started on non-commercial systems. In fact, NIST has been developing AI measurement and evaluation projects for years, focusing on functions like information retrieval, natural language processing, and speech processing. It just hasn't developed these projects into post-deployment evaluations like it has for facial recognition.
Post-deployment evaluations play a crucial role in ensuring AI models are functioning safely and effectively in real-world conditions, fostering AI accountability. AISIs are well-equipped to develop post-deployment evaluations that assess and communicate AI systems' performance in practice, identifying any limitations or risks that emerge after deployment. These evaluations supply the technical insights that policymakers need to decide the appropriate and responsible ways to use AI.
Although AISIs lack enforcement powers, their primary role is to advance the science of AI safety; spread and disseminate AI safety practices; and support institutions, communities, and collaboration around AI safety. For instance, NIST doesn't dictate where or when facial recognition technologies should be used; instead, it focuses on assessing how well these systems perform under various conditions. The decision of where and how these technologies are deployed should be left to regulatory bodies and policymakers.
To make post-deployment evaluations effective for foundation models and encourage more focused regulation, AISIs must go beyond simply applying the existing NIST evaluation projects.
First, AISIs should establish a global framework for tracking AI incidents—monitoring when and how AI systems break down in real-world conditions. While use cases for facial recognition technology are more defined, foundation models can be applied in a wide range of unpredictable areas, making it difficult to predict which applications pose the highest risk. Trying to track every possible use case and pinpoint the most important ones is impractical and ineffective for prioritizing risks. Instead, tracking actual incidents and failures as they occur can supply data to guide more precise, targeted evaluations, ensuring the focus remains on the most pressing risks.
Second, AISIs should expand post-deployment evaluations to include broader societal impacts of foundation models, which lie at the heart of many public concerns. At present, post-deployment evaluations focus primarily on technical performance, even when applied in real-world contexts. However, foundation models' risks extend beyond technical errors or confidence thresholds, as the public's understanding, trust, and misuse of these models can lead to real-world consequences. AISIs should explore how to evaluate not only their technical performance but also their impact on public behavior, trust, and societal outcomes. Additionally, concerns about energy consumption and environmental impact should be taken into account through post-deployment assessments of energy efficiency, measuring the performance of different models across various applications. By taking a sociotechnical approach, AISIs can supply a more comprehensive picture of how foundation models affect public trust, behavior, and broader societal outcomes.
America has a lot riding on the line at November's convening. With more frontier AI companies than the rest combined, U.S. policymakers have the most to gain and the most to lose if AI guardrails are built upon shaky ground. Policymakers should seize this opportunity on the global stage to establish an approach to AI safety that better considers the evolving risks of AI models in the real world.
Image Credit: AP Photo/Manuel Balce Ceneta, File
Enrichment Insights Integration:AISIs aim to enhance transparency and accountability by involving third-party evaluators, civil society groups, and public sector entities in reporting and evaluation processes of AI models and compute clusters post-deployment. They are also developing human-centered safety designs that facilitate human oversight and agency in AI-assisted tasks such as software engineering. Additionally, their network structure facilitates coordinated safety evaluations and regulatory alignment across countries, contributing to effective accountability mechanisms. Lastly, they are working on the creation of robust benchmarks, testing, and evaluation frameworks specifically tailored for foundation models, addressing their complexity and multimodal nature for real-world safety assurance.
- The lack of post-deployment evaluations for foundation models in high-risk areas, as proposed by the International Network of AI Safety Institutes (AISIs), can provide valuable insights into their performance and risks in real-world scenarios, ensuring technological advancements align with policy objectives.
- To adequately address the evolving risks associated with AI models, AISIs should consider expanding their focus beyond technical performance, incorporating evaluations of public understanding, trust, and societal outcomes to create a more comprehensive picture of potential impacts.
- The upcoming AISIs meeting offers an opportunity for policymakers to deliberate on a more effective approach to AI regulation by championing post-deployment evaluations that can provide the data and insights necessary to make informed decisions regarding AI usage.
- AISIs' role in AI safety includes fostering collaboration among third-party evaluators, civil society groups, and public sector entities to improve transparency, accountability, and human oversight in AI-assisted tasks, ultimately contributing to both innovation and responsible policy development.
