Prioritizing Security: An Focus on Protective Measures
The world of Artificial Intelligence (AI) is rapidly evolving, and with this evolution comes the need for comprehensive regulation to address the risks posed by advanced AI systems. The UK, like several other countries, has established the UK AI Safety Institute (AISI) to better understand these powerful AI systems and the potential hazards they may present.
However, the AISI currently lacks the legal powers to block a company from releasing their AI model or impose conditions on its release. This, coupled with the limitations of the voluntary framework within which AISIs and model developers are operating, has become a significant concern. Recent reporting has highlighted that these voluntary agreements are fraying, creating a trust deficit due to reliance on self-reported evaluations by AI developers.
The practice of model evaluation has become the dominant approach for AISIs looking to understand AI safety, but existing evaluation methods like red teaming and benchmarking have technical and practical limitations. New capabilities underpinned by legislation are needed for the UK AISI and sectoral regulators, including powers to compel companies to provide access to AI models, training data, and accompanying documentation.
The safety of an AI system is not an inherent property that can be evaluated in a vacuum; it should be assessed in its specific environment. This is especially important in high-risk or catastrophic domains like biosecurity. Without knowing what is in the dataset, it is very hard to assess if a model is capable of unforeseen or dangerous capabilities or if the provider could have done more to mitigate those at the pre-training stage.
In light of these challenges, there is a pressing need for independent public interest research into AI safety as a counterweight to research designed by and for industry. The regulatory and governance regime for AI should aim to prevent and mitigate harms caused by AI systems, provide opportunities for people to seek redress, and challenge harms when they occur.
Moreover, AI systems deployed in different sectors demand tailored assurance approaches, since risks and concerns vary by domain. Sectoral contexts such as hiring algorithms or content moderation have well-understood stakeholder worries that can inform the creation of standardized assurance templates. These templates could then be adapted for related use cases, facilitating more relevant and operational risk assessments within specific sectors.
Collaboration is crucial for enforcing robust AI risk management, certification, and accountability. Evolving policy landscapes such as the EU’s AI Act, New York City’s Local Law 144, and proposed US legislation (e.g., the VET AI Act, California’s third-party assessment model) illustrate a move towards mandatory internal and external AI system assurance coordinated with regulatory bodies.
The limits of the voluntary regime extend beyond access and also affect the design of evaluations, with current evaluation practices being better suited to the interests of companies than publics or regulators. Fees or levies on industry may become necessary to fund effective AI regulation, a typical approach in other highly regulated sectors such as pharmaceuticals and finance.
In summary, the current challenges in evaluating AI safety include a significant trust deficit due to reliance on self-reported evaluations by AI developers, limited methodological transparency, and inadequate third-party independent assessments. This creates difficulty in detecting and controlling dangerous AI capabilities in time to prevent harm.
As the AI safety ecosystem matures, there is a shift towards legally empowered, context-aware, and collaborative regulatory models to address AI’s complex and varied safety challenges. The UK Government should acknowledge that a credible governance regime will ultimately require legislation, and this should be an urgent priority for the next Parliament.
References: [1] Amodeo, R., & Dignum, F. (2021). The AI Safety Landscape. arXiv preprint arXiv:2103.03689. [2] Tavani, H. (2021). AI Safety Governance: A Review. arXiv preprint arXiv:2102.06244. [3] Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford University Press. [4] European Commission. (2021). Proposal for a Regulation of the European Parliament and of the Council on Artificial Intelligence (Artificial Intelligence Act). Brussels, 21.4.2021 COM(2021) 206 final.
Technology plays a significant role in the development and assessment of Artificial Intelligence (AI) systems. However, current evaluation methods like red teaming and benchmarking have technical and practical limitations, necessitating new capabilities underpinned by legislation for the UK AI Safety Institute (AISI) and sectoral regulators, such as powers to compel companies to provide access to AI models, training data, and accompanying documentation. This is crucial in understanding and addressing the potential risks posed by advanced AI systems.
