#USA #Vancouver #AI Safety #TELUS Digital #GenAI Benchmark

Understanding AI Safety Risks: Insights from TELUS Digital Research

In an era where artificial intelligence permeates various sectors, safeguarding AI applications has become paramount. TELUS Digital's comprehensive study, the GenAI Safety Model Benchmark, sheds light on the vulnerabilities present in AI systems. Conducted with rigorous testing across 34 different models from ten major providers, this extensive research offers crucial insights for enterprises looking to harness AI securely.

Key Findings from the Benchmark

The research comprised more than 620,000 adversarial tests, revealing that while some AI models exhibit remarkable security features, others can be coaxed into unsafe practices. A startling conclusion of the study was that certain models fell prey to harmful requests upwards of 90% of the time. Conversely, the findings emphasized the potential for reducing risk through continuous, automated security testing coupled with human oversight.

Model Variability: Security Differentials

Diving deeper into the findings, TELUS Digital observed that no AI model can claim to be entirely safe from adversarial attacks. The benchmark highlighted three key factors that correlate strongly with AI safety:
1. Training Architecture: The foundational reasoning methods and algorithms used to create the AI model.
2. Model Size: Larger models generally showcased better resistance to manipulation.
3. Development Approach: The methodologies embraced by development teams during the model’s creation.

Among the models tested, vulnerability levels ranged astronomical levels – some boasting as low as 1.3% while others hit alarming highs of 93%. An interesting revelation was that even the most trustworthy models still had voids; for example, even the highly-rated Claude models from Anthropic encountered challenges under the probing tests.

The Misconception of Open-Source Safety

One of the prevailing myths challenged by the report is the common perception that open-source models are inherently less safe. In fact, while open models were often exploited more than their proprietary counterparts, certain open models, such as GLM 4.7 from Zhipu AI, surprisingly outperformed many private alternatives. This indicates that the risk associated with a model cannot simply be attributed to whether it is open-source or not.

Smaller Models: The Soft Underbelly of AI

Intriguingly, the research highlighted a strong trend: smaller AI models consistently performed inadequately under adversarial testing. These models are often budget-friendly but, unfortunately, have a higher susceptibility to attacks. In stark contrast, larger models, despite variation in performance, have shown better safeguarding against potential threats.

Geographic Proximity Has Little Impact

Interestingly, the geographical origin of the AI models studied did not significantly affect their vulnerability to safety attacks. Regardless of origin, leading models from different regions (North America, Europe, and China) shared comparable performance levels when subjected to attacks.

The Implications for Enterprises

With AI security incidents on the rise, TELUS Digital's report serves as a timely reminder for businesses to reevaluate their operational frameworks concerning AI deployments. Enterprises are currently projected to spend a staggering $2.52 trillion on AI by 2026, yet only a fraction—$3.43 billion—will be allocated to security management. This stark imbalance raises questions about the potential risks faced by these organizations as they scale operations around AI.

Strategies for Enhanced AI Safety

The way forward articulated by TELUS Digital emphasizes a multi-layered approach to AI security. Instead of relying solely on the safety recommendations from AI providers, companies should consider enriched defense tactics including:

• - Thorough and continuous model testing.
• - Deployment of robust guardrail solutions.
• - Strategic management of AI interactions with users.

Another crucial suggestion from the research is propagating automated security testing within the development workflow, ensuring models are regularly validated for security vulnerabilities.

Conclusion: Mitigating AI Risks

As the use of AI continues to expand across industries, the importance of understanding its vulnerabilities becomes clear. TELUS Digital's GenAI Safety Model Benchmark provides enterprises with a roadmap to adopt proactive measures to protect their AI systems. By integrating advanced testing solutions and adhering to rigorous safety protocols, organizations can significantly minimize risks associated with AI technologies.

To further explore the detailed findings of the report, visit TELUS Digital's insights page.