#USA #Paris #AI Governance #Sopra Steria #Automated Decisions

The Emerging Importance of AI Governance

In a rapidly evolving technological landscape, the governance of artificial intelligence is coming into sharper focus. According to a recent report from Sopra Steria Next, a consulting arm within the Sopra Steria Group, a significant oversight persists regarding accountability in decision-making processes influenced by AI. The report emphasizes that large organizations that have spent the past decade mastering data governance have largely neglected the equally crucial aspect of decision-making governance.

The Alarming Statistics

The report highlights several crucial statistics that underline the urgency of the situation:

• - Ownership and Accountability: Less than 20% of decisions impacted or executed by AI have a clearly identified owner, which raises concerns about who is ultimately responsible for the outcomes.
• - Traceability of Decisions: A mere 25% of AI-driven decisions that are currently in production are traceable and replayable from start to finish. This lack of traceability can lead to severe implications if any of these automated decisions lead to unfavorable outcomes.
• - Impact of Automated Failures: The report notes that failures in automated decision-making can disproportionately affect operations, customers, or revenue—potentially impacting ten to one hundred times more than traditional IT incidents.

What is AI Sovereignty?

AI sovereignty extends beyond knowing where data is stored or where machine learning models are run; it encompasses the ability to demonstrate control over, explain, and arbitrate AI-driven decisions. This shift from infrastructure dependency to cognitive dependency presents a new challenge for organizations looking to adapt.

In the publication, Sopra Steria Next introduces a key concept: a five-layer 'sovereignty stack'. Each layer—data, models, platforms, APIs, and decisions—raises critical questions regarding operation, replaceability, and accountability. For instance, who operates a given model, who can step in if needed, and who bears ultimate accountability for the decisions made?

The Call for Governance

Valérie Thiblet, Partner of Data and AI at Sopra Steria Next, articulates a crucial concern, stating, "The new question we now have to ask concerns decisions that AI has already taken without anyone being accountable for them. A system that cannot be stopped in real time is not governed; it is merely observed. And what cannot be governed cannot be industrialized."

One of the pressing issues identified in the report is the advent of Shadow AI—unauthorized AI deployments within organizations that circumvent established governance protocols. This trend raises significant concerns about diminishing traceability and the potential for financial and operational risk that leaders may not even recognize until it is too late.

Additionally, the report mentions model governance, often treated as an optional layer, but necessary for ensuring reliability and adherence to standards. As regulatory frameworks like the AI Act emerge, organizations may face risks not from deliberate non-compliance but from inadvertent failures due to a lack of preparedness.

Conclusion

The essence of AI governance cannot be overstated. As AI systems increasingly move from providing recommendations to executing decisions autonomously, establishing a robust framework that holds individuals accountable will be paramount for any organization aiming to leverage AI responsibly. The findings of Sopra Steria Next must serve as a wake-up call for leaders in organizations to re-evaluate their governance strategies—ensuring that every layer of their AI's deployment is not only effective but also accountable. Without these measures in place, businesses risk not just operational inefficiencies, but potentially catastrophic failures that could undermine customer trust and organizational integrity.