AI governance has largely been treated as a secondary concern— addressed after systems are deployed, framed as policy rather than design and delegated away from core decision-making. That model is breaking down as AI systems begin to influence judgment, outcomes and institutional legitimacy.
At the American Arbitration Association (AAA), those pressures are not theoretical. Neutrality, procedural integrity and trust are foundational to the organization’s role in dispute resolution, shaping how AI is designed, governed and validated in practice.
We spoke with Diana Didia, Executive Vice President and Chief Information & Innovation Officer, about what it means to treat AI governance as architecture rather than oversight—and why that shift increasingly belongs at the executive and board level.
Why Governance Followed Deployment—and Why That No Longer Works
The lag between AI deployment and governance is not surprising. Generative AI capabilities are still evolving and organizations have been forced to build systems and invent governance in parallel. There is no universal governance template; frameworks must reflect institutional mission, risk tolerance and use case specificity.
Early experimentation—pilots, narrow deployments and beta tools—was necessary to surface real risks, from hallucinations to privacy exposure. In lower-risk environments, retrofitting governance was manageable.
That logic collapses as stakes rise. Once AI systems move beyond productivity and begin to influence outcomes, governance can no longer be layered on. For high-risk applications, Didia argues, governance must move upstream—embedded directly into system architecture and design.
Designing AI for Trust, Not Just Scale
The American Arbitration Association’s operating context fundamentally reshapes the AI design problem. In justice-adjacent environments, trust is not inferred from efficiency. It is earned through explainability, transparency and continuous assurance of rigor.
With the AAA’s AI Arbitrator, governance is inseparable from system design. Models are constrained to defined corpora. Prompts are carefully engineered. Techniques such as retrieval-augmented generation are used to ensure legal reasoning meets a high bar of consistency and accuracy. Performance is continuously monitored rather than assumed.
Equally important is how the system communicates its work. AI cannot function as a black box where legitimacy matters. Users are shown how their submissions were interpreted—what claims were identified, which issues surfaced and how the dispute was framed. These deliberate “read-outs” are not cosmetic features; they are trust mechanisms that allow parties to validate that the system understood them correctly. Human oversight is not symbolic.
Human arbitrators validate AI outputs, serving as a core governance control. Interfaces are designed so reviewers can assess outcomes holistically without missing critical details. In this model, governance is as much about experience design as it is about technical constraint.
From Principles to Infrastructure
Like many organizations, the AAA began with AI principles—ethical use, transparency, confidentiality.
Those principles evolved into policies, cross-functional governance committees and risk assessments, following a familiar cybersecurity-inspired trajectory.
The harder work lies in operationalization.
Governance becomes meaningful only when principles translate into procedures, processes and technical controls: where humans remain in the loop, how performance is scored, what thresholds trigger review and how remediation occurs. Model evolution exposes the gap. As underlying models are upgraded, operationalized systems must be revalidated—yet many organizations lack repeatable processes or even visibility into which models their tools are running on.
At the AAA, governance infrastructure includes continuous performance scoring, monitoring for degradation and routing issues to accountable teams. This is governance as system design, not policy documentation.
That operational layer is also where new roles are emerging. Monitoring, validation, lifecycle management and performance testing of AI systems are becoming core functions—not side responsibilities.
Scaling Judgment Without Replacing It
One of the most consequential design questions concerns human judgment. Didia draws a clear distinction between AI as a productivity aid and AI as an analytical actor.
In many legal tools—document summarization, Q&A—the arbitrator retains full judgment. AI assists without steering outcomes. The AI Arbitrator goes further, performing substantive legal analysis and proposing outcomes that human arbitrators must validate.
That influence is intentional and carefully governed. Human decision-makers are fallible; they experience fatigue, bias and time pressure. AI brings stamina, consistency and scale—reading every line, checking internal coherence and surfacing patterns that might otherwise be missed.
The objective is not replacement, but augmentation. By curating analysis in ways that invite interrogation and override, the system scales human judgment rather than displacing it. When governed properly, this combination can produce outcomes that are not only more efficient, but potentially fairer and more defensible.
When AI Governance Becomes Board Oversight
As AI systems move from experimentation into adjudicative and operational reliance, governance can no longer sit solely with technology, legal or innovation teams. The design choices embedded in AI systems—how judgment is structured, how performance is monitored, how errors are surfaced and corrected—carry institutional consequences that boards are ultimately accountable for.
In environments where trust underpins legitimacy, those consequences extend beyond operational risk into defensibility, reputation and authority. The AAA’s approach illustrates that AI governance is not merely a compliance function; it is a form of organizational oversight that boards will increasingly need to understand, interrogate and resource.
The question for leadership is no longer whether AI is being used responsibly. It is whether executives and boards have visibility into how judgment is being shaped, scaled and constrained inside the systems their institutions now rely on.
This article was previously published by Corporate Counsel Business Journal, which can be found here.