AI is becoming increasingly embedded in everyday business operations, changing how employees work, how organizations deliver services, and how decisions are made. As the technology evolves, organizations navigate new opportunities and risks, often while still figuring out how best to adopt and manage it. This creates a growing need for effective AI governance. While many organizations have established general policies and oversight structures, they are still working to translate governance principles into practice. Research from the American Arbitration Association® (AAA®) and the AAA-ICDR Institute™, published in From Principles to Practice: A Benchmark Study in AI Governance, found that 87% of surveyed senior leaders reported having some form of AI governance in place, but only 22% considered their systems effective. Beyond establishing policies, organizations must determine who makes decisions, how risks are evaluated, what safeguards are needed, and how those safeguards are maintained over time.
At the AAA, we approach AI governance from two perspectives: as an organization using AI internally and as a developer of AI-powered dispute resolution tools. Building on our existing principles and policies, we have developed a governance structure to review, approve, and monitor AI across the organization. Along the way, we've learned that effective governance depends not just on the policies an organization establishes, but on how those policies translate into everyday decisions.
Our experience has highlighted eight practical takeaways for organizations looking to make AI governance more effective, whether they're building a new program or strengthening an existing one.
1. Start With What You Already Have
AI governance doesn't need to start from scratch. Existing policies covering privacy, security, vendor management, and risk can provide a foundation for AI-specific policies, but organizations should work to identify where AI introduces new considerations. At the AAA, we built on existing principles, including the AAAi Standards for AI in ADR, and developed a charter, policy, framework, and review process to put those principles into practice. One of the first questions we asked our team was where existing safeguards were sufficient and where additional oversight would be needed.
2. Make Roles and Responsibilities Clear
Who approves an AI system, who evaluates its risks, and who is responsible once it's in use? Without clearly defined roles, decisions can stall, responsibilities can overlap, workload expectations become muddied, and accountability can become unclear. The AAA's governance structure separates technical development and testing from risk assessment and approval and establishes an AI Governance Committee to determine whether risks are acceptable. Each system also has a designated owner responsible for ongoing review needs.
3. Know Where AI Is Being Used and How It Is Entering Your Organization
AI use is often more widespread than an organization may realize and may not be limited to systems deliberately developed or purchased for employee use. Vendors may introduce AI features into existing software, employees may experiment with general-purpose tools, or teams may develop their own applications using low-code platforms. Understanding these different entry points is essential to identifying and managing AI-related risks. To account for these different entry points, the AAA established a single intake process covering internally developed systems, third-party tools, and employee-built applications. Maintaining an inventory of these systems, their owners, and their review status helps organizations understand where AI is being used and identify potential risks.
4. Let Risk Determine the Depth of Review
Different tools will likely require different levels of review. Applying the same level of scrutiny to every AI tool can create unnecessary delays or leave higher-risk systems without sufficient oversight. The AAA uses a consistent review process that adjusts the level of scrutiny based on a system's risks, potential impact, and intended use. A tool summarizing internal documents, for example, presents different risks than one supporting decisions in an arbitration case. Organizations should be able to explain not only how they reviewed a system, but why that level of review was appropriate.
5. Verify What You Are Told
Vendor descriptions, marketing claims, and internal documentation don't always tell the full story about how an AI system works or handles data. The AAA's review process looks beyond these assurances to underlying contracts, data-processing terms, technical documentation, and system functionality. Before approving a tool, organizations should verify claims about data retention, model training, security, and performance rather than accepting them at face value.
6. Make Human Oversight Meaningful
Having a human review AI-generated work is an important safeguard, but its effectiveness depends on what that person can reasonably be expected to catch. Reviewers may be well equipped to identify errors in familiar material but less likely to notice inconsistencies, calculation errors, or missing information. This distinction helped shape the AAA's approach to the AI Arbitrator, where parties review AI-generated summaries of their submissions, and a human arbitrator reviews the analysis and issues the final award. As part of our evaluation, we considered whether a human reviewer could reasonably identify an error, whether it could affect the outcome, and what additional safeguards may be needed to protect fairness and neutrality in the process.
7. Keep Reviewing Systems in Place After Approval
AI governance cannot end when a system launches. Models change, vendors update their products, and real-world use can reveal issues that initial testing missed. The AAA monitors systems after deployment and validates them again when underlying models change, using lessons from ongoing performance to inform improvements. Organizations should establish clear triggers for reassessment and assign responsibility for monitoring systems throughout their use.
8. Make Governance Easy to Follow
A governance process only works if people understand how to use it. Complicated approval requirements and unclear policies can encourage employees to work around the system. The AAA emphasizes plain-language guidance, consistent terminology, and practical instructions for employees using approved tools. Organizations should make it straightforward to understand when approval is needed, how to obtain it, and what responsible use looks like.
From Principles to Practice
Effective AI governance isn't about eliminating every risk or anticipating every possible development. It requires clear accountability, informed decision-making, and oversight that continues as systems evolve. Whether an organization is establishing its first governance program or refining an existing one, the goal is the same: to create processes that support responsible AI use not just at the point of approval, but throughout a system's lifecycle.