How to Start with AI Governance: A Guide for ESG and Compliance Leads

What does good AI governance look like?

AI governance is the set of roles, policies, processes and controls an organisation uses to manage the risks of AI and realise its benefits. Many organisations have agreed principles for responsible AI, but fewer have turned them into day-to-day practice. The World Economic Forum and Accenture found that less than 1% of organisations have fully operationalised responsible AI in a comprehensive and anticipatory way (WEF and Accenture, 2025). In PwC’s 2025 US Responsible AI survey, half of respondents said their biggest challenge was turning principles into scalable, repeatable processes (PwC, 2025).

Three frameworks are useful reference points:

• NIST AI Risk Management Framework (AI RMF 1.0, 2023): voluntary guidance from the US National Institute of Standards and Technology, organised around four functions: Govern, Map, Measure and Manage.
• ISO/IEC 42001:2023: the international standard for an AI management system. An accredited body can certify it, in the same way as ISO 27001 for information security.
• The EU AI Act (Regulation (EU) 2024/1689): the first comprehensive law on AI. It sets obligations by risk category and also applies to organisations outside the EU when the output of their AI systems is used in the EU.

The framework presented at the summit, based on BCG’s Sustainability in Private Markets report (October 2026), groups the elements of AI governance into five building blocks:

1. Strategy and policy: principles, risk appetite and approved use cases.
2. Governance: ownership, decision rights, accountability and KPIs.
3. Processes and controls: approval, testing, monitoring and human oversight.
4. Technology and tools: secure architecture, data controls and vendor standards.
5. Culture and capabilities: training, responsibilities and guidelines for adoption.

The risks that AI governance addresses fall into four areas: strategic risks, such as fragmented use cases; information security risks, such as confidential data being entered into AI tools; legal and compliance risks, such as privacy, intellectual property and regulation; and operational risks, such as inaccurate output and insufficient human oversight. In McKinsey’s 2026 AI Trust Maturity Survey of around 500 organisations, inaccuracy (74%) and cybersecurity (72%) were the risks most often rated as highly relevant (McKinsey, 2026). McKinsey also notes that AI agents add a further type of risk: in addition to producing incorrect content, they can take incorrect actions.

The research also links AI governance to business results. McKinsey found that organisations investing more in responsible AI report higher maturity and are more likely to see AI contribute more than 5% of EBIT. In PwC’s survey, 58% of respondents said responsible AI practices improve return on investment and efficiency. AI governance should therefore be seen as a means of enabling the safe expansion of AI use rather than as a restriction on it.

‍

Who should own AI governance?

The first step is to name one person who is accountable for AI governance. McKinsey’s research shows that ownership makes a measurable difference: organisations with explicit ownership for responsible AI score an average of 2.6 on its four-level maturity scale, compared with 1.8 for organisations without a clearly accountable function. As discussed at the summit, ownership sits with the sustainability team in some organisations and with compliance, risk, legal or data teams in others. The specific function is less important than having a named person with the authority to set rules and the support of senior leadership.

The owner is accountable for AI governance but does not carry out every task. PwC recommends applying the three lines of defence model to AI:

• First line: IT, data and business teams build and operate AI systems responsibly.
• Second line: governance and oversight functions set the rules, review AI use and report on it. ESG and compliance leads usually sit in this line.
• Third line: internal audit or external assurance providers check that the controls work.

The board should receive regular reporting on AI, as it does for cyber security or climate risk. Investors can apply the same structure at two levels: to their own use of AI and to the AI use of each portfolio company.

The owner’s first tasks are to:

• Agree principles and risk appetite: set out what AI may be used for, which uses need approval and which are not allowed.
• Approve tools: provide enterprise AI licences, which are easier to control than personal accounts. A speaker at the summit recommended blocking common personal AI tools on the company network once enterprise licences are available.
• Review information security: check who has access to which documents before rolling out enterprise AI, because AI tools can retrieve any file that a user is able to open.
• Adopt a short AI policy: set out these rules and who to contact with questions. Groups and investors can provide a policy template for subsidiaries or portfolio companies to adapt.

‍

How should organisations map and classify their AI use?

The second step is to build an inventory of every AI system the organisation uses. This should include AI features inside vendor software and tools that employees use without formal approval. Because more than half of employees say they do not disclose their use of AI, a short anonymous survey can help identify these tools. Where relevant, the inventory should also cover subsidiaries or portfolio companies. For each system, the inventory should record:

• its purpose, owner, users and vendor;
• the data it uses, including any personal or confidential data;
• whether it affects decisions about people; and
• whether it can take actions without human approval, such as sending messages, changing records or making payments, and which systems it can access.

Each system should then be classified using the four risk categories of the EU AI Act, as summarised by the Future of Life Institute (2026):

• Unacceptable risk (prohibited): practices such as social scoring, manipulative techniques that cause significant harm, and inferring emotions in the workplace except for medical or safety reasons.
• High risk: AI used in the eight areas listed in Annex III of the Act, including employment and workers management (such as screening job applications or evaluating performance), access to essential services (such as credit scoring and life and health insurance pricing), education, biometrics and critical infrastructure. AI that is a safety component of regulated products, such as machinery, is also high risk.
• Limited risk: systems with transparency obligations, such as chatbots, which must tell people they are interacting with AI, and AI-generated content such as deepfakes.
• Minimal risk: most other AI applications, which have no specific obligations beyond AI literacy.

For most sustainability teams, the classification is straightforward. Collecting and validating ESG data, calculating emissions, analysing trends and drafting report narratives are generally minimal risk. A chatbot that answers supplier or employee questions is limited risk. AI that screens job applicants or evaluates employee performance is high risk, even when its output is also used for social metrics.

Finally, organisations should record their role for each system. A provider develops an AI system and places it on the market under its own name, while a deployer uses an AI system in its business. Most organisations are deployers of the AI tools they buy, and the obligations differ for each role. Organisations that are not subject to the EU AI Act can still use its risk categories to decide where to focus their controls.

Which controls should organisations put in place?

The third step is to introduce controls that are proportionate to the risk of each use. A speaker at the summit described an approach that works across a group of businesses: a central tracker records and categorises each AI use case and applies a standard set of controls to each category. Specialist advice is reserved for customer-facing uses that may be high risk.

• Prohibited uses: discontinue them. These prohibitions have applied since 2 February 2025.
• Minimal risk uses: use approved tools, follow the AI policy and have a person review outputs before they are shared or relied on.
• Limited risk uses: inform people when they are interacting with AI or viewing AI-generated content. These transparency obligations apply from 2 August 2026.
• High-risk uses (as a deployer): use the system in line with the provider’s instructions, assign human oversight to trained people, monitor the system and report serious incidents to the provider, keep the logs it generates for at least six months, and inform workers before using it in the workplace. These obligations apply from 2 December 2027.

What additional controls do AI agents need?

Because AI agents can take actions as well as produce content, they need controls on which actions they are allowed to take. Deloitte describes mature governance for agentic AI as having three elements: clear rules on which decisions an agent can make independently and which require human approval, real-time monitoring that identifies unusual behaviour, and audit trails that record each action an agent takes. Guidance from the AI developers follows similar lines:

• Anthropic’s framework for developing safe and trustworthy agents (2025) is based on five principles: keeping humans in control, aligning agents with human values, securing agents’ interactions, transparency and privacy. In its examples, users decide which actions an agent may take without approval. For example, an agent may be allowed to read a calendar but must ask before sending an invitation.
• OpenAI’s white paper on governing agentic AI systems (2023) proposes seven practices, including limiting the actions an agent can take, making its reasoning visible, monitoring it automatically, being able to attribute actions to it, and being able to interrupt or stop it.

A prudent approach is to deploy agents first for lower-risk tasks such as reading and drafting, to require human approval for any action with an external effect (such as sending, paying, deleting or changing records), to keep a log of each agent’s actions and to ensure that a named person can stop an agent when required.

Vendor contracts should be reviewed at the same time. Organisations should ask AI vendors how they handle company data, what documentation they provide and, for high-risk systems, how they will support the organisation’s obligations as a deployer.

How should organisations build skills, measure and report on progress?

The fourth step is to ensure that employees have the knowledge to use AI responsibly. Research indicates that this remains an area of weakness for many organisations. Nearly 60% of respondents in McKinsey’s survey cited knowledge and training gaps as the main barrier to responsible AI, up from about 50% a year earlier. When EY asked C-suite leaders to identify the appropriate controls for five AI risks, 12% answered correctly. Under Article 4 of the EU AI Act, providers and deployers must take measures to support the AI literacy of staff who use AI systems. The July 2026 amendment softened the wording of this duty but kept it in place (Digital Workplace Group, 2026). This requires training tailored to each group of users, covering what AI can and cannot do, how to check its output and which tools are approved.

AI governance can then be measured and reported in the same way as other ESG topics. Useful indicators include:

• the number of AI systems in the inventory, by risk category;
• whether an AI policy is in place and when it was last reviewed;
• the share of staff who have completed AI training;
• the number of AI agents in use and the actions they are allowed to take;
• the number of AI vendors assessed against the organisation’s standards; and
• AI incidents, how they were handled and what changed as a result.

Collecting these indicators alongside other ESG data, across sites, subsidiaries or portfolio companies, gives the board and investors a consistent view. One investor at the summit is adding AI governance questions to its annual ESG questionnaire for portfolio companies, and corporate groups can do the same for their sites and subsidiaries.

Organisations should also have a process for handling AI incidents. McKinsey found that around 8% of organisations reported AI incidents, a share that has remained stable, but almost 60% of those affected rated their organisation’s response as only satisfactory or negative. The inventory and controls should be reviewed at least once a year and whenever a new AI tool or use case is introduced. As with GDPR, the most effective approach is proportionate, documented and clearly owned.

Key questions for boards and investment committees

The following questions can help boards and investment committees assess the current state of AI governance:

• Does the organisation have a complete record of the AI systems in use, including AI built into purchased software?
• Who is accountable for AI governance, and does that person have the authority to stop or change an AI use?
• Which AI uses would be classified as high risk under the EU AI Act, and is the organisation the provider or the deployer in each case?
• Are any AI systems or agents able to take actions without human approval?
• Can the organisation provide investors, customers or auditors with evidence that its controls are effective?
• Do AI vendor contracts address data handling, documentation and support for the organisation’s obligations?

Frequently asked questions

What is AI governance?

AI governance is the set of roles, policies, processes and controls an organisation uses to manage the risks of AI and capture its value. It covers who owns AI decisions, which uses are allowed, how AI systems are tested and monitored, and how progress is reported to the board and investors.

Who should be responsible for AI governance?

Responsibility should sit with one named owner who has senior support, often in sustainability, compliance, risk, legal or data. McKinsey’s 2026 research found that organisations with explicit ownership for responsible AI reach materially higher maturity than those without. The owner typically acts as the second line, setting rules and reporting, while IT, data and business teams run AI day to day.

How do the NIST AI RMF, ISO/IEC 42001 and the EU AI Act relate to each other?

The three are designed for different purposes and can be used together. The EU AI Act sets legal obligations, the NIST AI RMF offers a flexible structure for managing AI risk, and ISO/IEC 42001 provides a certifiable management system for organisations that want external assurance. Many organisations start with the EU AI Act categories and use the other two to structure their programme.

When do the EU AI Act obligations apply?

The Act applies in stages. The Digital Omnibus on AI, provisionally agreed on 7 May 2026 and in force since 27 July 2026 as Regulation (EU) 2026/1744, deferred the high-risk deadlines (Gibson Dunn; Future of Life Institute).

• 2 February 2025: prohibited practices and AI literacy.
• 2 August 2025: obligations for providers of general-purpose AI models.
• 2 August 2026: transparency obligations under Article 50.
• 2 December 2027: high-risk AI systems listed in Annex III.
• 2 August 2028: high-risk AI embedded in regulated products (Annex I).

Does the EU AI Act apply to organisations outside the EU?

Yes. It applies to providers that place AI systems on the EU market and to providers and deployers outside the EU where the output of their AI system is used in the EU.

What are the penalties under the EU AI Act?

Using prohibited practices can lead to fines of up to EUR 35 million or 7% of worldwide annual turnover. Breaches of most other obligations, including deployer and transparency obligations, can lead to fines of up to EUR 15 million or 3% of turnover (European Commission AI Act Service Desk, Article 99).

Are sustainability uses of AI high risk?

In most cases, no. ESG data collection, emissions calculations and report drafting are generally minimal risk. Uses in recruitment, performance evaluation, credit and insurance are more likely to be high risk.

Can KEY ESG support AI governance?

Yes. KEY ESG can support AI governance by adding AI governance questions to an organisation’s ESG data collection and reviewing the responses across sites, subsidiaries or portfolio companies.

How can KEY ESG help?

KEY ESG supports organisations with AI governance alongside their existing ESG reporting. Corporates and investors can add AI governance questions to their ESG data collection, covering areas such as the AI inventory, risk classification, policies, ownership, AI agents and AI literacy measures, and review the responses across sites, subsidiaries or portfolio companies in one place. To discuss how this would apply to your organisation, request a demo.