NewGenApps

What Is an AI Audit? What an Independent Audit of a Production AI System Covers, and Which Kind You Need

Last reviewed: November 2026.

In short. "AI audit" is used for four different things: a review of an organisation's AI governance, an assessment against a specific law or standard, the use of AI tools inside financial auditing, and an independent check of whether a particular AI system works. Each answers a different question. If you already run an AI system, or have a pilot that is not moving, the useful one is usually the last: an evidence-based check, done by someone other than the builder, of whether the system works on your data. It ends in findings and a recommendation, not a certificate.

Someone has told you to "get the AI audited": a board, a customer, an insurer, or your own unease about a pilot nobody can vouch for. The first task is to find out which question is being asked, because the same phrase covers work with very different scope, cost and output.

What is an AI audit?

An AI audit is a structured examination of an AI system, or of the organisation that builds and runs it, against a stated standard, with written findings. The examiner gathers evidence, compares it with the standard, and reports what holds and what does not.

Everything turns on the words "against a stated standard". The standard can be a law, a published framework, or criteria agreed at the start for one system. An audit with no agreed standard is an opinion. An audit with one tells you whether the thing you looked at meets it, and nothing more than that.

What are the different kinds of AI audit?

Kind The question it answers Who usually asks What it examines
Governance and management review Does the organisation have the roles, policies and risk processes to run AI responsibly? Board, risk committee, procurement Policies, ownership, risk registers, documented processes
Legal or standards assessment Does this system, or this organisation, meet a particular law or standard? Legal, compliance, a regulator or customer Documents and controls mapped to the requirement named
AI in auditing Which AI software helps auditors and accountants do their own work? Finance and audit teams Software features, not your AI system
Independent audit of a running system Does this system work on our data, and can anyone show that it does? CIO, CTO, the budget owner The system itself: data, evaluation, deployment, monitoring, ownership

The last row is the focus of this page. The first two are real and sometimes mandatory, but they mostly examine documents, controls and process. A system can sit inside a well-documented governance process and still be producing wrong answers on stale data, because reviewing the documents does not run the system.

The third kind is a search-result neighbour, not a relative. If you are looking for audit software for an accounting team, this is the wrong page.

One public reference for the governance side is the NIST AI Risk Management Framework, which NIST describes as released on 26 January 2023 and intended for voluntary use. A framework like this gives an organisation a vocabulary and a way to organise risk work. Which laws and standards apply to your system depends on where you operate and what the system does, and that is a question for your legal advisers, not for this article.

What does an audit of a working AI system examine?

An independent audit of a running system looks at five things, at the level of what is checked rather than how:

The same checks are described for a system you are about to ship in how you know an AI system works in production.

Why should the auditor not be the builder?

Because a builder tends to test the cases they expected. The person who wrote the prompts, picked the data and tuned the thresholds knows where the system is strong, and checks those places. An independent examiner has no stake in the answer and no habit of avoiding the weak spots.

Independence is a property of who does the checking, not of how thorough the checking is. A thorough review by the team that built the system is still a self-review. It is useful, and it is not an audit in the sense this page uses. The same rule runs through our own work: whoever builds something does not sign it off.

What does an AI audit deliver, and what does it not prove?

A useful audit leaves you with a written report and a decision. At the category level that means:

An audit also has limits that its report should state:

Which kind of audit do you need first?

Your situation Start with Why
A board, customer or insurer asks for evidence of AI governance A governance review, plus legal advice on what applies The request is about process and documents
A specific law or standard has been named An assessment against that text, with your legal advisers Only the named text defines "pass"
A live AI system, and no one can show it works An independent audit of the running system The question is whether it works on your data
A pilot that has stalled The same check, then a decision to recover, re-scope or retire The cause of a stall is often visible in the data and the evaluation
Several AI ideas and no clear order A readiness or opportunity assessment Nothing is running yet to examine

If the earlier question is whether to bring in outside help at all, when to bring in an AI implementation partner covers that decision.

How is an AI audit different from an AI opportunity audit or a proof of concept?

They answer different questions at different points.

If you are unsure which applies, three questions place you on the ladder, and an AI production-readiness checklist is a self-check you can run first.

Frequently asked questions

Is an AI audit the same as an AI compliance assessment?

No. A compliance assessment measures a system or organisation against a named law or standard. An audit of a running system measures whether it works, against criteria agreed at the start: what correct output, fresh data and acceptable risk mean for that system. The two can be done together, but passing one does not imply the other, and neither is a substitute for legal advice.

How long does an audit of an AI system take?

It depends on scope. A narrow, fixed-scope independent check of one system can take days; a broader review across several systems can take weeks. We would rather agree the scope in a short conversation than quote a duration without one.

Can we audit our own AI system?

You can review it, and you should. A self-review is not independent, though, for the reason above. If the result will be shown to a board, a customer or an insurer, it carries more weight when someone outside the team produced it.

What do we need to provide?

Typically a plain description of what the system does and who uses its output, read-only access to the system, its logs and a sample of the data it works on, whatever evaluation or test material exists (or a clear statement that there is none, which is itself a finding), and one person who can say how it runs today rather than how it is documented.

Talk it through

If you are not sure which kind of audit you are being asked for, or whether a live system or stalled pilot would stand up to one, we can work through it in a free 30-minute AI working session.

NewGenApps: production AI, proven. Stay a step ahead, always.

Book an AI working session