The fast answer: evaluating an AI vendor means asking for evidence instead of promises. Before you sign, you want four things in writing: independent test results you can check, clear terms on how your data is used and kept, a commitment to monitor the system after go-live, and a way to verify every safety claim the vendor makes without taking their word for it. The rest of this guide turns that into a working process, a question list you can send to vendors, and the contract terms that make the answers stick.

AI due diligence differs from ordinary software review because an AI system can produce a confident wrong answer, be talked out of its own rules, leak data, or change behaviour after a model update the vendor pushes on a Tuesday. You find those risks by testing behaviour, and you control them with contract terms that say what happens when behaviour drifts.

Why AI vendor due diligence is now a formal expectation

Buying AI used to be treated as a software purchase with an extra paragraph on data. Public buyers and lawmakers on both sides of the Atlantic have changed that.

US federal buyers: OMB memo M-25-22

On April 3, 2025, the White House Office of Management and Budget issued memorandum M-25-22, Driving Efficient Acquisition of Artificial Intelligence in Government. It applies to contracts awarded under solicitations issued 180 days or more after the memo, and to renewal options exercised after that point. Section 4(d) says agencies "must include terms that address" a list of topics in contracts for AI systems and services. Among them:

• Ongoing performance monitoring. Contracts must let the agency regularly monitor and evaluate the system's performance, risks and effectiveness, with quarterly or biannual reviews given as examples.
• Independent testing. Agencies should test with their own datasets that "should not be accessible to the vendor", vendors must provide "the access and time necessary for agencies to complete independent evaluation", and testing results should be "detailed enough for the testing to be independently verified or reproduced, if practicable."
• Protection against lock-in. Terms can include knowledge transfer, "data and model portability", and transparency on licensing and pricing.
• Limits on training with your data. Section 3(e) requires contracts to "permanently prohibit the use of nonpublic inputted agency data and outputted results" to further train publicly or commercially available AI, unless the agency explicitly consents.
• Notice before changes. Agencies should consider requiring vendors to notify them before new AI enhancements are integrated, and are encouraged to require vendors to meet performance standards before deploying new versions and to roll back if needed.

A second memo, OMB M-26-04 of December 11, 2025, says it "complements" M-25-22. For large language model contracts it lists information agencies should request from vendors: an acceptable use policy, model, system or data cards, end-user resources, and a mechanism for end-user feedback. It also lets agencies ask for more, such as evaluation results.

These memos bind federal agencies, not private companies. They are still the most detailed public statement of what a sophisticated AI buyer asks for, and private procurement teams can borrow their structure directly.

EU deployers: what the AI Act asks of the buyer

The EU AI Act, Regulation (EU) 2024/1689, calls the organisation that uses an AI system under its own authority the "deployer". If you buy a high-risk AI system and put it to work, Article 26 places duties on you, separate from the duties on the vendor. In the consolidated text published on the European Commission's AI Act Service Desk, Article 26 says:

• Deployers "shall take appropriate technical and organisational measures to ensure they use such systems in accordance with the instructions for use accompanying the systems" (Article 26(1)).
• Deployers "shall assign human oversight to natural persons who have the necessary competence, training and authority" (Article 26(2)).
• Deployers "shall monitor the operation of the high-risk AI system on the basis of the instructions for use", and if they have reason to think the system presents a risk, they must inform the provider or distributor and the relevant market surveillance authority and suspend use (Article 26(5)).
• Deployers "shall keep the logs automatically generated by that high-risk AI system" to the extent those logs are under their control, for at least six months unless other law says otherwise (Article 26(6)).
• Deployers of Annex III systems that make or assist in decisions about people "shall inform the natural persons that they are subject to the use of the high-risk AI system" (Article 26(11)).

Each of those duties depends on something you can only get from the vendor. You cannot follow instructions for use you never received, keep logs the system does not export, or monitor behaviour you have no way to measure. Article 13 is the vendor side of the same bargain: high-risk systems must come with instructions for use that include, among other things, "the level of accuracy, including its metrics, robustness and cybersecurity" (Article 13(3)(b)(ii)). Due diligence is where you confirm those materials exist and are good enough for you to do your part.

Timing matters. Under the Digital Omnibus amendments, Article 113 now applies the high-risk rules from 2 December 2027 for stand-alone systems listed in Annex III and from 2 August 2028 for AI embedded in products covered by Annex I. Contracts you sign today will often still be running on those dates, so it makes sense to write the obligations in now. Separately, Article 4 asks providers and deployers of AI systems generally, not only high-risk ones, to "take measures to support the development of AI literacy" of the staff who operate and use them. For the wider legal picture, see our complete guide to the EU AI Act and US AI laws.

The voluntary frameworks point the same way

The NIST AI Risk Management Framework core gives third-party AI its own category. GOVERN 6.1 reads "Policies and procedures are in place that address AI risks associated with third-party entities", GOVERN 6.2 reads "Contingency processes are in place to handle failures or incidents in third-party data or AI systems deemed to be high-risk", and MANAGE 3.1 says risks from third-party resources should be "regularly monitored, and risk controls are applied and documented." If your organisation already aligns with the NIST AI RMF or runs an ISO/IEC 42001 management system, vendor due diligence is how you meet those expectations for the AI you buy. Our guide to NIST AI RMF, ISO/IEC 42001 and related standards covers how the frameworks fit together.

The six areas to evaluate

Every AI vendor review comes down to six areas.

1. What the system is and what it is for

Start with scope. Which models sit underneath the product, and is the vendor their developer or a reseller wrapping someone else's? What intended purpose does the vendor document, and does your planned use fall inside it? Under the EU AI Act, intended purpose is the first item the instructions for use must state, and using a system outside it means taking on risk the vendor never tested for.

2. Evidence of safe behaviour

This is the area most buyers handle worst, because it is the hardest to ask about in a questionnaire. "Do you test for bias?" gets a yes from everyone. Better questions ask for the method and the result: what was tested, against which scenarios, with what pass rate, by whom, and when. The behavioural risks worth covering line up with the seven pillars of the A.E.G.I.S.™ framework:

• Hallucination Integrity: does the system state falsehoods as facts, especially under pressure, ambiguity or domain-specific questioning?
• Guardrail Resistance: can it be manipulated into unsafe behaviour through jailbreak attempts or prompt injection?
• Data Handling and Privacy: what does it collect, retain and leak, including personal data?
• Ethical Alignment: does it behave appropriately with vulnerable users and sensitive topics?
• Autonomy and Control: how far can it act on its own, and can a human stop it?
• Adversarial Robustness: how does it hold up against a motivated attacker?
• Explainability and Transparency: can its behaviour be understood and audited after the fact?

For how each of these is actually tested, see how AI systems are tested for safety. The key distinction for a buyer is between results the vendor produced about itself and results someone independent produced. Internal testing has value, but it is the vendor grading its own homework. M-25-22 makes the same point from the buyer's side when it asks for testing detailed enough to be "independently verified or reproduced."

3. Data handling

Ask where your data goes, how long it stays, who can see it, and whether it trains anything. The federal standard in M-25-22 is a useful benchmark: vendors may collect and retain agency information only when "reasonably necessary" for the contract, and nonpublic inputs and outputs may not be used to train public or commercial models without consent. Ask too whether the system's logs are available to you, since EU deployers of high-risk systems must keep the logs under their control.

4. Oversight and control

Find out what a human can do when the system is wrong. Can an operator review or override outputs, or pause the system? Article 26(2) assumes you will assign competent people to oversight; the vendor has to give them something to work with.

5. Change management and monitoring

AI products change underneath you: a new base model or a retrain, and the behaviour you approved is no longer the behaviour in production. Ask how the vendor tells you about material changes, whether you can test before a new version reaches your users, and whether a version can be rolled back. M-25-22 asks agencies to consider or require all three. Then ask what monitoring exists after go-live, and whether results are shared with you.

6. Incidents and accountability

Ask for the vendor's incident response process for AI-specific failures such as a harmful output or a successful jailbreak. EU deployers of high-risk systems have to inform the provider and the market surveillance authority when they believe a system presents a risk, so your contract should make sure the vendor will work with you when that happens. Our guide to documented AI incidents and what testing catches shows the failure patterns worth planning for.

A question list you can send to AI vendors

Copy these into your RFP. Each asks for something you can check.

Scope and purpose

1. Which models power the product, who develops each one, and which versions are in production today?
2. What is the documented intended purpose, and are there uses you advise against? Please share the instructions for use or equivalent documentation.
3. Do you publish model, system or data cards for the product? Please attach them.
4. What is your acceptable use policy for end users?

Testing and evidence

5. Which safety risks have you tested the product against (for example hallucination, jailbreaks and prompt injection, privacy leakage, behaviour with vulnerable users, autonomy limits, adversarial attack, explainability)? For each, what was the method and the result?
6. Were any of those tests performed by an independent party? Who, when, against which version, and can we verify the result ourselves?
7. Will you give us the access and time to run our own evaluation, using test data you do not see?
8. What are the documented accuracy, robustness and cybersecurity levels, and how were they measured?

Data

9. What data from our use do you collect and retain, for how long, and why is each item necessary?
10. Will our inputs or outputs be used to train or improve any model, yours or a third party's? Can that be contractually prohibited?
11. Which subprocessors, including model providers, receive our data?
12. Which logs does the system generate, and can we export and retain them?

Control, change and incidents

13. What human oversight and override controls are available to our operators?
14. How and when will you notify us before a material change to the model or its behaviour? Can we test a new version before it reaches our users, and can we roll back?
15. How do you monitor the product after deployment, and will you share monitoring results with us?
16. What is your incident response process for AI-specific failures, and what are your notification commitments to us?
17. If we leave, how do we get our data and any customisations out, and in what format?

Turning answers into contract terms

Questionnaire answers are representations. The contract is where they become obligations. M-25-22 gives a template for the topics, and private buyers can adapt it without much change.

• Performance and safety commitments. Write the tested behaviour into the agreement: the documented accuracy and safety levels, a periodic review schedule, and what happens if results fall below them.
• Right to evaluate. Reserve the right to run your own testing, or have an independent party test, with access and time from the vendor.
• Data use limits. Prohibit training on your inputs and outputs without written consent, limit retention to what is necessary, and name the subprocessors.
• Change notice. Require advance notice of material model changes and a test window before they reach production, with a rollback option.
• Logging and records. Require the system to generate and export the logs you need, and set how long the vendor keeps its own.
• Incident cooperation. Set notification timelines for AI incidents and an obligation to help you meet any reporting duty you have.
• Portability. Cover data and model portability, knowledge transfer and transparent licensing so you can leave if you need to.

M-26-04 adds a useful drafting technique: it tells agencies to identify certain requirements as "material to eligibility and payment under the contract." Making safety and transparency terms material gives them teeth, because breaching them becomes a basis for remedies rather than a footnote.

Where due diligence usually breaks down

Most failed AI vendor reviews fail in one of a few predictable ways.

The questionnaire is a security form with AI questions bolted on. A SOC 2 report, in the AICPA's own description, is an examination of controls at a service organization relevant to security, availability, processing integrity, confidentiality, or privacy. That is valuable, and it answers a different question from whether the model can be jailbroken or how often it invents facts. Keep information security and AI behaviour as separate sections with separate evidence.

Every answer is self-reported. The reviewer has no way to check the form or the attached policy, so the buyer either accepts claims it cannot verify or commissions its own testing.

Approval is treated as permanent. Six months after purchase the vendor has changed the model and nobody re-checked. Both M-25-22 and the NIST AI RMF treat monitoring as continuous for exactly this reason.

Evidence is not comparable across vendors. Three vendors describe their testing three different ways, so price and features decide by default.

Why a verifiable independent rating shortens the process

The last two problems are the expensive ones. When an AI system has been evaluated by an independent party against a standard framework, three things change for the buyer.

First, the evidence is independent. RateAI has no stake in a system's rating and its fees are tied to evaluation milestones, not outcomes. A system can fail certification; when it does, the developer receives a detailed findings report and a remediation path, and can re-enter evaluation once the issues are fixed.

Second, the evidence is comparable. Every A.E.G.I.S.™ evaluation scores the same seven pillars, and the scoring parameters are fixed by a scoring epoch so that evaluations stay comparable over time. Certified systems receive an A.E.G.I.S.™ scorecard with an Overall Safety Rating from A (Maximum Safety) to F (Dangerous), a grade for each pillar, and a certification seal from A-5 Guardian down to A-1 Unrestricted. Your reviewers can see in one place where a system is strong and where caution is warranted, and compare two vendors on the same scale.

Third, the evidence is verifiable and current. Every prompt case, validator vote and certification decision is chained with SHA-256 hashing in a write-once audit trail, so records cannot be altered or backdated. Anyone can check a RateAI certification in the public Safety Registry without logging in, and confirm that it is current and has not been downgraded. Certifications also carry a re-certification cycle, and material changes to a system trigger reassessment, which speaks directly to the change-management problem above.

A rating does not replace your due diligence. You still need answers on scope, contract terms, data flows and fit for your use case, and a RateAI certification is not a legal determination: it supports your compliance work, and regulatory decisions remain with the relevant authorities. RateAI's Pro and Enterprise tiers include compliance mapping to the EU AI Act, NIST AI RMF and ISO 42001. What a verifiable rating does is answer the evidence questions, numbers 5 to 8 on the list above, with something your team can check instead of trust. For a deeper look at how third-party certification works, read what AI safety certification is and how it works.

A practical due diligence workflow

1. Classify the use case. Who does the system affect, can it act on its own, and might it be high-risk under the EU AI Act? Higher stakes mean deeper review.
2. Send the question list early. Include it in the RFP so vendors who cannot answer drop out before you invest time in demos.
3. Check for independent evidence. Look the vendor's product up in any independent registry it claims to be listed in, and confirm the rating, date and version match what you are buying.
4. Test what remains. Where independent evidence is missing or does not cover your use case, run your own evaluation on data the vendor has not seen, or commission one.
5. Negotiate the terms. Convert the answers into contract obligations: performance, testing rights, data use, change notice, logging, incidents and portability.
6. Set the monitoring cadence. Agree how often you will review performance and what triggers a re-review, such as a model change or a reported incident.
7. Record the decision. Keep the evidence, your scoring and the reasons for approval.

If you are on the vendor side of this process, the same list tells you what buyers are going to ask. Having independent evidence ready before the questionnaire arrives shortens every sales cycle that follows. RateAI's certification and monitoring tiers describe how an engagement is scoped, and a free safety benchmark for one AI agent is available as a starting point.

Frequently asked questions

What is AI vendor due diligence?

It is the process of checking an AI vendor's system, data practices and controls before you buy, and keeping those checks current afterwards. Compared with ordinary software review, it adds testing of the system's behaviour, such as hallucination, jailbreak resistance and privacy leakage.

Is a SOC 2 report enough to approve an AI vendor?

Usually not on its own. A SOC 2 report examines a service organization's controls relevant to security, availability, processing integrity, confidentiality or privacy, so you still need separate evidence on how the AI model behaves, such as false answers, manipulation and leakage of data through outputs.

What does the EU AI Act require from companies that buy AI?

Organisations that use high-risk AI systems are "deployers" under Article 26, and must use the system according to its instructions, assign competent human oversight, monitor its operation, keep the logs under their control and, in some cases, inform the people affected. Those high-risk rules apply from 2 December 2027 for Annex III systems and 2 August 2028 for AI embedded in products covered by Annex I.

What contract terms should an AI purchase include?

At minimum: performance and safety commitments, a right to independent testing, limits on using your data for training, advance notice and testing of model changes, log access, incident cooperation and portability. OMB memo M-25-22 lists most of these as required topics for US federal AI contracts.

How does an independent AI safety rating help procurement?

It replaces self-reported claims with standardised results that a third party produced and that you can verify. An A.E.G.I.S.™ rating grades a system on the same seven pillars as every other rated system, so you can compare vendors and confirm the certification is current in the public Safety Registry.

Does a RateAI certification make an AI system legally compliant?

No certification can make that determination; formal regulatory decisions remain with the relevant authorities. RateAI certification supports compliance work, and its Pro and Enterprise tiers include compliance mapping to the EU AI Act, NIST AI RMF and ISO 42001.

Check a vendor's rating before you sign

The fastest due diligence question is whether the system has already been independently evaluated. Search the RateAI Safety Registry to see how certified AI systems are listed, with their A.E.G.I.S.™ rating and certification status, and ask your vendors whether their product is on it.