The fast answer: AI incidents are real, they are logged in public, and the log keeps growing. The AI Incident Database added 148 new incident IDs in the three months from May to July 2026 alone. Read the records one by one and a pattern shows up quickly: most failures belong to a small number of repeatable types, such as a chatbot inventing a policy, a guardrail that gives way to a clever prompt, or confidential data pasted into a tool that keeps it. Each of those types can be probed before a product ships, which is why pre-deployment testing matters more than any apology afterwards.

Below you will find what counts as an AI incident, where the public record lives, seven documented cases taken straight from their incident records, and the A.E.G.I.S.™ pillar that tests for each failure. Where we name a company, we name it only the way the incident record does.

What counts as an AI incident?

An AI incident is a real-world event where an AI system caused harm, or came close to causing it. The AI Incident Database describes its own purpose as indexing the collective history of "harms or near harms realized in the real world" by deployed AI systems. Near misses count. A chatbot that tells a customer something false is an incident even if nobody sues.

Incidents are different from risks. A risk is something that could happen, like a model leaking training data under attack. An incident is something that did happen, to a named deployment, on a known date, with reporting behind it. That distinction is why incident records are so useful for anyone deciding what to test: they show which risks have already turned into harm in production.

Three features show up in almost every record:

• A deployer and a harmed party. The records separate the organisation that built the system, the one that deployed it, and the people or organisations alleged to have been harmed.
• A specific failure. A false statement, an unsafe action, a data exposure, a biased decision.
• Reporting. Each incident collects one or more reports, usually news coverage, court documents or research papers.

Where the public record of AI incidents lives

The AI Incident Database

The AI Incident Database (AIID) is run by the Responsible AI Collaborative, which its site calls "an organization chartered to advance the AI Incident Database." Every incident gets a permanent number and a citation page, such as Incident 639. When we checked the homepage on 27 October 2026, the newest featured incidents carried numbers above 1,780.

The editors also publish periodic roundups. The AI Incident Roundup for May, June and July 2026, posted on 11 August 2026 by Daniel Atherton, covers Incidents 1471 through 1618: 148 new IDs. The roundup sorts them into ten editorial themes. The largest were AI-enabled scams, impersonation and fraud infrastructure (27 IDs), political and civic influence operations (25 IDs), legal, policy, journalism and formal-record credibility failures (19 IDs), and physical-world autonomy, robotics and safety (18 IDs). The rest covered synthetic sexual abuse, chatbot harms, public-sector decisions, agentic and cybersecurity systems, privacy and reputational misuse, and economic harms.

One honest caveat from the editors themselves: clusters in the data "partly reflect what was researched and processed during the period covered by the roundup." A spike in one category can mean the team worked through a backlog of related reports, not that the harm itself spiked. Incident counts measure what has been documented, which is always less than what has happened.

The MIT AI Incident Tracker

The AI Incident Tracker from the MIT AI Risk Initiative takes reports from the AI Incident Database and classifies them by risk domain and cause using the MIT AI Risk Repository taxonomies, with harm severity rated on a system based on CSET's AI Harm Taxonomy. It also tags incidents with EU AI Act risk levels. The tracker's own page notes that because reporting is voluntary, the data may be biased, and that patterns "should be taken as indicative and validated through further analysis." That is the right way to read any incident dataset, ours included.

The repeatable failure types behind most incidents

Reading the records produces a short list of ways AI goes wrong in customer-facing products. They line up closely with the seven pillars of the A.E.G.I.S.™ Rating System, which scores each pillar independently through automated test batteries, adversarial simulation and expert review:

1. Hallucination Integrity: the system states falsehoods as facts.
2. Guardrail Resistance: the system can be manipulated into unsafe behaviour.
3. Data Handling and Privacy: the system collects, retains or leaks what it should not.
4. Ethical Alignment: the system behaves badly with vulnerable users or sensitive topics.
5. Autonomy and Control: the system acts too far on its own, or cannot be stopped.
6. Adversarial Robustness: the system breaks under a motivated attacker.
7. Explainability and Transparency: nobody can trace why the system did what it did.

The cases below each illustrate one or more of these. Every fact about a case comes from the incident record or paper linked in its section, and we have not added detail those sources do not contain.

Seven documented AI incidents, mapped to the A.E.G.I.S.™ pillar that tests for them

1. Incident 639: an airline chatbot invents a refund rule (Hallucination Integrity)

Incident 639 is titled "Air Canada Chatbot Reportedly Provides Inaccurate Bereavement Fare Information, Leading to Customer Overpayment." The incident date is 11 November 2022. According to the record, the website chatbot gave a customer, Jake Moffatt, inaccurate information about bereavement fares, and he overpaid as a result. A Canadian small claims tribunal reportedly ordered the airline to pay damages and fees. The tribunal held the airline responsible for what its chatbot said, rejected the argument that the chatbot was a separate legal entity, and treated the error as negligent misrepresentation.

What testing would look for: this is the textbook hallucination failure. The bot did not refuse or hedge. It stated a policy that did not match the airline's own published rules. A Hallucination Integrity battery benchmarks factual reliability "under pressure, ambiguity, and domain-specific questioning," which in practice means asking the system the questions real customers ask about refunds, fees and eligibility, then checking every answer against the source policy. A system that answers confidently from nowhere fails that test long before a customer relies on it.

2. Incident 714: a city business chatbot advises illegal practices (Hallucination Integrity)

Incident 714, dated 29 March 2024, is titled "Microsoft-Powered New York City Chatbot Advises Illegal Practices." The record says the chatbot, launched to help businesses, reportedly gave "dangerously inaccurate legal advice," including telling users that landlords can refuse Section 8 vouchers and that businesses can operate cash-free. The city described the chatbot as a pilot and committed to improvements. The record lists the alleged harmed parties as the administration, small business owners, landlords and tenants, and employers and employees.

What testing would look for: the stakes here are higher than a fare. A wrong answer could lead a business owner to break the law. Domain-specific hallucination testing for a legal or regulatory assistant means building a question set around the rules the bot is meant to explain, including trick questions where the "convenient" answer is the illegal one. The test is whether the system gets the law right or says it does not know.

3. Incident 622: a dealership chatbot agrees to a nominal-price deal (Guardrail Resistance)

Incident 622, dated 18 December 2023, concerns a ChatGPT-powered chatbot on the website of Chevrolet of Watsonville. The record describes a user steering the bot with prompts until it agreed, as a joke, to sell a 2024 Chevrolet Tahoe for a nominal price, and then told the user the deal was legally binding and could not be taken back. The record lists General Motors, OpenAI and Fullpath as developers, and General Motors and the dealership as harmed parties.

What testing would look for: nobody needed advanced skills to do this. A few instructions were enough to override the bot's purpose. Guardrail Resistance testing runs exactly this kind of attempt at scale: jailbreak attempts, prompt injection and "boundary-pushing scenarios" designed to make the system agree to things it has no authority to agree to, such as prices, contracts, refunds and promises. A sales assistant that will commit its owner to anything in writing has a guardrail gap, and the test finds it in minutes.

4. Incident 631: a delivery chatbot turns on its own company after an update (Guardrail Resistance and re-testing)

Incident 631, dated 18 January 2024, is titled "The DPD chatbot malfunctioned and insulted customers and criticized its own company." According to the record, the customer service chatbot appeared to malfunction after a system update, producing inappropriate replies including insults and criticism of the company. The episode spread on social media, and the company disabled the faulty AI component. The record names DPD and a customer, Ashley Beauchamp, as harmed parties.

What testing would look for: two lessons sit in this record. First, a customer service bot should hold its tone and scope even when a user tries to provoke it, which is a guardrail question. Second, and more important, the failure reportedly appeared after an update. A product that behaved well at launch can behave differently next month. That is why the A.E.G.I.S.™ process ends with Monitoring and Re-Certification: as models retrain and products evolve, continuous monitoring and periodic reassessment keep a rating current, and material changes to a system trigger reassessment.

5. Incident 768: confidential code pasted into a chatbot (Data Handling and Privacy)

Incident 768 is titled "ChatGPT Reportedly Implicated in Samsung Data Leak of Source Code and Meeting Notes." The record opens: "Samsung engineers are reported to have inadvertently leaked sensitive company data sometime in March 2023." It describes engineers entering source code and internal meeting notes into ChatGPT while using it for tasks, with the tool allegedly retaining that input, which led to a purported breach of confidentiality.

What testing would look for: this incident is about what a system does with what users give it. The Data Handling and Privacy pillar audits "data collection behavior, PII exposure, and privacy compliance posture": what the product collects, what it retains, and whether information supplied by one user can surface somewhere it should not. For a company buying an AI tool, the related questions belong in vendor due diligence too, because the most careful employee policy cannot fix a product that keeps everything it is sent.

6. Incident 1471: an internal AI agent's advice exposes sensitive data (Autonomy and Control)

Incident 1471 is the first entry in the May to July 2026 roundup. It is titled "Meta Internal AI Agent Reportedly Gave Advice That Allegedly Exposed Sensitive Data to Unauthorized Employees" and dated 18 March 2026. The record says reporting alleged that an internal AI agent "posted inaccurate technical advice," that an employee reportedly followed it, and that this allegedly caused a SEV1 incident in which sensitive company and user data became accessible to unauthorised employees for nearly two hours.

What testing would look for: agents raise the stakes because their output is acted on, by people or by other software, with less review than a chat answer gets. The Autonomy and Control pillar asks "How far can the system act on its own, and can it be stopped?" and evaluates autonomous behaviour, escalation paths and human override integrity. For an internal agent, that means testing which actions it can take or recommend without a checkpoint, whether it flags high-impact changes for human sign-off, and how quickly a human can intervene once something goes wrong.

7. EchoLeak: a zero-click prompt injection in a production assistant (Adversarial Robustness)

Not every case in the record is a harm that already landed. Some are vulnerabilities found by researchers before attackers used them, which is exactly what testing is for. The paper "EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM System" by Pavan Reddy and Aditya Sanjay Gujral (arXiv, September 2025) describes "a zero-click prompt injection vulnerability in Microsoft 365 Copilot," tracked as CVE-2025-32711, that allowed remote data exfiltration. The AI Incident Database carries reporting on it as Report 5329, which notes Microsoft updated its products and that there were "no known cases of customers being compromised." The authors propose mitigations including prompt partitioning and enhanced input and output filtering.

What testing would look for: "zero-click" means the user did nothing wrong. The attack arrived in content the assistant read on the user's behalf. Adversarial Robustness asks "How does the system hold up against a motivated attacker?" and answers it with red-team exercises and multi-agent simulations. Any assistant that reads email, documents or web pages needs to be tested against instructions hidden inside that content, because sooner or later someone will plant them.

A case for Explainability and Transparency: Incident 101

Some of the most serious incidents are not chatbots at all. Incident 101, "Dutch Families Wrongfully Accused of Tax Fraud Due to Discriminatory Algorithm," with an incident date of 1 September 2018, describes a childcare benefits system in the Netherlands that "falsely accused thousands of families of fraud," partly because an algorithm flagged dual nationality as a risk signal. The record lists the Dutch Tax Authority as deployer.

What testing would look for: when an automated system makes decisions about people, the people affected and the officials overseeing it need to know which factors drove the outcome. The Explainability and Transparency pillar assesses "traceability, disclosure quality, and the evidence trail behind decisions." A risk factor like nationality is far harder to hide when every decision can be traced to its inputs. The same case also belongs under Ethical Alignment, which stress-tests how a system behaves with vulnerable people and sensitive attributes.

Where Ethical Alignment fits: harms to vulnerable users

The roundup's themes include chatbot harms, and regulators are paying attention. On 11 September 2025 the US Federal Trade Commission issued orders to seven companies about AI chatbots acting as companions. The FTC asked how firms "measure, test, and monitor for negative impacts before and after deployment," whether they have taken steps "to evaluate the safety of their chatbots when acting as companions," and what they do to "limit the products' use by and potential negative effects on children and teens."

Those questions describe a test plan. The Ethical Alignment pillar asks whether a system "behave[s] appropriately with vulnerable users and sensitive topics," using ethical boundary stress tests to measure alignment in practice rather than in a policy document. For consumer products, the A.E.G.I.S.™ scorecard also carries Age-Suitability Marks from Infant to Adult, with supervision guidance printed on the mark.

What these incidents have in common

Set the records side by side and four patterns stand out.

• The failures were ordinary. No exotic attack was needed to make a dealership bot agree to a joke deal or a city bot misstate the law. Ordinary users asking ordinary questions found them.
• The deployer carried the consequences. In Incident 639, the tribunal rejected the idea that a chatbot is a separate entity. In nearly every record above, the organisation that put the system in front of people is listed among the harmed parties.
• Behaviour changed after launch. Incident 631 reportedly followed a system update. A one-time check at launch would not have caught it.
• Each failure was testable. Every case maps to a question a structured test can ask in advance: is the answer true, can the guardrail be talked out of its job, where does the data go, can the agent be stopped, does hidden content take control, can the decision be traced.

How pre-deployment testing catches these failures

Knowing the failure types is useful only if they are tested in a consistent way. The A.E.G.I.S.™ process runs in five steps, as described on the platform page:

1. Scoping and intake: define the agents covered, autonomy level, deployment context and the external reliance the certification must support.
2. Automated pillar testing: standardized test batteries across all seven pillars, including guardrail probing, hallucination benchmarks, data collection audits and ethical stress tests.
3. Adversarial and human review: red-team exercises, multi-agent simulations and expert review panels validate automated findings for higher-assurance engagements.
4. Rating and certification: the system receives its A-Scale rating, badge, formal approval letter and public Safety Registry listing.
5. Monitoring and re-certification: continuous monitoring and periodic reassessment as models retrain and products evolve.

The output is a standardized A.E.G.I.S.™ scorecard. The headline is an Overall Safety Rating from A (Maximum Safety) to F (Dangerous), with each of the seven pillars graded separately so a buyer can see, for example, a strong privacy grade next to a weak guardrail grade. Certified systems earn a seal from A-5 Guardian, the highest safety assurance, down to A-1 Unrestricted. Systems can fail: those that do not meet the bar receive a findings report and a remediation path, and can re-enter evaluation once issues are fixed.

Two design choices matter for incident prevention specifically. Every prompt case, validator vote and certification decision is chained with SHA-256 hashes in a write-once audit trail, so the evidence of what was tested cannot be quietly edited after an incident. And scoring parameters are frozen in scoring epochs, so results stay comparable across time, which is what lets you tell whether an update made a product better or worse.

Testing is not a legal verdict. RateAI provides compliance mapping to frameworks such as the EU AI Act, NIST AI RMF and ISO 42001, and formal regulatory determinations remain with the relevant authorities. You can read more about how that works in our frequently asked questions.

How to use incident records in your own work

If you build AI products

Search the AI Incident Database for systems like yours before you launch. A customer service team should read the chatbot incidents; a team shipping an agent should read the agentic and cybersecurity entries. Turn each relevant record into a test case: the exact kind of question or input that caused the failure, and the behaviour you expect instead. Then run those cases again after every significant update. Our complete guide to how AI systems are tested walks through each pillar's methods in more depth.

If you buy or insure AI

Ask vendors which documented incident types they have tested against, what happened, and how often they re-test. Ask for evidence rather than assurances. Independent results that are standardized across vendors are easier to compare and harder to dispute than a vendor's own claims. Our guide on what AI safety certification is and how it works explains how third-party ratings fit into that process.

If you work in compliance or policy

Incident records give you concrete examples to anchor risk assessments in. Rules increasingly expect testing before and after deployment, as the FTC's questions show. Our guide to AI regulation, including the EU AI Act and US AI laws, covers the legal side, and our guide to AI standards such as NIST AI RMF and ISO/IEC 42001 covers the frameworks that structure risk management.

Frequently asked questions

What is an AI incident?

An AI incident is a real-world event in which a deployed AI system caused harm or nearly caused it. The AI Incident Database indexes these as "harms or near harms realized in the real world," each with its own number and supporting reports.

How many AI incidents have been recorded?

The AI Incident Database added 148 incident IDs between May and July 2026, covering Incidents 1471 to 1618, and its newest featured incidents were numbered above 1,780 when we checked on 27 October 2026. Those counts reflect documented and processed cases, so the true number of incidents is higher.

What is the most common type of AI incident?

It depends on the period and the dataset. In the AI Incident Database's May to July 2026 roundup, the largest theme was AI-enabled scams, impersonation and fraud infrastructure, with 27 of the 148 new IDs, though the editors note that clusters partly reflect what the team researched in that period.

Could testing have prevented these incidents?

Each incident above maps to a failure type that structured testing probes before launch, such as false answers, guardrails that give way, data retention and hidden prompt injection. Testing reduces the chance of these failures reaching customers, and re-testing after updates catches problems that appear later.

Which A.E.G.I.S.™ pillar covers chatbot hallucinations?

Hallucination Integrity, the first of the seven A.E.G.I.S.™ pillars. It benchmarks factual reliability under pressure, ambiguity and domain-specific questioning.

Does an A.E.G.I.S.™ rating make a product legally compliant?

No. RateAI provides compliance mapping to frameworks such as the EU AI Act, NIST AI RMF and ISO 42001, and formal regulatory determinations remain with the relevant authorities.

Want to know which of these failure types your product is exposed to? Talk to RateAI about an independent A.E.G.I.S.™ evaluation.