How to Stop AI Fault Diagnosis Hallucinations in Industrial Cranes

Picture this early morning scenario: in the control room of a steel mill's overhead crane maintenance team, an AI assistant fires off an alarm—"Main hoist brake friction lining wear has reached the scrapping limit. Immediate shutdown and replacement recommended." It even cites a "standard clause" claiming the remaining lining thickness must not fall below 40% of the original. The duty operator follows the instruction, halts the crane, and tears down the brake overnight—only to find the lining still has 62% thickness left, nowhere near the scrapping threshold. The entire production line sat idle for six hours for nothing.

The operator's first instinct is to suspect a faulty sensor. They pull up the vibration and temperature history curves and cross-check them repeatedly—all data reads normal. Then they look up the cited "standard clause," combing through the relevant design specifications and safety monitoring management standards. The clause the AI referenced simply does not exist—it fabricated a non-existent provision but made the standard number, the figures, and even the "scope of application" look entirely plausible. To make matters worse, the AI delivered its statement with absolute certainty and a well-structured format, making it nearly impossible for a standard operator to spot the fabrication at a glance.

The root cause is not the sensor, nor the database—it lies in how the LLM itself works. It generates text by selecting the next word probabilistically, optimizing for "sounds plausible" fluency rather than "is actually correct" factual accuracy. When the training data lacks a particular standard clause, the model does not say "I don't know"—it fills in the missing provision using the most likely language patterns. This is precisely what makes it so dangerous in industrial settings: an AI that pretends to know but doesn't is far more dangerous than one that admits its limitations.

Confident Yet Fabricated Alarms: How a Hallucination Escalates from False Alarm to Wrong Action

The scenario above is not an isolated case—it is a typical manifestation of LLM hallucination in industrial environments. The danger lies not in "giving a wrong answer" but in "giving a wrong answer that sounds convincing." The human brain has a deeply ingrained tendency: the more a response cites specific figures, standard numbers, and technical jargon, the more likely it is to be accepted as credible. And LLMs excel at packing their output with these "authority signals"—even when the content they stack up does not exist.

From this perspective, the harm of hallucination follows a three-stage amplification chain.

Stage one—false alarm: the model generates a seemingly professional but incorrect conclusion, whether a fabricated standard limit value or a misattributed fault diagnosis.

Stage two—misjudgment: on-site personnel, swayed by how "plausible" the conclusion looks, skip verification and treat the model output as the judgment basis.

Stage three—wrong action: based on the flawed judgment, they execute shutdown, disassembly, or part replacement, wasting labor hours and downtime—or worse, missing the real hazard and letting it develop into an accident.

Kelude's field research has repeatedly observed the same pattern: the more "specific numbers" an AI output contains, the more trust maintenance personnel place in it. This is exactly where the caution flag should go up—because hallucinations are most likely to be forged precisely in the areas of "specific numbers" and "standard clauses."

Why AI Fabricates with Confidence: The Dual Gap Between Probabilistic Generation and Factual Constraints

To understand why hallucinations are so hard to prevent, you first need to see how LLMs generate text. At its core, an LLM is a probabilistic model: given the preceding context, it calculates the most likely next word, then generates word by word. This mechanism is inherently good at producing fluent, coherent, grammatically natural text—but there is no step anywhere in the process that checks "is this statement true in reality?"

In other words, what the LLM does is "linguistically plausible," not "factually correct." When asked about a standard clause that does not exist in its training data, it does not return "no such provision found." Instead, it stitches together a "standard number + technical term + a seemingly reasonable limit value" based on the language patterns it has learned. The more professional and specific the fabricated content looks, the harder it is to detect.

This is precisely why standards like ISO 24621:2022, "Cranes — AI-based fault diagnosis", place such strong emphasis on traceability of diagnostic conclusions: a qualified fault diagnosis must be traceable back to specific sensor data and the judgment basis—not an unverifiable assertion. The default output of an LLM lacks exactly this "traceability chain," which explains why plugging it directly into alarm decision-making leads to problems.

Kelude's engineers describe this phenomenon as "fluency hallucination": the more fluent the output, the more easily it masks the absence of facts. Industrial environments demand the exact opposite—better a "not certain, needs verification" response than a polished but wrong conclusion.

The Three Most Common Industrial Hallucinations: Fabricated Standards, Fabricated Procedures, and Fabricated Data

In the specific context of crane maintenance, hallucinations typically appear in three forms, each with a different harm pathway.

The first is fabricated standard clauses. This is the most dangerous type. The AI may cite a non-existent specification number, or attach a wrong limit value to a real specification. If maintenance personnel act on it to shut down or release equipment, they are effectively letting a fictional "standard" make decisions on behalf of the real one.

The second is fabricated maintenance procedures. LLM-generated repair procedures often look "complete" but may omit mandatory safety steps such as power isolation, pressure release, or energy lockout. If the sequence or completeness of the steps is wrong, the direct threat is to the personal safety of the workers performing the task.

The third is fabricated measurement data. The AI may produce a vibration amplitude or temperature reading that no sensor ever captured, and use it to conclude "bearing degradation" or "gear wear." Such fabricated data puts condition assessment on an unstable foundation, and any replacement decision built on it loses its basis entirely.

The table below compares the harm pathways and prevention points of these three types of hallucinations:

From Point Corrections to Systemic Safeguards: A Preventive Framework for Trustworthy AI Deployment

Since hallucinations stem from the generation mechanism itself, expecting to "make the model infallible" is unrealistic. The right approach for industrial deployment is to shift the line of defense from "correcting errors after the fact" to "systemic safeguards," ensuring AI never has the opportunity to deliver hallucinated output into decision-making processes.

The first line of defense is anchoring AI-generated conclusions to verifiable sources. Any standard clause, limit value, or measurement data cited in an alarm must be traceable to its original source in the standard library and sensor records; anything that cannot be verified is excluded from the handling process. This requirement aligns with the principle emphasized in GB/T 28264 Safety Monitoring and Management System for Lifting Appliances—that safety monitoring parameters must have authentic records—AI conclusions cannot exist independently of real monitoring data.

The second line of defense is defining the capability boundary for AI: it can alert, summarize, and retrieve information, but it must not make decisions such as shutting down equipment, releasing loads, or replacing parts. When requests fall outside these boundaries, the model should explicitly decline or escalate to manual review rather than improvising an answer.

The third line of defense is manual review and audit trail. Kelude Heavy Industry adheres to this principle in practice: any AI output that affects equipment status assessment must go through manual confirmation, with the AI's raw output, cited evidence, and review conclusions all documented together. This way, even if a hallucination slips past the first two defenses, it can still be caught during review and traced through the audit trail.

Kelude Heavy Industry's recommendation: rather than pursuing "making AI never wrong," it is wiser to assume "AI can be wrong at any time" and build solid safeguards through review, documentation, and boundary enforcement. Hallucinations cannot be eliminated, but they can be kept out of decision-making.

📖 Related Reading: What Can Large Models Actually Do for Crane Maintenance? Practical Scenarios and Capability Boundaries | Equipment Health Management (PHM): Big Data and ML-Driven Predictive Maintenance for Overhead Cranes

FAQ: AI Alarms vs. Manual Inspections