AI Vision Overhead Crane Load Detection & Weighing System
AI Vision for Overhead Cranes uses a deep learning YOLOv8s model and ByteTrack multi-object tracking algorithm for real-time load detection, achieving detection accuracy ≥95% (mAP@0.5). The LSTM dynamic weighing model delivers dynamic weighing error ≤2% FS, and combined with a vibration-damped industrial camera (1920×1080@30fps) and an embedded AI computing module (TOPS≥4), it forms a complete AI vision system. Kelude's AI vision solutions have been deployed in over 30 units across foundry, steel, and warehousing industries.
The AI vision recognition system for overhead cranes consists of three components: a vibration-damped industrial camera, an embedded AI computing module, and vision analysis software. The camera is mounted beneath the trolley frame, looking down at the load area. Image data is transmitted to the AI computing module via Ethernet, where the YOLOv8s model performs load detection and object classification at the edge. The system also integrates electronic scale weighing functionality: an LSTM deep learning model compensates in real time for load deformation and wire rope sway visible in the video feed, enabling dynamic weighing during lifting operations. The architecture diagram below illustrates the complete data flow from image acquisition to detection result output. The system design must comply with ISO 4301 Crane Design Standard — Key Clause Interpretation: Load, Structure, Mechanism, Electrical, and Safety Systems and JJG 555-2016 related standards.
Vision Recognition System Architecture
The AI vision system architecture is organized into three layers: image acquisition, edge inference, and result output. Image acquisition layer: The industrial vibration-damped camera is mounted beneath the trolley frame, capturing 1920×1080 resolution at 30fps. An 850nm infrared fill light ensures clear imaging under low-illuminance conditions, and the camera has an IP67 protection rating. Edge inference layer: The embedded AI module uses an NVIDIA Jetson series or Rockchip RK3588 processor, delivering ≥4 TOPS (INT8) of computing power. It runs the YOLOv8s model for per-frame inference at ≥25fps to meet real-time requirements. Result output layer: Detection results — including load category, load position coordinates, and dynamic weighing values — are transmitted to the crane PLC control system via RS485 or Ethernet.
Communication between the vision system and the crane PLC uses the Modbus TCP protocol. The detection result data frame contains load ID, coordinates, dimensions, weight, confidence score, and checksum, with a total frame length of 16 bytes and a communication cycle of 100ms. Kelude's AI vision solution complements the visual SLAM approach used in the crane positioning system: SLAM handles environment mapping and path planning, while AI vision handles load detection and weighing.
Load Detection Technical Solution
Load detection uses the YOLOv8s model as the base detector. The training dataset covers 12 common load categories (steel plate, coil, structural steel, casting, box, pipe, bulk material, container, die, equipment, empty spreader, and abnormal state), with ≥2,000 training samples per category and over 25,000 annotated images in total. Data augmentation strategies include multi-angle rotation (±15°), lighting variation, occlusion simulation, and motion blur. The model input size is 640×640 pixels, with an NMS threshold of 0.45 and a confidence threshold of 0.5, achieving a final detection accuracy of mAP@0.5 ≥95%.
Multi-object tracking uses the ByteTrack algorithm, which combines Kalman filter prediction with Hungarian algorithm matching to bind load IDs across consecutive frames. ByteTrack's key advantage lies in its secondary matching process for low-confidence detection boxes, which significantly reduces ID switches caused by occlusion and viewpoint changes. In field tests on a 10t overhead crane, the ID switch rate was less than 1 per 1,000 frames, and the multi-object tracking accuracy (MOTA) reached ≥93%. Tracking results output load movement trajectories and work-cycle statistics, providing a data foundation for production efficiency analysis.
Technical Manager's Note — Kelude Heavy Industry:
“The biggest real-world challenge in AI vision load detection is not model accuracy — it's the unpredictability of on-site lighting conditions. During deployment projects in foundry workshops and steel mills, Kelude encountered a typical problem: reflective surfaces on castings and intense furnace lighting caused overexposure in the camera feed, dropping YOLOv8s detection confidence from 95% to 40%. Our solution was threefold: first, we switched to a global shutter sensor with hardware WDR (wide dynamic range >120dB); second, we added overexposure samples to the training dataset for data augmentation; and third, we deployed an adaptive exposure algorithm on the AI computing module that dynamically adjusts exposure time and gain based on the brightness average of the previous frame. This approach restored detection confidence to above 90% in high-glare environments.”
Intelligent Weighing with Electronic Scale
Dynamic weighing is based on an LSTM deep learning model that compensates in real time for wire rope sway beneath the load, load rotation, and camera vibration visible in the video feed. Traditional electronic scales require the load to come to a complete stop (hovering for ≥3 seconds) before a stable reading can be obtained. The AI vision scale, by contrast, learns the time-series characteristics of load sway through the LSTM model and outputs a stable weight value during the lifting operation itself. The LSTM model input features include the rate of change in the load bounding box area, the acceleration of the load center displacement, and the wire rope sway angle and sway velocity; the output is the compensated weight value.
The LSTM training dataset was built from calibration data collected on Kelude's test platform: video features and load cell ground-truth values were captured synchronously during lifting operations with test weights of known mass, yielding approximately 8,000 training sequences. Field measurements show dynamic weighing error ≤2% FS, with a single weighing cycle of under 1 second (compared to 5–8 seconds for traditional methods). At a duty cycle of 25 lifts per hour, this saves approximately 3–5 minutes of weighing time per day.
Key Parameters of the AI Vision System
| ParameterItem | visual inspection | Dynamic Weighing | ConventionalElectronic Scale | Description |
|---|---|---|---|---|
| core algorithm | YOLOv8s+ByteTrack | LSTMTime-series Model | ResistanceStrain-gauge TypeSensor | AINon-contact Solution |
| Accuracy | mAP≥95% | ≤2%FS | ≤0.5%FS | ConventionalAccuracyHigher Accuracy but Requires Static Condition |
| Response time | 40ms(Inference) | <1s(Dynamic) | 3~8s(Requires Hovering) | AIDynamic WeighingFast4~8Times |
| Hardware Cost | 2~410k CNY | Included invision systemWithin | 0.5~210k CNY | AIAdditional Computing Power of the SolutionModule |
| Application Scenarios | Bulk Material/Castings/Plates/Sheets | Continuous operationWeighing | Precision Weighing | Complementary Use of Both |
Key System Data
| detection accuracy ≥95% YOLOv8s mAP@0.5 | Weighing Error ≤2%FS LSTMDynamic Compensation | inference speed 25fps Jetsonedge device |
| Computing Power Requirement 4TOPS INT8Quantized Inference | Training Samples 25000Images 12Category-based Load Annotation | IDSwitching Rate <1Times/Thousand Frames ByteTrackMultipleobject tracking |
FAQ: AI Vision for Overhead Cranes
Q: What is the investment required for an AI vision recognition system on an overhead crane?
A: Hardware costs for the AI vision system range from approximately $3,000 to $6,000 per unit, including a stabilized industrial camera with global shutter (about $450–$900), an embedded AI computing module (roughly $1,500–$3,700), and mounting accessories. Software licensing is billed annually at about $750–$1,500 per crane per year. Compared to a traditional electronic scale ($750–$3,000), the AI vision solution delivers both load detection and dynamic weighing in a single system, offering better overall value. Kelude provides free on-site installation and commissioning for the first deployment at each customer site.
Q: Which is more accurate — an AI vision scale or a traditional electronic scale?
A: A traditional electronic scale offers higher static accuracy (≤0.5% FS) but requires the load to be held stationary for 3–8 seconds, which slows down operations. The AI vision scale provides dynamic accuracy of ≤2% FS — slightly lower — but requires no dwell time, saving 3–5 seconds per lift. At a typical frequency of 25 lifts per hour, this adds up to 3–5 minutes of weighing time saved per shift. Kelude's recommendation: keep a traditional electronic scale where high-precision, trade-level weighing is required, and use the AI vision scale for production statistics and material management applications.
Q: Can the AI vision system operate reliably in harsh environments?
A: Yes. Kelude's AI vision solution is specifically engineered for harsh conditions found in foundry workshops (high heat and dust), steel mills (intense light and steam), and ports (salt spray and humidity). The camera features an IP67 protection rating and a global shutter sensor with a wide dynamic range exceeding 120 dB. The AI computing module uses an industrial-grade thermal design rated for operating temperatures from -20°C to 70°C (-4°F to 158°F). Field data from foundry deployments shows that with WDR hardware exposure optimization, detection confidence remains above 90% even under intense lighting conditions.
Q: How does the AI vision system integrate with an existing overhead crane control system?
A: The AI vision system communicates with the crane PLC via Modbus TCP, transmitting detection results in a standard 16-byte data frame format. Hardware integration requires only a single Ethernet cable connecting the AI computing module to the PLC network switch. For software, a Modbus client program must be written in the PLC to parse the data frames. Kelude provides standard integration libraries for Siemens S7-1200/S7-1500 and Mitsubishi FX5U/L-series PLCs, and commissioning typically takes about 2 working days.
