Crane AI Vision Algorithm Training & Testing Platform

Kelude Heavy Industry has established a dedicated training and testing platform for crane AI vision algorithms, amassing more than 500,000 annotated industrial defect images. This dataset provides a solid foundation for the ongoing development of the company's crane AI vision-based visual safety monitoring system. The platform integrates data acquisition, annotation management, model training, performance evaluation, and hardware-in-the-loop testing into a single infrastructure — the first of its kind in the domestic crane industry purpose-built for AI vision algorithm development.

Platform Purpose and Development Background

The performance of AI vision algorithms depends heavily on the quality and scale of training data. When Kelude Heavy Industry launched its AI vision R&D initiative in 2023, the company recognized early on that the lack of industry-specific annotated datasets was the critical bottleneck preventing crane AI vision technology from reaching real-world deployment. General-purpose datasets such as COCO and ImageNet transfer poorly to industrial environments — dust, changing light conditions, occlusion, and vibration on the factory floor cause a significant drop in detection accuracy when generic models are deployed on-site. To address this, the company invested over CNY 3 million (approximately USD 445,000) to build a dedicated AI vision algorithm training and testing platform. Systematic on-site data collection and manual annotation began in September 2023, and over a 22-month period the platform was fully commissioned and the 500,000-image annotated dataset was accumulated.

Data Acquisition System

Data acquisition covers four lighting conditions — daytime, nighttime, dawn, and dusk — and four weather conditions: sunny, overcast, rainy, and foggy. The acquisition setup includes eight 5-megapixel industrial cameras mounted on the underside of the crane main girder, plus one mobile monitoring camera positioned beside the lifting and transport area. All cameras use global shutter sensors, operate at 25 fps, and support HDR wide dynamic range. The annotation platform supports four annotation types — bounding boxes, polygons, keypoints, and semantic segmentation — covering detection, defect segmentation, and keypoint localization tasks. A three-tier quality control workflow is enforced: initial annotation by annotators, review by team leads, and spot-checking by algorithm engineers. The sampling rate is no less than 20%, and annotation accuracy is required to exceed 97%.

Dataset Composition

The 500,000 annotated images are organized into five major detection categories. The personnel intrusion detection dataset contains 250,000 images, with annotation classes covering four subcategories: personnel with visible helmets, personnel without visible helmets, personnel not wearing helmets, and personnel approaching outside the work zone. It covers four postures — walking, squatting, bending, and operating — across three distance ranges: distant (pixel height <50px), medium (50–150px), and close (>150px). The suspended load detection dataset contains 120,000 images, with annotation classes for steel coils (horizontal and vertical), steel plates (flat and angled), equipment components (boxes and frames), and bulk materials (piles and irregular shapes). It captures load visual characteristics across different lifting spreader types — wire rope slings, lifting slings, and lifting magnets — and various load configurations. The weld seam defect detection dataset contains 80,000 images, with annotation classes for five defect types: cracks, incomplete penetration, porosity, slag inclusion, and undercut. Data sources include standard specimens photographed in the laboratory and weld images captured on the factory floor, with every image verified through metallographic analysis to confirm the presence of the annotated defect. The wire rope inspection dataset contains 30,000 images, covering five anomaly classes: wire breaks, wear, corrosion, deformation, and reduced rope diameter. The crane rail anomaly detection dataset contains 20,000 images, with annotation classes for rail misalignment, rail cracks, and foreign objects on the rail.

Model Training and Iteration

The training platform runs two independent training environments. The online training environment is built on 4 NVIDIA RTX 4090 GPUs and supports both PyTorch and TensorFlow frameworks, with Docker-based containerization enabling rapid task deployment and resource isolation. The offline training environment is equipped with 2 NVIDIA A100 80GB GPUs for accelerated training of large models and large-scale datasets. The platform includes an automated training pipeline that manages the full process from data version management, hyperparameter search, model training, and evaluation comparison to model registration. Models are trained on the YOLOv8 architecture using Mosaic data augmentation, adaptive anchor box computation, and CIoU loss. After multiple training iterations on the 500,000-image dataset, personnel intrusion detection mAP improved from an initial 67.3% to 92.3%, suspended load detection mAP reached 89.1%, and weld seam defect detection mAP reached 91.5%.

Testing and Verification Facilities

In addition to training capabilities, the platform includes a hardware-in-the-loop testing facility. The test rig consists of a 5-ton experimental overhead crane, a programmable environment simulation system, and a data acquisition and analysis system. The experimental crane is installed in an indoor test workshop with a rail span of 12 meters and a lifting height of 6 meters, supporting both manual and automatic operation modes. The programmable environment simulation system controls lighting (0–50,000 lux adjustable), temperature and humidity (temperature −10°C to 50°C, humidity 20%–90% RH adjustable), and dust concentration (0–10 mg/m³ adjustable). The data acquisition and analysis system synchronously records AI vision output, encoder position data, and environmental parameters, generating a comprehensive assessment report for each frame. Hardware-in-the-loop testing validates AI vision algorithm performance under controlled experimental conditions across extreme operating conditions, providing reliable laboratory evidence for algorithm stability assessment before deployment on host equipment.

Data Security and Privacy

Industrial vision datasets may contain sensitive information about customer factory layouts and equipment configurations, making data security management a key consideration in platform design. All raw data is stored on an on-premises NAS storage array with a total capacity of 200TB, configured with RAID6 redundancy. Data access is managed through tiered permission levels — raw data access, annotation data access, and model data access — with all access operations recorded in an audit log. When working with external annotation teams, the company employs data desensitization and block-based annotation strategies: each data block contains only a single scene segment with no factory-identifiable features, and final data merging and quality inspection are performed by internal personnel after annotation is complete. Data sharing with the Zhengzhou University joint laboratory follows a signed data usage agreement and is restricted to academic analysis purposes; the laboratory is prohibited from using the data for any commercial projects.

Future Development Roadmap

Kelude Heavy Industry plans to expand the annotated dataset to 1 million images by 2027, with a focus on adding edge-case data for adverse weather and low-light conditions to address the long-tail distribution problem in extreme operating conditions. The company will also begin building time-series datasets to accumulate continuous-frame annotations for video-based behavior recognition and trajectory prediction model training, with each time-series annotation containing a continuous detection sequence of no fewer than 300 frames. Additionally, the platform will release selected desensitized datasets for academic research collaboration, helping to establish benchmark testing standards for the crane AI vision field.

Frequently Asked Questions (FAQ)