AI Anti-Sway, Vision Recognition & Predictive Maintenance

In the industrial crane sector, the shift toward intelligent systems has moved well beyond the proof-of-concept stage and into full-scale implementation. While much of the industry is still debating whether to adopt these technologies, Kelude Heavy Industry has already established a defensible competitive edge through six parallel technology tracks—ranging from reinforcement learning-based anti-sway control and real-time visual AI positioning to predictive maintenance and system-wide optimization via intelligent scheduling and Digital Twin. This article breaks down each of these six core technologies and examines how they work in concert to create a complete closed loop, from individual machine control to system-level intelligence.

Reinforcement Learning for Anti-Sway Control

Controlling the sway of a crane's lifting spreader is one of the classic, most challenging problems in industrial automation. Traditional anti-sway methods, which rely on PID Control or input shaping techniques, perform adequately under fixed operating conditions. However, their control accuracy degrades significantly when faced with dynamic variables such as changing rope lengths, variable loads, or external wind forces. Kelude Heavy Industry's technical team chose a more demanding yet higher-potential path: adaptive anti-sway control based on reinforcement learning.

PPO Algorithm: A Stable and Efficient Core Controller

Among various reinforcement learning algorithms, the team prioritized Proximal Policy Optimization (PPO) as the core controller. PPO's clipping mechanism limits the step size of policy updates, mitigating the training instability commonly associated with traditional policy gradient methods. In the crane anti-sway scenario, the PPO state space is designed as a six-dimensional vector: rope length, current spreader sway angle, sway angular velocity, trolley position, trolley speed, and load mass. The action space is the continuous output value of trolley acceleration, constrained within a range of -2 m/s² to 2 m/s².

The design of the reward function is critical to the system's success. The Kelude team implemented a piecewise weighted reward strategy: the primary reward is the negative square of the spreader sway angle (encouraging a zero angle), while auxiliary rewards include a penalty for trolley positioning error (ensuring precise stopping) and a penalty for high jerk (preventing mechanical stress from sudden starts and stops). After approximately 2 million steps of simulation training, the PPO policy achieved a control accuracy of residual sway angle < 0.3° for Cross Travel Speeds ranging from 0.5 m/s to 2.0 m/s, demonstrating roughly 60% better disturbance rejection compared to traditional input shaping methods.

During training, the Kelude team also incorporated a Curriculum Learning strategy. The agent first learned basic anti-sway policies in a simplified environment with a fixed rope length and no wind disturbance. Subsequently, interfering factors such as rope length variations, random wind fields, and sudden load changes were progressively introduced. This incremental training approach significantly reduced the probability of random divergence in early training, cutting the number of training steps required for policy convergence by approximately 35%. Additionally, the team established a comprehensive policy evaluation baseline—performing a frozen evaluation every 100,000 training steps to record residual sway angle and positioning deviation in standardized test scenarios, ensuring no performance regression during model iteration.

SAC Algorithm: Soft Exploration for Varying Conditions

While PPO performs exceptionally well in highly repetitive conditions, its exploration capability can be insufficient when operating conditions change drastically—for instance, when the rope length is dynamically adjusted during operation or the load mass suddenly changes by more than 30%. To address this, the Kelude team introduced the Soft Actor-Critic (SAC) algorithm as a complementary strategy. SAC's key advantage lies in its maximum entropy framework, which maximizes the policy's entropy alongside cumulative reward, compelling the agent to continuously explore new behaviors during training.

In simulation environments with random wind field disturbances, the SAC policy converged approximately 40% faster than PPO. Moreover, in extreme tests where the load mass abruptly changed from 500 kg to 2,000 kg, the SAC controller maintained the residual sway angle within 0.5°. For real-world deployment, Kelude employs a dual-strategy switching mechanism: the PPO controller operates during steady-state conditions to ensure response efficiency and repeatability. When detected changes in operating parameters exceed a preset threshold, the system smoothly switches to the SAC controller to handle uncertainty. The online switching latency between the two strategies is kept under 50 ms, making the transition completely transparent to the operator.

The engineering implementation of this dual-strategy switching mechanism utilizes a Shared Feature Extractor architecture. PPO and SAC share the same state encoder, with the policy network diverging only at the output layer. This design eliminates the need to re-encode state information during switching and reduces the parameter count by approximately 40%. The controller software runs on a Beckhoff TwinCAT soft PLC, communicating with the Servo Drive via the ADS protocol, with a control cycle of 4 ms. This meets the real-time requirements for anti-sway control during high-speed crane operation. The entire anti-sway control system design strictly adheres to the ISO 4301 Crane Design Standard, achieving an engineering balance between algorithmic precision and structural safety margins.

Vision AI for Positioning and Identification

In crane operation scenarios, precise positioning of target objects is a prerequisite for automated loading and unloading. LiDAR and encoder-based solutions often suffer from high costs, complex maintenance, and susceptibility to dust interference. Kelude Heavy Industry has taken a different approach by building a deep vision-based AI positioning and identification system. Its core components include the YOLOv8 object detection network and the TensorRT inference engine.

YOLOv8: A Multi-Task Vision Engine from Detection to Segmentation

YOLOv8, Ultralytics' flagship vision model, achieves an optimal balance between speed and accuracy in object detection. The Kelude team customized the YOLOv8n (Nano version, only 3.2M parameters) model: a Coordinate Attention module was integrated into the Backbone to enhance the model's perception of the spatial relationship between the spreader and target racks, and a Bidirectional Feature Pyramid Network (BiFPN) was added to the Neck layer to improve multi-scale feature fusion efficiency.

The training dataset was compiled from 8 industrial cameras deployed in Kelude's own smart factory, accumulating over 120,000 annotated images. This dataset covers various lighting conditions (dawn/dusk, noon, overcast), different weather scenarios (rain/fog, dust), and a range of spreader angles (0° to 60°). After data augmentation (Mosaic, MixUp, random erasing) and 150 epochs of distributed training, the model achieved a mAP@0.5 of 97.3% on the test set, with an inference speed of 62 FPS on the NVIDIA Jetson Orin NX platform.

Notably, the Kelude team employed a semi-automatic annotation strategy. A pre-trained YOLOv8m model initially generated candidate boxes on raw images, which were then refined by human annotators. Compared to fully manual annotation from scratch, this strategy improved annotation efficiency by approximately 5 times, compressing the total man-hours for 120,000 annotated images from an estimated 2,400 hours to about 500 hours. The team also integrated an Active Learning loop—after each model iteration, ambiguous samples with inference confidence scores between 0.4 and 0.6 are automatically selected and prioritized for human annotation. This "let the model choose what it needs to learn most" strategy yielded over 5% accuracy improvement for a maximum of 30% additional annotation effort.

TensorRT Deployment: Real-Time Inference on the Edge

Deploying the YOLOv8 model onto edge computing devices in industrial settings presents a core challenge: higher model accuracy typically leads to greater inference latency. The Kelude team addressed this using the NVIDIA TensorRT inference optimization framework. Key techniques included: FP16 quantization, which compressed model weights from 32-bit to 16-bit floating point, boosting inference speed by approximately 1.8 times; Layer Fusion, which merged adjacent convolutional layers, batch normalization layers, and activation functions into a single kernel execution; and Dynamic Shape Inference, which supports adaptive switching of input resolution between 640×480 and 1280×720.

After TensorRT optimization, the inference latency for the YOLOv8n model on the Jetson Orin NX dropped from an original 23 ms to 7 ms, with power consumption maintained under 15W. This provides ample computational headroom for tracking objects across frames (using the ByteTrack algorithm) and calculating spatial coordinates, even with a 60 FPS camera input. For high-precision workstations, Kelude deployed the YOLOv8s model (Small version, 11.2M parameters). Combined with TensorRT INT8 quantization, this configuration achieved 98.1% detection accuracy at 45 FPS, specifically for tasks like container lock hole identification requiring millimeter-level positioning.

The calibration and maintenance of the vision system were also engineered for practicality. Each crane's vision module undergoes intrinsic parameter calibration (camera matrix and distortion coefficients) before delivery. After installation, hand-eye calibration (the transformation matrix between the camera coordinate system and the crane base coordinate system) is performed using a checkerboard target. Once deployed, the system automatically executes a self-check procedure every hour: the spreader moves to a preset calibration position and is compared against a virtual calibration point in the Digital Twin space. If the deviation exceeds 5 pixels, a recalibration alert is automatically triggered. This self-check mechanism addresses the long-standing operational pain point of "calibration drift" in industrial vision systems from an architectural design perspective.

Predictive Maintenance System

Unplanned crane downtime represents a significant cost burden in manufacturing. A single hour of halted production due to a critical component failure in a 50-ton Bridge Crane can result in comprehensive losses amounting to tens of thousands of yuan. Kelude Heavy Industry's Predictive Maintenance (PdM) system uses a combined neural network architecture of 1D-CNN and LSTM to identify early signs of failure from multi-dimensional sensor data, including vibration, temperature, and current.

1D-CNN: Local Feature Extraction from Vibration Signals

Vibration signals from industrial equipment are essentially time-series data with complex frequency components. The Kelude team selected a 1D convolutional neural network (1D-CNN) as the primary feature extractor for these signals. Unlike 2D-CNNs that process spectrograms, the 1D-CNN applies convolutional kernels directly onto the raw acceleration time-series data, preserving the temporal phase information of the signal. Each sensor signal channel (3 channels corresponding to X/Y/Z axes) is passed through 3 layers of 1D convolution and max pooling, resulting in a 128-dimensional high-level feature vector.

In bearing fault detection for the crane's Hoisting mechanism, the 1D-CNN model achieved a classification accuracy of 94.7% on a test set containing four states: normal, inner ring wear, outer ring pitting, and rolling element cracks. Critically, the model's detection rate for outer ring pitting begins to rise significantly approximately 72 hours before the fault occurs. This provides operators with a full three-day window to schedule planned maintenance, rather than passively waiting for a failure to happen.

The sensor deployment strategy is equally meticulous: vibration sensors are installed at six critical measurement points—the drive-end and non-drive-end bearing housings of the hoisting motor, the input and output shaft bearing housings of the gearbox, and the drum bearing housing. Temperature sensors are integrated into the windings and bearing housings, while current sensors are clamped onto the motor's three-phase supply lines. All sensor data is aggregated through a workshop-level IoT gateway and transmitted to an edge server via Modbus TCP. The sampling strategy employs adaptive frequency: during steady-state operation, a 5-second data segment is captured every 10 minutes (at a 12,800 Hz sampling rate); when vibration energy exceeds 1.5 times the baseline threshold, the system automatically switches to continuous acquisition mode.

LSTM: Trend Prediction for Temporal Evolution

While 1D-CNN excels at extracting features from local segments, its capacity for modeling long-term temporal dependencies is limited. The Kelude team addressed this by appending a two-layer LSTM (Long Short-Term Memory) module after the 1D-CNN output layer to capture evolution trends in sensor data over timescales ranging from hours to days. The LSTM hidden layer dimension is set to 256, with dropout (rate 0.3) applied to prevent overfitting.

In deployment, the hybrid architecture (1D-CNN + LSTM) achieved a Mean Absolute Error (MAE) of 12.4 hours in predicting the Remaining Useful Life (RUL) of hoisting motor bearings—meaning the system's prediction error for bearing remaining life stays within half a day. This level of accuracy supports a "schedule replacement one day in advance" maintenance strategy, reducing unplanned downtime by 76%.

The system also integrates a self-attention mechanism, enabling the model to automatically focus on the sensor channels most influential to the current state during prediction. For instance, during the early-stage wear of gearbox components, when characteristic frequency variations (sidebands around the meshing frequency) emerge, attention weights automatically shift to the corresponding acceleration and acoustic emission sensor channels, enhancing model explainability and operator trust.

Predictive maintenance outputs are pushed in real time to the maintenance team via a mobile app and workshop dashboards. The system employs a three-tier alert scheme: Green (normal, remaining life > 72 hours), Yellow (caution, remaining life 24–72 hours, scheduled maintenance recommended), and Red (critical, remaining life < 24 hours, immediate replacement required). Maintenance work orders are directly integrated with the ERP system—once an alert is triggered, spare parts requisitions and repair work orders are generated automatically. Since deployment, Kelude's in-house factory has seen spare parts inventory turnover improve by 28% and emergency procurement frequency drop by 63%. System maintenance cycles and inspection procedures follow the ISO 4309—Cranes—Maintenance standard, ensuring seamless alignment between predictive maintenance strategies and industry best practices.

Intelligent Dispatching Optimization

When multiple cranes operate collaboratively in the same workshop, the dispatching problem becomes exponentially complex—path conflicts, uneven task allocation, and efficiency losses from emergency stops are all frequent pain points in real-world operations. Kelude Heavy Industry's intelligent dispatching system integrates a Genetic Algorithm (GA) with a Deep Q-Network (DQN), striking a practical balance between traditional operations research optimization and deep learning.

Genetic Algorithm: Offline Optimization for Global Path Planning

The genetic algorithm is a search heuristic inspired by natural evolution, particularly well-suited for large-scale combinatorial optimization problems. In the crane dispatching context, a "chromosome" encodes a complete task sequence and path allocation plan, while the fitness function evaluates three metrics: makespan (total completion time), total energy consumption, and path conflict count. The Kelude team designed adaptive crossover and mutation rates: when population diversity declines, the mutation rate automatically increases to escape local optima; when the optimal solution stabilizes, the mutation rate decreases to enable finer-grained search.

In simulation tests involving 8 cranes and 120 tasks, the GA-optimized dispatch plan reduced average total completion time by 22% and energy consumption by 18% compared to manual scheduling, while eliminating path conflicts entirely. The GA's offline computation time is approximately 47 seconds—perfectly acceptable for scheduling scenarios measured in hours.

In engineering deployment, the GA solution also incorporates physical workshop constraints: each crane's available working area (impassable physical obstacles, restricted maintenance zones, etc.), material buffer capacity at each workstation, and priority weights for different task types. The constraint model is maintained as a YAML configuration file, independent of the algorithm code—after a workshop layout change, only the configuration file needs updating, not the algorithm itself. This significantly enhances system adaptability in production line retrofit scenarios.

DQN: Real-Time Dynamic Rescheduling Brain

While the offline GA solution provides globally optimal task allocation, it can become instantly obsolete when unexpected events occur on the shop floor—a crane fault shutdown, an urgent order insertion, or an operator stepping away. To address this, the Kelude team built an online rescheduling module based on a Deep Q-Network. The DQN state space includes: each crane's current position and status (idle/working/faulted), the current progress of each task, and a quantitative measure of workshop congestion. The action space consists of discrete decisions to assign pending tasks to available cranes.

After approximately 500,000 steps of Q-learning training, the DQN policy achieves rescheduling latency of under 0.8 seconds when unexpected events trigger, far faster than the minute-level response of manual intervention. In combined fault-and-urgent-order stress tests, the DQN rescheduling approach reduced task delay losses by an average of 23% compared to manual emergency handling. The GA+DQN combination delivers a two-tier dispatching strategy—"offline global optimization plus online real-time adjustment"—ensuring both global optimality and resilience to uncertainty.

After system deployment, the Kelude team also introduced a reinforcement learning-based dispatch evaluation dashboard that quantifies scheduling performance for each shift: GA-planned completion time predictions, DQN actual execution deviations, manual intervention frequency, and cause distribution. Operations supervisors can use this data to continuously fine-tune dispatching parameters. After three months of iterative tuning, overall workshop scheduling efficiency improved by 31% compared to traditional manual dispatching, with average task waiting time reduced from 12 minutes to 4.5 minutes.

Digital Twin Platform

The Digital Twin is the core infrastructure connecting the physical and digital worlds. Kelude Heavy Industry's digital twin platform is not merely a 3D visualization tool—it is a closed-loop system integrating real-time data acquisition, physics-based simulation engines, and AI-driven decision feedback.

At the data acquisition layer, every crane—from large overhead cranes to flexible modular KBK flexible crane standard components—is equipped with approximately 40 sensor nodes, including motor encoders (position and speed), six-axis inertial measurement units (attitude and vibration), temperature probes (bearings and windings), current clamps (load and power consumption), and vision cameras (environmental awareness). Sensor data is transmitted at 100 Hz over the workshop's 5G private network to edge servers, then aggregated via MQTT protocol into the platform's data middle platform.

The simulation engine layer is built on Unity's industrial extension capabilities, driven by a hybrid of discrete event simulation (DES) and physics-based continuous simulation. The physics engine is based on NVIDIA PhysX, providing real-time dynamic modeling of the crane's three primary mechanisms: hoisting, traveling, and slewing. In the digital twin space, the virtual lifting spreader's sway behavior stays synchronized with the actual spreader in the physical world, with error sources fully transparent—mechanical clearances, flexible deformation, and sensor noise can each be independently superimposed, providing a controlled simulation sandbox for algorithm R&D.

The digital twin platform is deeply coupled with the five technologies described above: the reinforcement learning anti-sway controller's training environment directly invokes the twin model's physics engine; the visual AI's annotated data is augmented through the twin space's automatic annotation module (synthetic data augmentation); and predictive maintenance fault samples are also generated in the twin space—simulating sensor response curves under conditions such as bearing inner ring cracks and gear tooth breakage, dramatically reducing reliance on real-world fault data.

On the operations management side, the digital twin platform also serves as a "smart command center." Managers can view the entire workshop's real-time operational status in a single 3D view: each crane's current task progress, predicted remaining maintenance time, energy consumption heatmaps, congestion hotspot areas, and AI-recommended adjustments. The system has completed 18 months of trial operation at Kelude's own factory, during which overall equipment effectiveness (OEE) improved by 14%.

Another key capability of the digital twin platform is its "replay" and "look-ahead" modes. Replay mode allows managers to select any historical time window and replay the motion trajectories and operational events of all cranes during that period, supporting accident investigation and efficiency reviews. Look-ahead mode enables "what-if" analysis—managers can create hypothetical scenarios in the twin space (e.g., adding a crane, adjusting workshop layout, or changing task scheduling strategies), and the system runs simulations to output comparative analysis of key indicators such as production capacity, energy consumption, and congestion probability, providing quantitative evidence for management decisions.

Technology R&D System

The successful implementation of these six core technologies—rather than remaining theoretical concepts—is underpinned by a complete R&D system. Kelude Heavy Industry's technical team spans four disciplines: algorithm engineering, embedded development, mechanical design, and industrial software, following a "simulation-first, scenario-validated, iterative refinement" development methodology.

The typical R&D cycle for a new algorithm is: week one involves building simulation scenarios and baseline comparisons (against conventional approaches) in the digital twin environment; weeks two through four focus on algorithm iteration and parameter tuning; week five deploys the algorithm to laboratory test benches for small-scale validation; and weeks six through eight move to on-site factory trial runs. Each phase has clear pass/fail gates—algorithms that don't meet criteria do not advance to the next stage, ensuring stability upon deployment. The design and verification of safety-related parts of control systems follow the ISO 13849-1:2023—Safety of machinery—Safety-related parts of control systems standard, using the performance level (PL) assessment method to quantitatively grade safety functions.

Kelude has also established joint laboratories with the automation and computer science departments of three domestic universities, conducting frontier research at the fundamental algorithm level. To date, 47 patents related to intelligent crane control systems have been filed or granted, including 22 invention patents, spanning four major technology clusters: anti-sway control algorithms, vision-based positioning methods, fault diagnosis models, and dispatching optimization strategies.

In managing code and algorithmic assets, Kelude has established a unified AI Model Registry. Every trained model is logged with a standardized assessment report—including metadata such as accuracy, latency, power consumption, and training data coverage—before being reviewed and approved by a committee for field deployment. This process ensures end-to-end quality trust from algorithm development to real-world operation: every model running on-site can be traced back to its complete training history and evaluation data within the Digital Twin environment. The platform's overall functional safety architecture complies with the IEC 61508—Functional Safety of Electrical/Electronic/Programmable Electronic Safety-Related Systems standard, covering the entire safety life cycle from sensor data acquisition to control command execution.

From traditional manual equipment—such as the LXS manual single-girder suspension crane—to today's intelligent unmanned systems, Kelude Heavy Industry has both witnessed and driven the technological leap across the crane industry. Returning to the question posed at the outset: what exactly is Kelude's technical moat? It is not the sophistication of any single algorithm, but rather the systemic capability of six core technologies that reinforce and operate in concert with one another. Without the simulation environment provided by Digital Twin, reinforcement learning training efficiency would drop significantly; without the high-precision positioning delivered by vision AI, the execution accuracy of scheduling optimization would be unattainable; without the equipment health data supplied by Predictive Maintenance, the simulation boundaries of Digital Twin would be severely distorted. These six capabilities form an interconnected, mutually reinforcing technology network—that is what makes Kelude the hardest to replicate.

Traditional vs. AI-Powered Solutions: A Technical Comparison

The following comparison highlights key differences between conventional engineering approaches and Kelude's AI-driven solutions across five core technology dimensions, offering a clear view of the performance gains achieved through this upgrade.

This comparison table clearly shows that while conventional solutions can hold up under a single, fixed operating condition, AI-driven approaches deliver a generational leap across accuracy, robustness, cost, and efficiency when faced with the dynamic, uncertain, and multi-objective constraints of real industrial environments. Kelude systematically integrates these AI capabilities to build a complete technology loop—from modular cranes to intelligent systems.

FAQ