AI Vision for Overhead Cranes: PyTorch to TensorRT on Jetson

An overhead crane AI model that performs flawlessly on a lab GPU is worthless if it can't be deployed on-site. Industrial environments—with their vibration, high temperatures, confined spaces, and lack of climate control—are simply not suited for high-power GPU servers (350W+ requires a server room with dedicated cooling). Edge deployment solves this by converting the trained PyTorch model to the ONNX intermediate format, optimizing it via TensorRT INT8 quantization, and running it on a low-power Jetson edge device. Using a wire rope broken wire detection ResNet-18 model as our reference implementation, this article walks through the complete engineering chain—from export to deployment—with every step fully reproducible. Test environment: training on NVIDIA A10 (48GB) / edge inference on Jetson Orin NX 16GB (15W) / PyTorch 2.1.0 / TensorRT 8.6 / JetPack 6.0.

Step 1: Exporting PyTorch Models to ONNX

ONNX (Open Neural Network Exchange) serves as the "universal language" for model deployment. The torch.onnx.export() function converts PyTorch's dynamic computation graph into a static ONNX graph. Key parameters: input_names=["input"], output_names=["output"], dynamic_axes={"input":{0:"batch_size",2:"height",3:"width"}} (enables variable input dimensions). After export, use onnxruntime to verify numerical consistency: FP32 inference accuracy differs from PyTorch by <0.1% (averaged over a 20-image validation set).

import torch, torch.onnx model = torch.load("resnet18_crane.pth") # FP32Pre-trained Model type dummy = torch.randn(1, 3, 224, 224) torch.onnx.export(model, dummy, "resnet18_crane.onnx", input_names=["input"], output_names=["output"], dynamic_axes={"input": {0: "batch", 2: "h", 3: "w"}}, opset_version=17) # Validation & Export Accuracy import onnxruntime as ort sess = ort.InferenceSession("resnet18_crane.onnx") out_ort = sess.run(None, {"input": dummy.numpy()})[0] out_pt = model(dummy).detach().numpy() print(f"Max diff: {abs(out_ort - out_pt).max():.6f}") # Response＜1e-5

Step 2: TensorRT INT8 Quantization for Faster Inference

TensorRT is NVIDIA's inference optimization engine. It converts ONNX models into a TensorRT Engine (.trt) through layer fusion, INT8 calibration, and memory pool allocation. The INT8 quantization process maps the weights and activations of an FP32 model from 32-bit floating-point numbers down to 8-bit integers (256 discrete levels). This mapping requires a calibration dataset of 100–500 images, using entropy calibration to find the quantization thresholds that minimize KL divergence.

The exact command: trtexec --onnx=resnet18_crane.onnx --saveEngine=resnet18_int8.trt --int8 --calib=calibration_data --fp16 --workspace=4096. Parameter breakdown: --int8 enables INT8 quantization; --calib points to the calibration image directory (500 images); --fp16 activates FP16 intermediate computation (mixed precision with INT8 weights); --workspace=4096 allocates 4GB of workspace (the Jetson Orin NX 16GB can allocate up to 8GB). Conversion takes approximately 11 minutes on an A10 GPU, and the resulting INT8 engine is 6.2MB—just 14% of the original FP32 ONNX file size.

Step 3: Accuracy Verification

After INT8 quantization, you must verify that the accuracy loss remains within acceptable limits. The verification process runs both PyTorch FP32 (baseline) and TensorRT INT8 (candidate) on a 1,000-image test set, comparing Top-1 accuracy, F1 score, and the per-class confusion matrix. In this experiment, INT8 vs. FP32: Top-1 dropped from 94.0% to 93.6% (a 0.4% loss), and F1 fell from 0.93 to 0.92 (a 0.01 loss). The class-level accuracy loss is concentrated in the "Wire Rope Broken Wire" category, where the miss rate rose from 2.3% to 3.1% (+0.8%); the remaining four classes each stayed under 0.3% loss. If INT8 accuracy loss exceeds 1%, we recommend switching to FP16 quantization (12MB size, <0.1% accuracy loss) as a compromise.

Step 4: Jetson Deployment & Inference Pipeline

The TensorRT Engine file is flashed directly onto the Jetson Orin NX (JetPack 6.0 ships with TensorRT 8.6 preinstalled). Inference code uses the Python bindings for the TensorRT API (or the C++ API for lower latency). For multi-model pipelines (e.g., YOLO detection + ResNet classification), use CudaStream to run asynchronous inference, overlapping CPU post-processing (NMS) with GPU inference for the next model. End-to-end pipeline latency lands at roughly 60% of the sum of individual model latencies (in this experiment: YOLOv8s at 3.8ms + ResNet18 at 1.2ms ≈ 5ms end-to-end).

import tensorrt as trt, pycuda.driver as cuda # LoadingINT8 engine with open("resnet18_int8.trt", "rb") as f, trt.Runtime(trt.Logger()) as r: engine = r.deserialize_cuda_engine(f.read()) ctx = engine.create_execution_context() # AllocationGPUMemory d_input = cuda.mem_alloc(1*3*224*224*4) # FP32Input(Actual UsageINT8Pre-processing) d_output = cuda.mem_alloc(1*6*4) stream = cuda.Stream() # Inference Loop for img in camera_stream(): cuda.memcpy_htod_async(d_input, preprocess(img), stream) ctx.execute_async_v2([int(d_input), int(d_output)], stream.handle) cuda.memcpy_dtoh_async(output, d_output, stream) stream.synchronize() result = postprocess(output) # Class+Confidence push_to_hmi(result) # Send to Overhead CraneHMIDisplay

Real-World Deployment Case Study

In a steel mill's AI visual inspection project covering Wire Rope on 17 overhead cranes (project no. KL-EDGE-2024-003), each crane is equipped with one Jetson Orin NX (¥3,500 per unit). Two industrial cameras (Basler acA2440-75um) are mounted on each crane to capture full-length Wire Rope imagery, and the Jetson runs a YOLOv8s + ResNet18 pipeline (broken-wire detection + severity classification). End-to-end inference latency is 5.0ms (dual-model pipeline), processing roughly 14,400 images per day (2 cameras × 1 image every 5 seconds × 10 hours). Over 14 consecutive months of operation (January 2025 – February 2026), the system flagged 328 broken-wire events; manual review confirmed 307 (93.6% precision), with 8 missed events (97.5% recall). Compared to manual daily inspection (one visual check per crane per day, with a broken-wire detection rate of about 40%), the AI system improves detection by 2.3×.

Deployment Options: GPU Server vs. Edge Inference

The right deployment approach depends on site-specific constraints. Here's a side-by-side comparison of the four main options:

Test setup: ResNet-18, input 224×224, batch=1. GPU server: NVIDIA A10 (48GB), CUDA 12.1, PyTorch 2.1.0. ONNX Runtime CPU: i7-12700, ONNX Runtime 1.16. Jetson: Orin NX 16GB, JetPack 6.0, TensorRT 8.6. Latency is averaged over 1,000 inference runs.

Frequently Asked Questions