Learning-Based Excavator Automation

Abstract

This dissertation addresses the automation of hydraulic excavators, specifically, the modeling and controlling of the highly nonlinear machine dynamics and the interaction with the soil during excavation operations. Despite continuing efforts by the research community over the past decades, the automation of heavy-duty equipment is only slowly transitioning into real-world applications. The goal of this thesis is to accelerate this process by facilitating the automation process and extending the capabilities of autonomous excavators. Thereby, ML-based methods are leveraged to model and control excavators for accurate bucket control and efficient soil excavation. Many tasks, such as manipulating stones and trees or grading (surface leveling), require precise and accurate bucket trajectory tracking. Unlike traditional control methods, which rely on accurate modeling and laborious hand-tuning, we propose a data-driven approach to model and control the excavator. Rather than requiring an analytical model of the system, a neural-network model is used that is trained on data collected during operation of the machine. The data-driven model effectively represents the actuator dynamics, including the cylinder-to-joint-space conversion. Requiring only knowledge about the distances between the individual joints, a simulation is set up to train control policies using RL. The policy outputs pilot stage control commands that can be directly applied to the machine without further fine-tuning or unfounded filtering. In a first step towards RL-based excavator automation, the policy is trained to track randomized position targets. For deployment, the position target is continuously updated to track desired trajectories. The results demonstrate the feasibility of directly applying control policies trained in simulation to the physical excavator for accurate and stable position tracking. However, due to the position control paradigm, the controller always required an offset to the desired trajectory point in order to move, leading to larger trajectory tracking errors. Also, the orientation of the bucket is not considered, which limits its practical utility. To improve the shortcomings of this approach, the control paradigm was changed to account for bucket velocities and include the orientation. With these modifications, the trajectory tracking performance was improved significantly. Compared to a commercial grading controller, which requires laborious hand-tuning by expert engineers, the learned controller shows higher tracking accuracy, indicating that the achieved performance is sufficient for practical application on construction sites. Besides accurate trajectory tracking, one of the most fundamental tasks for an excavator is to excavate soil efficiently. Soil properties are hard to predict and can vary even within one scoop, which requires a controller that can adapt online to the encountered soil conditions. The objective is to fill the bucket with excavation material while respecting machine limitations to prevent stalling or lifting of the machine. To this end, we train a control policy in simulation using RL. The soil interactions are modeled based on the FEE with heavily randomized soil parameters to expose the agent to a wide range of different conditions. The agent learns to output joint velocity commands, which can be directly applied to the standard proportional valves of the real machine. The experiments demonstrate that the controller can adapt online to changing conditions without the explicit knowledge of the soil parameters, solely from proprioceptive observations assuming flat ground. The capabilities of this controller are then extended to take into account the current terrain elevation and adhere to a maximum-depth constraint to achieve a desired excavation design. The controller is integrated into an autonomous excavation planning system to excavate a complete trench. The experiments demonstrate that the controller can robustly adapt the excavation trajectory based on the encountered conditions and shows competitive performance compared to a professional machine operator.