Learning to Predict Pedestrians for Urban Automated Driving

Abstract

Navigating through densely populated urban areas is one of the most important challenges for self-driving vehicles. Accurate predictions are required to enable safe and efficient interactions with other road users. Pedestrians in particular pose major problems for current state of the art prediction systems. Apart from well-understood short-term predictions, used for Automatic Emergency Braking Systems, long-term predictions remain largely unresolved. In this thesis, we aim to advance pedestrian prediction systems to enable human- understandable automated driving. In view of a vehicle-centred road infrastructure with high vehicle speeds and scarce pedestrian crossings, early detection of pedestrian intentions and movements are the key to enabling such behaviour. These detections can enable automated vehicles to perform light brake manoeuvres at an early stage in order to let a pedestrian pass. This can eliminate the need to stop, which could significantly improve traffic flow and at the same time increase overall safety. Long-term full trajectory predictions are both costly and error-prone. Therefore we propose a hierarchical prediction system that splits the prediction into multiple simplified sub-problems using domain knowledge. Each sub-problem is designed to predict a meaningful part of pedestrian movement and to detect and remove pedestrians that are irrelevant for the current scenario as early as possible. Utilizing the given road geometry to identify crosswalks we first predict the pedestrians’ hidden intent to cross the road. For all crossing pedestrians we then propose a sparse motion prediction, providing a small set of key figures instead of a full trajectory. We claim that these domain-specific key figures, namely a time-to- cross and designated crossing point, are more than sufficient to describe future pedestrian motions for the planning system of an automated vehicle. To overcome problems from over-confident single value predictions we propose to utilize Quantile Regression techniques to predict reasonable uncertainties. Our evaluations show that we are able to robustly classify the pedestrians’ hidden intent using both standard and deep learning algorithms. Additionally we show that our hierarchical prediction system, including the sparse motion prediction, is suitable for a real-time system integration. With our large real-world dataset, featuring recordings from different crosswalks and days, we provide an evaluation regarding prediction accuracy, computational load and generalizability. During this analysis, we also found indications that it might be possible to transfer trained models to previously unseen pedestrian crossings if the road geometry has at least approximately the same pavement dimensions. Furthermore, we show how our sparse motion prediction can be integrated into a situation-based planning approach to allow safe and efficient real-time interactions with other traffic participants. For this we evaluate different interaction scenarios regarding safety, time efficiency and comfort impairment. We were able to show that in most of the scenarios it is possible to minimize movement jerks and eliminate the need to stop. The overall performance is only limited by very high traffic densities.