PointDrive : a point-based self-driving policy

Vision-based urban driving is hard since the image representation learned solely from action supervision cannot generalize well to new scenarios. Is there a better representation of the traffic scenarios? We propose PointDrive, a simple driving policy conditioned on feature vectors of nearby agents. With semantic points representation, we can perform realistic and efficient data augmentations to alleviate distributional shift. Also, this point representation is easy to get from existing computer vision algorithms. Experiments show that PointDrive substantially outperforms other offline methods on the CARLA benchmark. Furthermore, we perform extensive ablation studies and show that the current bottleneck of our system is in the perception module, suggesting better model design or learning algorithm is needed to produce a robust vision model for autonomous driving.