AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning

Xijun Wang*, Ruiqi Xian*, Tianrui Guan, Celso M. de Melo, Stephen M. Nogar, Aniket Bera, Dinesh Manocha

Published in International Conference on Robotics and Automation, 2023


We propose a novel approach for aerial video action recognition. Our method is designed for videos captured using UAVs and can run on edge or mobile devices. We present a learning-based approach that uses customized auto zoom to automatically identify the human target and scale it appropriately. This makes it easier to extract the key features and reduces the computational overhead. We also present an efficient temporal reasoning algorithm to capture the action information along the spatial and temporal domains within a controllable computational cost. Our approach has been implemented and evaluated both on the desktop with high-end GPUs and on the low power Robotics RB5 Platform for robots and drones. In practice, we achieve 6.1-7.4% improvement over SOTA in Top-1 accuracy on the RoCoG-v2 dataset, 8.3-10.4% improvement on the UAV-Human dataset and 3.2% improvement on the Drone Action dataset.

The paper is available here. Please cite our work if you found it useful,

  author={Wang, Xijun and Xian, Ruiqi and Guan, Tianrui and de Melo, Celso M. and Nogar, Stephen M. and Bera, Aniket and Manocha, Dinesh},
  booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)}, 
  title={AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning},