Nazr-CNN: Object Detection and Fine-Grained Classification in Crowdsourced UAV Images

N. Attari, F. Ofli, M. Awad, J. Lucas, S. Chawla

Published 2016-11-20Version 1

We propose Nazr-CNN, a deep learning pipeline for object detection and fine-grained classification in images acquired from Unmanned Aerial Vehicles (UAVs). The UAVs were deployed in the Island of Vanuatu to assess damage in the aftermath of cyclone PAM in 2015. The images were labeled by a crowdsourcing effort and the labeling categories consisted of fine-grained levels of damage to built structures. Nazr-CNN consists of two components. The function of the first component is to localize objects (e.g. houses) in an image by carrying out a pixel-level classification. In the second component, a hidden layer of a Convolutional Neural Network (CNN) is used to encode Fisher Vectors (FV) of the segments generated from the first component in order to help discriminate between between different levels of damage. Since our data set is relatively small, a pre-trained network for pixel-level classification and FV encoding was used. Nazr-CNN attains promising results both for object detection and damage assessment suggesting that the integrated pipeline is robust in the face of small data sets and labeling errors by annotators. While the focus of Nazr-CNN is on assessment of UAV images in a post-disaster scenario, our solution is general and can be applied in many diverse settings.