O. Pizarro, S.B. Williams, M.V. Jakuba, M. Johnson-Roberson, I. Mahon, M. Bryson, D. Steinberg, A.Friedman, D. Dansereau, N. Nourani-Vatani, D. Bongiorno, M. Bewley, A. Bender, N. Ashan, B. Douillard
Underwater Technology Symposium
Australias Integrated Marine Observing System
(IMOS) has a strategic focus on the impact of major boundary
currents on continental shelf environments, ecosystems and
biodiversity. To improve our understanding of natural, climate
change, and human-induced variability in shelf environments, the
IMOS Autonomous Underwater Vehicle (AUV) facility has been
charged with generating physical and biological observations
of benthic variables that cannot be cost-effectively obtained
by other means. Starting in 2010, the IMOS AUV facility
began collecting precisely navigated benthic imagery using AUVs
at selected reference sites on Australia's shelf. This observing
program capitalizes on the unique capabilities of AUVs that
have allowed repeated visits to the reference sites, providing a
critical observational link between oceanographic and benthic
processes. This paper provides a brief overview of the relevant
capabilities of the AUV facility, the design of the IMOS benthic
sampling program, and some preliminary results. We also report
on some of the challenges and potential benefits to be realized
from a benthic observation system that collects several TB of
geo-referenced stereo imagery a year. This includes collaborative
semi-automated image analysis, clustering and classification,
large scale visualization and data mining, and lighting correction
for change detection and characterization. We also mention some
of the lessons from operating an AUV-based monitoring program
and future work in this area.
N. Nourani-Vatani, M. De Deuge, B. Douillard, S. Williams
ICCV 2013 Workshop on Underwater Vision
While the importance of the choice of color space for color descriptors has been studied extensively, a similar study for image texture descriptors is missing. This publication investigates the effect of color-to-monochrome conversions, image normalization, and metrics on the discriminative power of texture descriptors. The measure of the discriminative power of a feature is formulated as supervised spectral feature analysis. This analysis allows to measure the relative performance of a feature under varying conditions as long as the feature dimension is maintained. Feature discrimination evaluation is applied to Local Binary Patterns texture descriptors and it is shown how the proposed metric directly maps to classification performance. Based on this metric, we demonstrate that the choice of color-to-monochrome conversion and normalization can have a significant effect on the performance of the LBP descriptors.
M.S. Bewley, N. Nourani-Vatani, D. Rao, B. Douillard, O. Pizarro, S.B. Williams
Field and Service Robotics
In recent years, Autonomous Underwater Vehicles (AUVs) have been used extensively to gather imagery and other environmental variables for ocean monitoring. Processing of this vast amount of collected imagery to label content is difficult, expensive and time consuming. Because of this, typically only a small subset of images are labelled, and only at a small number of points. In order to make full use of the raw data returned from the AUV, this process needs to be automated. In this work the single species classification problem is extended to a multi-species classification problem following a taxonomical hierarchy. We demonstrate the application of techniques used in areas such as computer vision, text classification and medical diagnosis to the supervised hierarchical classification of benthic images. We also discuss critical aspects such as training topology and various prediction and scoring methodologies. An interesting aspect of the presented work is that the ground truth labels are sparse and incomplete, i.e. not all labels go to the leaf node, which brings with it other interesting challenges. We find that the best classification results are obtained using Local Binary Patterns (LBP), training a network of binary classifiers with probabilistic output, and applying “one-vs-rest” classification at each level of the hierarchy for prediction. This work presents a working solution that allows AUV images to be automatically labelled with the most appropriate node in a hierarchy of 19 species and morphologies. The result is that the output of the AUV system can include a semantic map using the taxonomy prescribed by marine scientists. This has the potential to not only reduce the manual labelling workload, but also to reduce the current dependence that marine scientists have on extrapolating information from a relatively small number of sparsely labelled points.
A method for segmenting three-dimensional data of underwater unstructured terrains is presented. The three-dimensional point clouds are converted to two-dimensional elevation maps and analyzed for seg- mentation in the frequency domain. The lower frequency components represent the slower varying undulations of the underlying ground. The cut-off frequency, below which the frequency components form the ground sur- face, is determined automatically using peak detection. The user can also specify a maximum admissible size of objects to drive the automatic detection of the cut-off frequency. The points above the estimated ground sur- face are clustered via standard proximity clustering to form object segments. The precision of the segmenta- tion is compared against ground truth hand labelled data acquired by a stereo camera pair and a struc- tured light sensor. It is also evaluated for registration error when the extracted segments are used for sub- map alignment. The proposed approach is compared to three point cloud based and two image based segmen- tation algorithms. The results show that the approach is applicable to a range of different terrains and is able to generate features useful for navigation.
Publisher's version Springer: http://link.springer.com/article/10.1007%2Fs10514-013-9353-0 DIO: 10.1007/s10514-013-9353-0
N Nourani-Vatani, P VK Borges, J M Roberts, M V Srinivasan
Journal of Intelligent and Robotic Systems: Volume 74, Issue 3 (2014), Page 817-846
We propose the use of optical flow information as a method for detecting and describing changes in the environment, from the perspective of a mobile camera. We analyze the characteristics of the optical flow signal and demonstrate how robust flow vectors can be generated and used for the detection of depth discontinuities and appearance changes at key locations. To successfully achieve this task, a full discussion on camera positioning, distortion compensation, noise filtering, and parameter estimation is presented. We then extract statistical attributes from the flow signal to describe the location of the scene changes. We also employ clustering and dominant shape of vectors to increase the descriptiveness. Once a database of nodes (where a node is a detected scene change) and their corresponding flow features is created, matching can be performed whenever nodes are encountered, such that topological localization can be achieved. We retrieve the most likely node according to the Mahalanobis and Chi-square distances between the current frame and the database. The results illustrate the applicability of the technique for detecting and describing scene changes in diverse lighting conditions, considering indoor and outdoor environments and different robot platforms.
Australasian Conference of Robotics and Automation, Wellington, New Zealand
Sparse optical flow algorithms, such as the Lucas-Kanade approach, provide more robustness to noise than dense optical flow algorithms and are the preferred approach in many scenarios. Sparse optical flow algorithms estimate the displacement for a selected number of pixels in the image. These pixels can be chosen randomly. However, pixels in regions with more variance between the neighbors will produce more reliable displacement estimates. The selected pixel locations should therefore be chosen wisely. In this study, the suitability of Harris corners, Shi-Tomasi’s “Good features to track”, SIFT and SURF interest point extractors, Canny edges, and random pixel selection for the purpose of frame-by-frame tracking using a pyramidical Lucas-Kanade algorithm is investigated. The evaluation considers the important factors of processing time, feature count, and feature trackability in indoor and outdoor scenarios using ground vehicles and unmanned areal vehicles, and for the purpose of visual odometry estimation.
M.S. Bewley, B. Douillard, N. Nourani-Vatani, A. Friedman, O. Pizarro, S.B. Williams
Australasian Conference of Robotics and Automation, Wellington, New Zealand
This paper presents an experimental study of automated species detection systems suitable for use with Autonomous Underwater Vehicle (AUV) data. The automated detection systems presented in this paper use supervised learning; a support vector machine and local image features are used to predict the presence or ab- sence of Ecklonia Radiata (kelp) in sea floor images. A comparison study was done using a variety of descriptors (such as local binary patterns and principal component analysis) and image scales. The performance was tested on a large data set of images from 14 AUV missions, with more than 60,000 expert labelled points. The best performing model was then analysed in greater detail, to estimate performance on generalising to unseen AUV missions, and char- acterise errors that may impact the utility of the species detection system for marine scientists.