Geometric Multi-Model Extraction For Robotics

//Geometric Multi-Model Extraction For Robotics
Load More Posts

The extraction of geometric models has long been of interest to the robotics community. Many interesting applications such as homography,plane estimation and ego-motion estimation demand the ability to fit geometric models onto noisy data. Additionally, these geometric models are often drivers of other algorithms for algorithms  navigation, perception and 3D reconstructions creating an increased desire for algorithms that are not only robust to contamination but exhibit quick and repeatable run-times.

In this blog post,  we’ll take a look at a fast parallel technique[1] for multi-model geometric extraction that has been developed within the Institute together with a proposed application in the improvement of stereo depth maps[2].

Convex Relaxation Algorithm (CORAL)

Many traditional approaches to geometric multi-model fitting tend to greedily maximise the inliers to the proposed model, whether through clustering or sampling based approaches. These, while often producing good results show a strong deterioration in performance in highly cluttered environments, which unfortunately characterise the workspace of most robotics systems.

In contrast  global energy optimisation approaches offer a principled way of jointly fitting all models present in multi-structural data. This is  by not only accounting for the model error as in the traditional techniques, but encouraging spatial smoothness and compactness in the number of models through regularisation term. This transforms the geometric multi-model fitting problem  into a multi-labelling problem where the optimal solution can be obtained through an energy minimisation.

In [1] we offer a Convex Relaxation Algorithm (CORAL) that performs the energy minimisation through a first-order primal-dual optimisation. As this supports per-element evaluation it facilitates a parallel implementation on General Purpose Graphical Processing Unit (GPGPU) hardware. Enabling fast and accurate real-time evaluation on a multitude of problems as shown below.


(a) Motion Segmentation                                                                                                    (b) Cubic Road Boundary Detection


(c) Line Road Marking Detection                                                                                      (d) Multi-Homography Detection

Depth-Map Improvement

Dense depth-maps created from monocular/stereo cameras offer a low-cost solution that can naturally deal with both the scale and lighting conditions of outdoor environments. While current state-of-the-art methods for creating dense depth-maps with cameras are based on powerful variational optimisation algorithms, these struggle  in dealing with large, plain and planar structures. With urban scenes characterised by a multitude of man-made objects such as roads and buildings which by construction contain planar surfaces this results in noisy and sometimes erroneous depth-maps.

In [2] we present a vision-only pipeline that uses CORAL to quickly and accurately extracts planar priors through homographies. Then follows this with a subsequent optimisation, using CORAL, to obtain a per-pixel labelling that allocates pixels to the extracted planar priors resulting in more accurate depth-map estimation as shown.


(a) Homography Detection                                                                                                       (b) Fast Per pixel Labelling

Depth Map from Total Generalised Variation                                                                      Depth Map after Prior Improvement

It can be seen that the introduction of the planar prior creates a more visually-accurate depth-map. Coupled with the speed of CORAL execution a real-time solution for improving depth-maps can be thus obtained.


[1] P.Amayo, P. Piniés, L. Paz, and P. Newman. “Geometric Multi-Model Fitting with a Convex Relaxation Algorithm”. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, USA, June 2018
[2] P. Amayo, P. Piniés, L. Paz, and P. Newman. “Fast Global Labelling For Depth-Map Improvement Via Architectural Priors”. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, May 2018.