Retourner à la liste

Apprentissage profond de séries temporelles de nuages de points 3D

Description

Context

Observing the earth surface from space and airborne platforms using remote sensing (RS) technologies plays a significant role in studying the planet's physical and biological systems. In the last decade, the number of earth observation (EO) missions has been increased dramatically, providing a huge amount of RS data including time series of 3D point clouds acquired from LiDAR scanners or stereo reconstruction. Thanks to the ability to provide flexible and scalable geometric representation of the observed scene, 3D point clouds become prospective for modern EO applications such as 3D reconstruction, updating and monitoring of urban, agricultural and forestry areas. Future EO missions such as CNES Co3D will increase the potential impact of 3D point clouds but also the need for efficient data processing and analysis methods.

The analysis and processing both of 3D point cloud and temporal data is currently an active research topic which has attracted many researchers from different application domains. As for the description of 3D point cloud, while handcrafted features are still struggling to deal with 3D structures, the recent overwhelming success of convolutional neural networks (CNNs) for image analysis reveals a huge potential of deep learning frameworks adapted to 3D point clouds. However such adaptation is not straightforward. Some studies have attempted to convert irregular 3D point clouds to regular domains such as a voxel grid (Voxnet [1], SegCloud [2]) or a set of virtual 2D image snapshots (SnapNet [3]) before applying regular CNNs. However, such approaches may not capture the intrinsic structures of point cloud data and hence, their performance remains limited due to information loss at fine ne details. Others methods have been specifically designed to handle the irregularity of point clouds in order to directly manipulate raw 3D data, pioneered by PointNet [4] and its extensions [5, 6]. Nevertheless, their negligence of the geometric relationships among points still involves a fundamental limitation of local feature missing. Regarding time series, Recurrent Neural Networks (RNN) has been successfully used to cope with temporal data, and several attempts have been made to apply them in remote sensing [7, 8]. 3D points clouds have received a particular attention in this 1 context [9, 10], due to the increasing interest in autonomous driving. However, such first works call for further understanding and development.

Graph structures have been proved to be efficient for representation and modeling of temporal and irregular data including 3D point clouds. A graph can be constructed from raw point cloud data to encode the intrinsic inter-connection and geometrical relation among points. Recent developments of signal processing on graphs [11] and geometric deep learning on graphs [12] provide promising tools to perform intelligent representation learning of 3D point clouds. Deep CNNs on graphs are among the active research topics in computer vision with an increasing number of works in the last couple of years [13, 14, 15] tackling 3D object recognition and scene segmentation at reduced scale. Therefore, the investigation and adaptation of graph-based deep networks specifically developed for large-scale 3D point cloud data within remote sensing field, with an extension to time series, will lead to significant breakthroughs in the field.

 

Objectives

The main objective of this PhD project is to develop scalable deep learning frameworks applied to large-scale 3D point clouds with an extension to time series of such data. A particular attention will be given to develop deep neural networks on graph structures efficiently constructed from 3D data, including both stereo reconstruction and LiDAR point clouds. In terms of applications, the developed frameworks are expected to achieve convincing performance for both unsupervised and supervised tasks including 3D scene reconstruction, semantic segmentation at large-scale, object detection as well as change detection and object tracking from time series.

Work program

In order to address the aforementioned objectives, a tentative work program is given below.

1. Deep learning on 3D points clouds 

(a) Study of existing deep frameworks dedicated to 3D data (currently mostly focusing on indoor environments);

(b) Study of existing deep frameworks on irregular domains and structured representations (e.g. graphs) that allows to describe and manage the point clouds;

(c) Design of new deep architectures dedicated to unsupervised or supervised analysis of 3D data;

(d) Comparison with existing methods applied on prior rasterized data;

(e) Validation of the proposed methods on various remotely-sensed datasets, either provided by CNES (temporal multi-angular Pleiades imagery especially in urban environments, simulation of CO3D data) or by OSUR { Observatory for Universe Sciences, Rennes  (airborne and 2 terrestrial LiDAR of geophysical flows); furthermore, a specific attention will be given to participation in international contests such as the yearly IEEE Data Fusion Contest;

2. Extension to time series of 3D points clouds

(a) Study of existing deep frameworks dedicated to time series, especially those coupling spatial and temporal dimensions;

(b) Extension of the solutions proposed in the first part to embed time information;

(c) Validation on datasets provided by CNES and OSUR.

 

Environment

The OBELIX research group (www.irisa.fr/obelix) from IRISA (UMR 6074) is located in Vannes (UBS) and Rennes (through strong connection with the UMR LETG and OSUR). It is composed of ca. 20 researchers (8 permanents researchers, 3 postdocs, 10+ PhD students). Its research activities focus on machine learning and image processing for remote sensing of the environment. The group has a strong expertise in deep learning frameworks applied to Earth Observation, with several ongoing PhD on this topic (e.g. Nicolas Audebert with ONERA, Ahmed Nassar with ETHZ, etc.), postdocs (DeepTree and DeepOT projects), and collaborative projects (ANR Sesame, ANR DeepDetect). Its activities also include processing of 3D points clouds (PhD of Florent Guiotte, collaboration with Tellus Environment) and of time series data (PhD of Adeline Bailly, ANR MATS). The group is also hosting the GeoData Science track of the European Copernicus Master in Digital Earth.

IRISA (www.irisa.fr) is one of the main research institutes in computer science in France, with ca 700-800 members located in Brittany (Rennes, Lannion, Vannes, Brest). Université Bretagne Sud (UBS, www.univ-ubs.fr) serves as the hosting university, and is a recent, multidisciplinary university with ca. 9000+ students located in Vannes and Lorient (South of Brittany). Data science is one of the 4 topics identified as strategic for the university.

The PhD thesis will be jointly supervised by Prof. Sébastien Lefèvre 1, Full Professor in UBS Vannes, and Dr. Thomas Corpetti 2, Senior Scientist in CNRS { LETG Rennes. Dr. Minh-Tan Pham 3, currently postdoc in OBELIX, completes the supervision team.

 References

Maturana, D. and Scherer, S. Voxnet: A 3d convolutional neural network for realtime object recognition. In Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on (pp. 922-928)

Tchapmi, L., Choy, C., Armeni, I., Gwak, J. and Savarese, S. Segcloud: Semantic segmentation of 3d point clouds 3D Vision (3DV), 2017 International Conference on (pp. 537-547). IEEE.

Boulch, A., Guerry, J., Le Saux, B. and Audebert, N. 2018. SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks. Computers & Graphics, 71, pp.189-198

Qi, C.R., Su, H., Mo, K. and Guibas, L.J. Pointnet: Deep learning on point sets for 3d classification and segmentation. Proc. CVPR 2017.

Qi, C.R., Yi, L., Su, H. and Guibas, L.J. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In NIPS 2017 (pp. 5099-5108).

Hua, B.S., Tran, M.K. and Yeung, S.K. Pointwise convolutional neural networks. In CVPR 2018 (pp. 984-993).

Ienco, D., Gaetano, R., Dupaquier, C., Maurel, P., 2017. Land Cover Classification via Multi temporal Spatial Data by Deep Recurrent Neural Networks IEEE Geoscience and Remote Sensing Letters, 14(10), pp.1685-1689.

Russwurm, M., Korner, M., 2017. Temporal Vegetation Modelling using Long Short-Term Memory Networks for Crop Identi_cation from Medium-Resolution Multi-Spectral Satellite Images IEEE/ISPRS Workshop on Large Scale Computer Vision for Remote Sensing Imagery

Wu, B., Wan, A., Yue, X. and Keutzer, K., 2018, May. Squeezeseg: Convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3d lidar point cloud.In IEEE ICRA 2018 (pp. 1887-1893)

Sun, L., Yan, Z., Zaganidis, A., Zhao, C., Duckett, T., 2018, Oct. Recurrent-OctoMap: Learning State-Based Map Refinement for Long-Term Semantic Mapping With 3-D-Lidar Data. IEEE Robotics and Automation Letters, 3(4), pp.3749-3756.

Shuman, D.I., Narang, S.K., Frossard, P., Ortega, A. and Vandergheynst, P. 2013. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, 30(3), pp.83-98.

Bronstein, M.M., Bruna, J., LeCun, Y., Szlam, A. and Vandergheynst, P. 2017. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4), pp.18-42.

Defferrard, M., Bresson, X. and Vandergheynst, P. Convolutional neural networks on graphs with fast localized spectral filtering. In NIPS 2016 (pp. 3844-3852).

Monti, F., Boscaini, D., Masci, J., Rodola, E., Svoboda, J. and Bronstein, M.M. Geometric deep learning on graphs and manifolds using mixture model CNNs. In Proc.CVPR 2017.

Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M. and Solom

Profil

MSc or Engineering degree with excellent academic track and proven research experience in one of the following fields: computer science, applied maths, signal processing ;

Experience with deep learning ;

Interests for environment and earth observation applications ;

Skills and interest in coding (knowledge of python, and framework such as keras and/or Tensorow/Theano will be appreciated) ;

Excellent communication skills (spoken/written English) is required ;

Ability to work in a team.

Description de la structure
Laboratoire d'accueil : UMR 6074 IRISA (équipe OBELIX site de Vannes)
Directeur(rice) de thèse/recherche : Lefèvre Sébastien
E-mail du directeur(rice) de thèse/recherche : sebastien.lefevre@irisa.fr
Responsable Cnes de l'offre : LASSALLE Pierre

To apply, we invite you to contact the PhD/research supervisor and fill, with him/her, the co-financing part of the online application form (Reply to the offer) by April 1st, 2019.

Répondre à l'offre Retourner à la liste