Registration of 3D scans with hyperspectral scans in the context of environmental applications

Finished
Songül Polat with her research in the hyperspectral laboratory Vanessa Liebler for the i3mainz, CC BY SA 4.0

The combination of hyperspectral and geometric data is of great interest for many fields of application, notably building monitoring, agriculture, environmental issues, historical building preservation and process control in production.

Motivation

Songül Polat is researching the linkage of hyperspectral and geometric data for her doctoral dissertation. While geometric data describes morphological properties, hyperspectral data provides information about material properties, allowing a much more precise analysis of objects. The combination of this data can add value, especially for environmental applications and the energy industry, and enable us to obtain a more comprehensive picture of the world and its resources.

The hyperspectral laboratory that has been set up for the research includes the following systems:

  • Laboratory equipment consisting of 2 line scan cameras to cover the spectral range from 400 - 1700 nm.

    • Specim FX 10 for the VIS range (400 - 1000 nm)

      • 1024 pixels per line
      • spectral resolution 5.5 nm

    • Specim FX 17 for the NIR range (900 - 1700 nm)

      • 640 pixels per line
      • spectral resolution 8 nm

  • A mobile system is available for outdoor measurements.

    • Senop HSC-2 for the VIS range (500 - 900 nm

      • Snapshot camera / area sensing system
      • Designed for drone applications
      • 1 MPix spatial resolution
      • Spectral resolution < 2 nm

Activities

The project “Registration of 3D scans with hyperspectral scans in the context of context of environmental applications” in cooperation with V&R Vision und Robotics GmbH is supervised by Prof. Dr. Alain Trémeau, Université Jean Monnet in Saint-Étienne and by Prof. Dr. Frank Boochs. Funding for the cooperative doctorate comes from the European Regional Development Fund (EFRE), and the Ministry of Science, Continuing Education and Culture of Rhineland-Palatinate through its InnoProm - Innovation and Promotion funding program.

Results

The rapid developments in hyperspectral imaging give rise to new opportunities for a better understanding of the physical aspects of materials and scenes in a variety of applications due to their high spatial and spectral resolution, while 3D technologies help to understand scenes in more detail by exploiting geometric, topological and depth information. The research in this thesis aims at the combined use of 3D and hyperspectral data, and demonstrates the potential and added value of a combined approach through a variety of applications. Special attention is given to the identification and extraction of features in both domains and the use of these features to detect objects of interest.

In particular, we propose different approaches to combining 3D and hyperspectral data depending on the HSI/3D technologies used, and show how each sensor can compensate for the weaknesses of the other.

In addition, a new shape- and rule-based method for spectral signature analysis was developed and presented. The strengths and weaknesses compared to existing approaches are discussed, and the superiority over SVM methods is demonstrated using practical findings from the cultural heritage and waste management fields.

In addition, a newly developed analysis method based on 3D and hyperspectral features is presented. The evaluation of this methodology is based on a practical example from the WEEE field, and focuses on the separation of materials such as plastics, printed circuit boards and electronic components on PCBs. The results obtained confirm that an improvement in classification results was achieved compared to previously proposed methods. The aim of the individual methods and procedures developed in this work is generality and easy transferability to any area of application.