MULTISENSOR INTEGRATION
Real-time resource characterization.
OVERVIEW
Optical sensors have become key to most on- and in-line scanning applications in the field of material characterization, digitalization, and monitoring. They offer non-invasive sensing technologies to identify the composition of primary and secondary materials without time- and cost-consuming sample preparation and chemical analyses. Based on material-specific physical properties, such sensors provide huge advantages for a selected material group, but show limitations, when it comes to materials beyond that group, as well as for complex and/or unknown, variable material streams.
To innovate available technology, overcome limitations of individual sensors and expand the range of detectable materials, we focus on (1) the development of new, tailored sensor solutions and on (2) the integration of sensors into smart and agile networks. In particular, we combine reflectance hyperspectral imaging (HSI) sensors from VNIR to LWIR with emission spectroscopy using laser-induced fluorescence (LiF) to map material types and their abundances. We investigate further integration concepts for refining position information in 3D (RGB, laser profiler) for improved signal acquisition. We evaluate concepts on how resulting data can enable efficient control mechanisms and provide the guidance of subsequent sensors in automated analysis cycles for validation and updating mapping results. Developed concepts then allow to expand the toolkit of sensors (e.g. by Raman, XRF or LIBS) within the network according to the scope of individual research questions and application cases.
RESEARCH FOCUS
The HELIOS lab is a research infrastructure to explore the potentials and limits of optical sensors for raw material identification. The concept of HELIOS represents a combination of the EYES with the BRAIN. The EYES are our sensors to record selected physical parameters and the BRAIN provides an efficient and learning data processing architecture. Developments provide technical concepts for operational conditions and workflows for real-time data processing as a basis for process control and monitoring.
sensor development and integration for innovative concepts in material stream characterisation and digitalisation in real-time
-
Laser-induced fluorescence (LiF) line-can sensor integrated with HSI sensor, prototype (inSPECTor project) for drill cores
-
Raman sensor for point validation integrated within HSI and LiF sensor network, prototype (RAMSES-4-CE project)
in-line, spectroscopy-based sensor networks for hyperspectral reflectance (HSI), laser-induced fluorescence (LiF), Raman, x-ray fluorescence (XRF) and laser-induced breakdown spectroscopy (LIBS)
-
infrastructure for battery recycling (XRF, LIBS, robots, conveyor, GPU, InfraDatRec project)
-
high-performance computing (GPU, CirculAIre)
primary raw materials: esp. critical raw materials in drill cores and re-mining material:
-
rare-earth elements (REE) (inSPECTor project, RAMSES-4-CE project)
secondary raw materials: esp. complex, variable waste streams including valuable as well as contaminating components. Solutions explore material- and object-based approaches to meet the current and future challenges in recycling:
-
automated material detection in e-waste recycling (RAMSES-4-CE project)
-
automated material detection in battery recycling (DIGISORT & RegioREC & RESTORE project)
-
automated material detection in electrolyser recycling (AI4H2/ ReNaRe/H2Giga)
-
automated material detection in automotive recycling (Car2Car & KOLLEKT project)
-
automated detection of additives in polymer recycling (FINEST project)
RESEARCH PROJECTS
inSPECtor:
EIT RawMaterials upscaling project (2017 - 2020)
development of an integrated reflectance and laser-induced fluorescence sensor system for REE mapping in drill cores
partners: HZDR-HIF, TU Bergakademie Freiberg, Geological Survey of Finland
(GTK), Freiberg Instruments GmbH, Spectral Imaging Ltd. (SPECIM)
link: https://eitrawmaterials.eu/project/inspector/
publications:
IEEEsensors 2019:
Abend et al. 2019: Line-scan detection system to identify rare earth elements in rocks," 2019
IEEE SENSORS, Montreal, QC, Canada, 2019, pp. 1-3, doi: 10.1109/SENSORS43011.2019.8956697
https://ieeexplore.ieee.org/document/8956697
Sensors 2019:
Seidel et al. 2019: Fast 2D Laser-Induced Fluorescence Spectroscopy Mapping of Rare Earth Elements
in Rock Samples. Sensors 2019, 19, 2219. https://doi.org/10.3390/s19102219
www.mdpi.com/1424-8220/19/10/2219
PhysChemChemPhys 2019:
Sharma et al. 2019: Extending the temperature sensing range using Eu3+ luminescence up to 865 K
in a single crystal of EuPO4. Phys Chem Chem Phys. 2019 Jul 24;21(29):16329-16336.
doi: 10.1039/c9cp03501j. PMID: 31309210.
https://pubmed.ncbi.nlm.nih.gov/31309210/
Advanced Sensor Research 2026
Abend et al. 2026: A New Concept for an Integrated Mapping of Rare-Earth Elements using
Laser–Induced Fluorescence and Reflectance Spectroscopy.” Advanced Sensor Research5, no. 3
(2026): e00103. https://doi.org/10.1002/adsr.202500103
https://advanced.onlinelibrary.wiley.com/doi/full/10.1002/adsr.202500103
data:
REE spectral library LIF:
https://essd.copernicus.org/articles/13/4465/2021/ (paper)
https://zenodo.org/records/4054606 (data)
InfraDatRec:
BMBF infrastructure - setup of Data Mining Lab Freiberg (2021 - 2023)
improvement of data availability in the field of battery materials and technologies
partners: HZDR-HIF, TU Bergakademie Freiberg IAP, ITC and MVTAT, Fraunhofer
IKTS and IISB
DataMiningLabFreiberg: https://dmlf.de/
DIGISORT:
BMBF cluster project - recycling / green batteries (2021 - 2023)
development an optical sensor system for the rapid characterisation of battery
components
digitalisation of material stream for subsequent recycling processing decisions
partners: HZDR-HIF, TU Bergakademie Freiberg IAPand MVTAT
link: https://www.greenbatt-cluster.de/de/projekte/digisort/
publications:
IEEE Whispers 2022:
Chouhan et al. 2022: Hyperspectral Unmixing Using Convolutional Autoencoder For Metal Detection
In Lithium-Ion Battery Recycling Applications. 2022 12th Workshop on Hyperspectral Imaging and
Signal Processing: Evolution in Remote Sensing (WHISPERS), Rome, Italy, 2022, pp. 1-5,
doi: 10.1109/WHISPERS56178.2022.9955076
https://ieeexplore.ieee.org/document/9955076
MDPI Metals 2024:
Richter et al. 2024: Spectral Characterization of Battery Components from Li-Ion Battery Recycling
Processes. Metals 2024, 14, 147. https://doi.org/10.3390/met14020147
https://www.mdpi.com/2075-4701/14/2/147
RAMSES-4-CE:
EIT RawMaterials upscaling project (2020 - 2024)
development of Raman spectroscopy integrated with an reflectance and
laser-induced fluorescence sensor system and combined with advanced machine
learning for efficient material characterisation of e-waste
partners: HZDR-HIF, TU Bergakademie Freiberg, Geological Survey of Finland
(GTK), Freiberg Instruments
reading & publications:
data:
polymer spectral library:
https://rodare.hzdr.de/record/2448
PCB-Vision - A Multiscene RGB-Hyperspectral Benchmark Dataset of Printed Circuit Boards:
https://arxiv.org/abs/2401.06528 (paper)
https://rodare.hzdr.de/record/2704 (dataset)
https://github.com/hifexplo/PCBVision (codes)
H2GIGA (Subproject ReNaRe-AI4H2):
BMBF cluster project - sub-project: recycling - sustainable resource use (2021 - 2025)
development of an efficient sensor combination for the rapid characterisation of
electrolyser cells
digitalisation of material composition for subsequent recycling processing decisions
partners: KIT, FhG-IPA, HZDR-HIF, RWTH-IME, TU Bergakademie Freiberg
IKFVW, Nickelhütte Aue, Heraeus Deutschland GmbH&Co KG, OEKO,
DECHEMA, FZJ
links: https://www.wasserstoff-leitprojekte.de/projects/h2giga
press release: https://manage.wix.com/dashboard/f0a54a12-890c-4109-9e84-1f4c110d9137/blog/overview?referralInfo=sitesdropdown (in German)
publications:
IEEE IGARSS 2023:
Rasti et al. 2023: Hyperspectral Domain Adaptation for the Detection of Material Types in Recycling
Streams at the Example of Electrolyzers. IGARSS 2023 - 2023 IEEE International Geoscience and
Remote Sensing Symposium, Pasadena, CA, USA, 2023, pp. 7618-7620,
doi: 10.1109/IGARSS52108.2023.10282872.
https://ieeexplore.ieee.org/document/10282872
WHIPSPERS 2023:
Faruk et al. 2023: Deep Learning-Based Architectures For Multimodal Instance Segmentation. 2023
13th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing
(WHISPERS), Athens, Greece, 2023, pp. 1-5, doi: 10.1109/WHISPERS61460.2023.10431001.
https://ieeexplore.ieee.org/document/10431001
data:
Electrolyzers-HSI (paper):
Arbash et al. 2025: Electrolyzers-HSI: Close-Range Multi-Scene Hyperspectral Imaging Benchmark
Dataset. Sci Data 12, 1818 (2025). https://doi.org/10.1038/s41597-025-06279-9
https://www.nature.com/articles/s41597-025-06279-9
Electrolyzers-HSI (dataset): https://arxiv.org/abs/2505.20507
Car2Car:
BMWK funded project - new automotive and system technologies (2023 - 2025)
automated solutions for an robust and real-time identification of car recycling
products with focus on advanced type discrimination for steel and aluminum in
complex material mixes using hyperspectral imaging and LIBS-based validation
methods
partners: BMW AG, Salzgitter Mannesmann Forschung GmbH, Scholz Recycling
GmbH, Steinert UniSort GmbH, ThyssenKrupp Steel Europe AG, HZDR-HIF,
TUBAF-MVTAT, TU Munich-FML and IWB and CE, associated Aurubis AG, Novelis
Deutschland GmbH, Oetinger Aluminium GmbH, Pilkington Automotive
Deutschland GmbH
links:
https://www.hzdr.de/db/Cms?pOid=68877&pNid=0
https://www.press.bmwgroup.com
videos
https://www.linkedin.com/posts/bmwgroup-innovation-car2car
https://www.facebook.com/BMWGroup/videos/a-circular-material-loop
https://www.youtube.com/shorts/5ynHJ-JC9lg
publications:
SPIE Photonics:
de Lima Ribeiro et al. 2026: Beyond the NIR: hyperspectral imaging in the MWIR range
for enhanced identification of composite objects in e-waste recycling applications", Proc. SPIE 14086,
Optical Sensing and Detection IX, 140861I (29 May 2026); https://doi.org/10.1117/12.3099870
FINEST - SP2: Finest Mineral Additives
Core Project in Helmholtz Sustainability Challenge (2022 - 2027)
development of optical sensor solution for mineral additives in polymers of building insolation material (ETICS)
partners: HZDR-HIF, KIT, UFZ, HZB, TUBAF, Uni Greifswald
website: https://finest-project.de/
links:
https://www.hzdr.de/db/Cms?pOid=66640&pNid=0
https://www.hzdr.de/db/Cms?pOid=67636&pNid=733
https://www.hzdr.de/db/Cms?pOid=70839&pNid=0
RESTORE
EU-funded project of the Horizon Europe programme (2025 - 2029)
European initiative focused on developing an integrated and scalable battery recycling process, specifically for end-of-life electric vehicle and domestic batteries - development of a sensor-based in-line monitoring of black mass benification
partners: ABB, ACCUREC-Recycling, Chalmers Technical University, D-LYTE Chemicals, Eco Recycling, E-LYTE Innovations, Fraunhofer, HZDR-HIF, Indumetal Recycling, PNO Innovation, SK tes, Solvay, Toratecnica, University of Agder, Vianode, Vito
website: https://www.restorebatteryrecycling.eu/
RegioREC
BMFTR-funded within RUBIN programme for entrepreneurial alliances (2025 - 2028)
Recycling and 2nd use of Li-Ion Batteries - development of sensor-based logistics for in-line battery identification and digitalisation
partners: Becker Umweltdienste, DAS, DKS, ERZLABOR, Fluorchemie, LioFit, Mercedes-Benz Energy, META, NOVUM, REELEMENTS, tecVenture, TRICERA, ULT, Viimagic, HZDR-HIF, FhG IKTS, TU Bergakademie Freiberg
info: https://www.innovation-strukturwandel.de
website: https://regiorec.de/
KOLLEKT
BMFTR-funded project (2026 - 2029)
Recycling and 2nd-Use of automotive lights - development of efficient sensor-solutions for sorting applications as well as for supporting disassembly control and automation
partners: HELLA, Covestro, BMW, geba, SW-Maschinenservice, HZDR, Fraunhofer, Heinz Nixdorf Institut, Hochschule Hamm-Lippstadt (associated: Steinert, MPM, Enespa)
RESEARCH INFRASTRUCTURE
The research on multi-sensor systems started with the setup of HELIOS mini (Figure 1, left) in 2018 with sensors for RGB, hyperspectral reflectance (HSI) and laser-induced fluorescence (LiF) covering the visible to near-infrared (VNIR), shortwave infrared (SWIR, and midwave infrared (MWIR) wavelength ranges for signal detection. Activities were expanded by adding robots, a laser profiler, Raman spectroscopy, x-ray fluorescence (XRF) and laser-induced breakdown spectroscopy (LIBS) in the circular HELIOS + setup (Figure 1, right) in 2023.
Additionally, LUNA lab provides capacities for further, in-depth luminescence spectroscopy aiming at method developments for dosimetric and geoscientific applications.
-
HELIOS mini
-
HELIOS +
