Thesis Work: Transfer Learning Strategies of Deep Neural Networks for 4D...

Figure: One example of our 4D radar point cloud and the corresponding camera image showing four pedestrians.

Background

Given a 4D radar point cloud of an environment, it is typically not obvious which points belong to certain objects and which type the object is. Hence, there is a need to develop algorithms to obtain these object detection and object classification capabilities. For other sensors doing these tasks, such as camera and lidar, the advancements in the field of deep machine learning have been utilized to revolutionize the performance of such algorithms. However, for radar sensors, this leap in performance is yet to be taken. With this master’s thesis work, we aim to bring these revolutionizing deep neural networks to our radar sensor.

To be able to train deep neural networks which have groundbreaking potential it is generally required to have a big data set of annotated data points. However, to annotate data is both an expensive and a time-consuming process. A training strategy that utilizes pre-existing annotations, or decreases the overall need for annotations, is therefore of high interest.

Goal of the thesis
The aim of this master’s thesis is therefore to investigate deep neural network training strategies that can use both available public data sets of 4D imaging radar point clouds together with point clouds generated by our Sensrad radar sensor. Moreover, a stretched goal of the work is also to investigate neural network architectures that are able to train in a self-supervised fashion to decrease the need of annotated data. We believe that a transformer architecture could be one interesting candidate for such architecture.

This thesis shall consider:

- Select suitable public data sets to include in a training pipeline for object detection and/or object classification using deep neural networks.

- Perform a literature study to find the most promising state-of-the-art deep neural networks that can be trained on multiple 4D radar sensors (i.e. the ones used by the public data sets and our Sensrad 4D radar sensor).

- Explore how to train the neural networks in a self-supervised fashion.

- Evaluate the quality of the developed transfer learning methods on a selected inference task (e.g. object detection and/or object classification), and investigate if the selected method can run in real-time on our radar system.

Our radar sensor is one of the most advanced in the world and we think this thesis has the potential to make a significant contribution to both the research community and to our customers. We would therefore be happy if your achievements could lead to a publication and make it into our product.