Cyber Valley Research Fund’s self-supervised robot project

Led by Prof. Dr. Jörg Stückler, Research Group Leader, and a dedicated team of researchers, the project, “Self-supervised learning of mobility affordances for vision-based navigation", has advanced the field of mobile robotics. They believe that mobile robots, whether in logistics or autonomous vehicles, should be able to properly adapt their navigation actions according to the surrounding environment. 

The Cyber Valley Research Fund supported the project from March 2020 until its conclusion in October 2023. In this project, the team investigated novel approaches for adapting kinematic and dynamics models of wheeled mobile robots in a self-supervised way. The team’s goal was to develop modularized perception and motion control methods that allow dynamics models to learn through interaction with the environment. Stückler’s team created a method that uses camera images and inertial measurements to estimate the robot's motion.  

They improved two types of robot platforms: one for indoor and one for outdoor use. Initially, they focused on a motion model for indoor wheeled robots controlled by setting target velocities. Then, they expanded this method to adjust parameters for a robot with car-like, steerable front wheels. Evaluating their online adaptation techniques showed better trajectory estimation and prediction accuracy compared to offline calibration or visual-inertial trajectory estimation without motion model information. 

The Cyber Valley Research Fund, supported by six partners, has invested five million euros in fostering innovation and scientific excellence. Research groups were invited to submit project proposals, with ethical and societal considerations playing a crucial role in the evaluation process.   

This project has yielded the following peer-reviewed publications: 

• Li, H., Stueckler, J. Tracking 6-DoF Object Motion from Events and Frames. In Proc. of IEEE Int. Conf. on Robotics and Automation (ICRA), 2021. DOI: 10.1109/ICRA48506.2021.9561760 

• Li, H., Stueckler, J. Visual-Inertial Odometry with Online Calibration of Velocity-Control Based Kinematic Motion Models. IEEE Robotics and Automation Letters (RA-L), 2022. DOI: 10.1109/LRA.2022.3169837 

• Li, H., Stueckler, J. Online Calibration of a Single-Track Ground Vehicle Dynamics Model by Tight Fusion with Visual-Inertial Odometry. Accepted for IEEE International Conference on Robotics and Automation (2024). Preprint arXiv:2309.11148. 

• Li, H., Stueckler, J. Observability Analysis of Visual-Inertial Odometry with Online Calibration of Velocity-Control Based Kinematic Motion Models abs/2204.06651, CoRR/arxiv, 2022