Copyright Interesting Engineering
MIT researchers have built a new AI system that allows robots to create detailed 3D maps of complex environments within seconds. The technology could transform how search-and-rescue robots navigate collapsed mines or disaster sites, where speed and accuracy can make the difference between life and death. The system combines recent advances in machine learning with classical computer vision principles. It can process an unlimited number of images from a robot’s onboard cameras, generating accurate 3D reconstructions while estimating the robot’s position in real time. Robots rely on a technique called simultaneous localization and mapping, or SLAM, to recreate their surroundings and determine where they are. Traditional SLAM methods often fail in crowded or visually complex environments and require pre-calibrated cameras. Machine learning models simplified the process but could only process about 60 images at once, making them unsuitable for real-world missions where a robot must analyze thousands of frames quickly. MIT graduate student Dominic Maggio, postdoctoral researcher Hyungtae Lim, and aerospace professor Luca Carlone set out to fix that. Their new approach breaks a scene into smaller “submaps” that are created and aligned incrementally. The system then stitches these submaps together into one coherent 3D model, allowing a robot to move quickly while maintaining spatial accuracy. “This seemed like a very simple solution, but when I first tried it, I was surprised that it didn’t work that well,” Maggio says. As Maggio explored older computer vision research, he discovered why. Machine-learning models often introduce subtle distortions in submaps, making them difficult to align correctly using standard rotation and translation techniques. Bridging geometry and AI Carlone and his team addressed the problem by borrowing techniques from traditional geometry. They developed a mathematical framework that captures and corrects deformations in each submap so the system can align them consistently. “We need to make sure all the submaps are deformed in a consistent way so we can align them well with each other,” Carlone explains. Once Maggio merged the strengths of machine learning and classical optimization, the results were immediate. “Once Dominic had the intuition to bridge these two worlds — learning-based approaches and traditional optimization methods — the implementation was fairly straightforward,” Carlone says. “Coming up with something this effective and simple has potential for a lot of applications.” From lab tests to real-world use The MIT system proved faster and more accurate than existing mapping techniques. It required no special camera calibration or additional processing tools. In one demonstration, the researchers captured a short cell phone video of the MIT Chapel and reconstructed a precise 3D model of the interior in seconds. The reconstructed scenes had an average error of less than five centimeters. The team believes this simplicity could help deploy the method in real-world robots, wearable AR or VR systems, and even warehouse automation. “Knowing about traditional geometry pays off. If you understand deeply what is going on in the model, you can get much better results and make things much more scalable,” Carlone says. The research will be presented at the Conference on Neural Information Processing Systems (NeurIPS) and is available on arXiv.
