MIT researchers have built a chip that allows small, low-power robots to construct detailed 3D maps of their surroundings in real time and plan a collision-free path through them, all while drawing about as much power as a single LED.
Presented at the IEEE Symposium on VLSI Technology and Circuits, the system-on-chip, named Gleanmer, is described by the team as the first fabricated chip to achieve real-time 3D occupancy mapping at under 6 mW for edge applications. The work points toward a future in which a drone could thread its way through the tight corners of an industrial HVAC duct to check for gas leaks without a heavy, power-hungry computer on board.
The team claims Gleanmer may help tiny, low-power robots avoid obstacles in industrial environments. One example: checking for gas leaks around tight corners inside an industrial HVAC system. Image from Adobe Stock (under license)
Mapping a 3D space in real time is typically expensive in the very resource that tiny robots lack. A robot has to store the images its camera captures and process every 3D pixel in each frame multiple times. The maps themselves are usually built from voxels, the cube-shaped 3D equivalent of pixels. Representing both obstacles and the open space around them with voxels consumes a great deal of memory, which is why conventional approaches draw so much power.
Trading Voxels for Gaussian Blobs
The MIT team's core idea is to drop voxels in favor of ellipsoidal blobs called Gaussians, whose size, shape, and thickness flex to fit the scene. A single elongated ellipsoid can stand in for a region that would otherwise require many voxels, so the same obstacle surfaces and free space can be captured far more compactly. The map records both obstacles and open space, and that combination lets the robot reason about where it can safely fly.
The team highlights that this chip is particularly noteworthy because it stores very large maps in a very small space—and it does so efficiently. The chip runs an algorithm the lab developed earlier, called GMMap, for building these Gaussian mixture maps, paired with custom hardware tuned to that math.
Co-Designing the Algorithm and the Silicon
The researchers describe two techniques to keep the workload small. The first generates accurate Gaussians from a depth image in a single pass, after which the image can be discarded, so the chip never has to hold a full frame in memory. Rather than compare every pixel to every other pixel, the algorithm assumes neighboring pixels belong to the same Gaussian and compares each pixel only to its neighbors. At any given time, the chip needs to store only a few pixels in memory, significantly reducing the memory footprint the algorithm requires.
The second technique fuses overlapping Gaussians directly as the robot revisits an object from new viewpoints, without going back to the original pixels, which keeps the map from growing as the robot explores. Because the map stays compact, the chip can hold the Gaussians it is actively using in small, fast on-chip memory next to the compute units, rather than fetching them from distant, power-hungry off-chip storage.
The chip-testing and map-visualization setup. Image used courtesy of Arxiv
Built on a 16-nm process, Gleanmer processes 640 x 480 images at more than 88 frames per second and answers more than 540,000 map queries per second. The co-design cuts map construction energy by up to 63% and query energy by up to 81%, and lets the robot plan a safe trajectory using only about 20% of the energy it would otherwise need. Compared with the best existing mapping chip, the team reports that Gleanmer requires roughly 2.5% as much power.
Beyond drones, the researchers see uses in lightweight augmented-reality headsets, which must continuously understand a room over long sessions without draining the battery, as well as in pipeline and duct inspection. They also want to explore representing schematics and blueprints as Gaussians so AI systems can reason over them more efficiently. Next, the group plans to further improve efficiency by moving the processing elements closer to the sensors that feed them data.
The research was supported by the MIT-MathWorks Fellowship, Amazon, the U.S. National Science Foundation, and Intel.
