We present a method for simultaneous real-time 3D reconstruction and 6-DoF camera tracking using a single event camera. Event cameras are novel bio-inspired vision sensors that output pixel-level brightness changes asynchronously, offering advantages such as very high dynamic range (140 dB), microsecond-level temporal resolution, and very low power consumption. Unlike standard cameras, they do not suffer from motion blur and can operate in extreme lighting conditions. We propose a probabilistic framework that jointly estimates the camera motion and a semi-dense 3D map of the scene from a stream of events. Our method operates in real-time at over 300 Hz and demonstrates robust performance in scenarios where conventional cameras fail, such as high-speed motion and high dynamic range scenes.

Simultaneous localization and mapping (SLAM) is a fundamental problem in robotics and computer vision. Traditional frame-based SLAM systems are limited by the characteristics of conventional cameras: they suffer from motion blur at high speeds, have limited dynamic range, and consume significant power. Event cameras, such as the Dynamic Vision Sensor (DVS), represent a paradigm shift in visual sensing. Rather than capturing frames at a fixed rate, each pixel independently and asynchronously reports brightness changes as they occur. This results in a continuous stream of events with microsecond resolution, enabling perception at speeds unattainable by standard cameras. We present the first system that achieves real-time 3D reconstruction and full 6-DoF tracking using only events.

An event camera generates events asynchronously whenever the change in log intensity at a pixel exceeds a threshold C: an event e = (x, y, t, p) is triggered when |log I(x,y,t) - log I(x,y,t-dt)| >= C, where p is the polarity (sign of the change). This model has several key properties: (i) no redundant information is transmitted — only changing pixels generate events; (ii) the temporal resolution is in the order of microseconds, independent of scene illumination; (iii) the dynamic range exceeds 140 dB, compared to about 60 dB for standard cameras. These properties make event cameras ideal for robotics applications where fast, low-power, and robust perception is needed.

Our approach uses a probabilistic generative model that relates the observed events to the camera motion and scene structure. We formulate the problem as maximum likelihood estimation, where we jointly optimize the camera trajectory (parameterized as a continuous-time spline) and a semi-dense 3D map. The map is represented as a collection of 3D points with associated normal vectors. Events are processed in small batches, and for each batch we perform interleaved optimization of the camera pose and the map. The key insight is that event generation can be predicted from the camera motion and scene geometry, enabling a photometric-style error formulation adapted to the event stream.

We evaluate our system on both synthetic and real-world datasets. On synthetic sequences, we achieve sub-millimeter accuracy in 3D reconstruction and sub-degree accuracy in rotation estimation. On real-world sequences captured with a DVS128 event camera, the system runs at over 300 Hz on a single CPU core, demonstrating true real-time performance. We show qualitative results on challenging scenarios including rapid camera rotation and scenes with extreme lighting variations. In comparison with frame-based methods, our approach maintains accuracy in conditions where conventional cameras produce completely unusable images due to motion blur or over/under-exposure.

We have presented the first real-time system for simultaneous 3D reconstruction and 6-DoF tracking using only an event camera. Our probabilistic framework effectively exploits the unique properties of event cameras — high temporal resolution, high dynamic range, and low latency — to achieve performance that is impossible with conventional cameras. The system opens new possibilities for visual perception in extreme conditions that are relevant for fast-moving robots, autonomous vehicles, and other demanding applications.