We propose a direct (featureless) monocular SLAM algorithm which, in contrast to current state-of-the-art regarding direct monocular SLAM methods that operate on small-scale maps only, allows to build large-scale, consistent maps of the environment. Along with a novel direct tracking method based on semi-dense depth maps, the method includes map optimization using pose graph optimization and scale-drift aware loop closure detection, which enables building consistent maps even over long trajectories.

Visual SLAM (Simultaneous Localization and Mapping) is one of the fundamental problems in robotics and computer vision. Traditional approaches rely on extracting and matching sparse feature points, such as those used in PTAM and ORB-SLAM. While effective, these feature-based methods discard most of the information in the image and can fail in textureless regions. Direct methods, on the other hand, operate directly on image intensities, using all available information and naturally providing semi-dense or dense reconstructions.

Our tracking method estimates the rigid-body transformation (rotation and translation) between the current frame and the active keyframe. This is done by minimizing the photometric error — the weighted sum of squared intensity differences — over all pixels with valid depth in the reference frame. The minimization is performed using a coarse-to-fine scheme with a robust Huber norm to handle outliers from occlusions and moving objects. The tracking operates on semi-dense depth maps, utilizing only pixels with sufficient image gradient.

For mapping, we maintain a pose graph of keyframes. When a new keyframe is created, we propagate depth information from the previous keyframe and refine it using stereo comparisons from subsequent frames. The global map is optimized using pose graph optimization with similarity transformations (Sim(3)), which explicitly models and corrects for scale drift — a unique challenge in monocular SLAM. This allows us to build consistent maps over hundreds of meters.

We evaluate LSD-SLAM on the TUM RGB-D benchmark and on outdoor sequences. On the TUM benchmark, LSD-SLAM achieves competitive accuracy with state-of-the-art feature-based methods while providing semi-dense depth maps as a byproduct. On outdoor sequences, we demonstrate the ability to build consistent maps over trajectories of several hundred meters. The method runs in real-time on a single CPU core, making it practical for robotics applications.

We have presented LSD-SLAM, the first direct monocular SLAM method capable of operating at large scale. By combining semi-dense direct tracking, scale-aware pose graph optimization, and scale-drift-aware loop closure, our method builds globally consistent maps from a single handheld camera. We believe that direct methods will play an increasingly important role in visual SLAM, offering richer scene representations than traditional feature-based approaches.