We present an imaging technique that manipulates direct and indirect light in scenes to enhance visual analysis. Our key observation is that direct light always obeys the epipolar geometry of a projector-camera pair, while indirect light overwhelmingly does not. By exploiting this principle, our method enables three capabilities: generating indirect-only video streams, improving structured-light shape recovery algorithms' robustness to indirect light, and enabling single-shot dynamic 3D shape capture that is robust to global illumination effects.

Structured light scanning is one of the most widely used techniques for acquiring 3D shape. A projector casts known patterns onto a scene, and a camera observes the deformed patterns to establish projector-camera correspondences that yield depth through triangulation. Despite its maturity, structured light remains susceptible to global illumination effects — interreflections, subsurface scattering, and volumetric scattering — which corrupt the observed patterns and cause systematic errors in reconstructed geometry.

Previous approaches to handling indirect light in structured light systems fall into two categories: designing illumination patterns that are inherently robust to global light transport, or explicitly separating direct and indirect components using high-frequency patterns. The former sacrifices pattern coding efficiency, while the latter requires many additional images that preclude real-time or single-shot acquisition. Our approach is fundamentally different: we exploit the epipolar geometry inherent in any projector-camera system to achieve separation without sacrificing coding efficiency or requiring extra captures.

Direct-indirect separation was pioneered by Nayar et al., who showed that high-frequency illumination patterns can isolate direct light from global illumination. Their method requires a minimum of three images per separation, making it impractical for dynamic scenes. Subsequent works by Gupta et al. explored logical coding strategies that combine separation with structured light decoding, but still require multiple shots. In contrast, epipolar-based methods from stereo vision have long exploited geometric constraints for correspondence, yet their application to light transport separation in projector-camera systems has not been explored.

Consider a projector-camera pair with known calibration. When the projector illuminates a single column, direct light from a scene point will arrive at the camera along the corresponding epipolar line. This is a fundamental consequence of epipolar geometry: the projector column, the scene point, and the camera pixel must be coplanar. Indirect light, however, undergoes one or more bounces in the scene before reaching the camera, breaking this coplanarity constraint. Therefore, by analyzing the spatial distribution of received light relative to epipolar lines, we can classify each light contribution as direct or indirect.

Our separation algorithm operates in two steps. First, we project a set of line patterns aligned with the projector's epipolar planes. For each camera pixel, the direct component produces a sharp peak in the epipolar direction, while the indirect component forms a diffuse spread across non-epipolar directions. Second, we apply a 1D filter along each epipolar line to extract the peak (direct) and subtract it from the total to obtain the indirect component. The filter is designed to be robust to noise and calibration imperfections.

With the direct-indirect separation in place, we integrate it into structured light shape recovery. Standard structured light algorithms decode binary or Gray code patterns to find projector-camera correspondences. When indirect light corrupts these patterns, decoding errors propagate to the reconstructed 3D geometry. Our approach first removes the indirect component from each captured frame, then applies standard decoding algorithms to the cleaned direct-only images. Furthermore, we demonstrate a single-shot variant that embeds the separation process within a single projected pattern using epipolar-aware design, enabling dynamic 3D capture of moving objects.

We evaluate our method on scenes exhibiting strong interreflections (concave metallic objects), subsurface scattering (translucent materials like wax and marble), and volumetric scattering (scenes with participating media). For shape recovery, we compare against standard Gray code decoding and the high-frequency separation method of Nayar et al. Our epipolar-based separation achieves comparable or superior 3D reconstruction accuracy while requiring significantly fewer input images. The single-shot variant successfully captures dynamic scenes at video frame rates with shape accuracy within 1-2 mm of the multi-shot baseline. Additionally, the indirect-only video streams reveal material properties invisible in standard imaging, such as subsurface scattering profiles.

We have presented a novel approach to structured light 3D scanning that exploits epipolar geometry to separate direct from indirect light transport. This geometric insight enables robust shape recovery in the presence of interreflections, subsurface scattering, and volumetric scattering, while also providing access to indirect-only appearance information. Our single-shot variant demonstrates the practical applicability of the approach for dynamic 3D capture. Future work includes extending the method to multi-projector systems and exploring the rich information contained in the separated indirect light transport for material classification and relighting applications.