We propose Picture, a probabilistic programming language for scene perception that allows researchers to express complex generative models of images as short, readable probabilistic programs. Picture programs describe a scene generation process — placing objects in a 3D scene, rendering the scene using graphics software, and comparing the rendered image to the observed image. Inference in Picture is achieved through a combination of Markov Chain Monte Carlo (MCMC) sampling and efficient approximate likelihood computations. The key insight is that scene understanding can be formulated as "inverse graphics" — inferring the latent scene description that most likely generated the observed image. We demonstrate Picture on 3D human pose estimation, 3D object reconstruction, and multi-object scene parsing, showing that short probabilistic programs can achieve results competitive with task-specific state-of-the-art systems.

Scene perception — understanding the 3D structure of a scene from an image — is a fundamental goal of computer vision. Modern approaches typically train discriminative models (e.g., deep neural networks) that directly map from pixels to scene properties, treating the pipeline as a black box. While achieving impressive results on benchmarks, these approaches lack interpretability, require large labeled datasets, and do not naturally express compositional scene structure. An alternative paradigm is analysis by synthesis, or inverse graphics: hypothesize a 3D scene, render it using a graphics engine, compare the rendering to the observed image, and iteratively refine the hypothesis. This approach is inherently interpretable, compositional, and data-efficient.

However, inverse graphics approaches have been difficult to implement, slow to run, and brittle in practice. We propose Picture to address these challenges: a probabilistic programming language specifically designed for scene perception. Picture provides high-level abstractions for defining 3D scene models, integrating graphics rendering engines, and performing efficient probabilistic inference. A researcher can express a scene model in just tens of lines of code, with the inference engine automatically handling the complex MCMC sampling. This separation of modeling (what the world looks like) from inference (how to compute the posterior) is a key design principle.

Probabilistic programming languages like Church, Venture, and Stan provide general frameworks for expressing and solving probabilistic models. However, they lack domain-specific primitives for vision tasks — such as 3D scene representation and rendering. Analysis-by-synthesis approaches in vision have been applied to faces, bodies, and indoor scenes, but each requires extensive custom engineering of both the generative model and the inference algorithm. Generative adversarial networks and variational autoencoders learn implicit generative models but do not provide explicit scene structure representations. Picture uniquely combines the expressiveness of probabilistic programming with domain-specific vision primitives and efficient graphics-based rendering.

A Picture program defines a generative model of scenes and images. The program first samples latent scene parameters from prior distributions — such as the number of objects, their identities, poses, lighting conditions, and camera parameters. It then constructs a 3D scene using these parameters, placing 3D meshes at appropriate positions and orientations. The scene is rendered using an approximate graphics engine (based on OpenGL) to produce a synthetic image. Finally, the program scores the hypothesis by comparing the rendered image to the observed image using a likelihood function. The entire process is expressed as a concise probabilistic program of typically 20-50 lines.

Inference in Picture amounts to sampling from the posterior distribution over latent scene parameters given the observed image. The inference engine combines several strategies: Metropolis-Hastings MCMC for exploring the scene parameter space, data-driven proposal distributions initialized by bottom-up detectors (e.g., CNN-based object detectors) to guide the sampler toward promising regions, and approximate likelihood computations that avoid pixel-exact comparisons. The data-driven proposals are crucial for practical efficiency: rather than exploring the vast parameter space uniformly, bottom-up recognition modules provide informed initial guesses that dramatically speed up convergence. This hybrid approach combines the flexibility of top-down generative models with the efficiency of bottom-up discriminative features.

We demonstrate Picture on three tasks. For 3D human pose estimation, a Picture program of approximately 50 lines specifies a body model with articulated joints, renders it, and infers joint angles from monocular images. On the HumanEva benchmark, Picture achieves results competitive with specialized pose estimation systems. For 3D face reconstruction, a 20-line program using a morphable face model achieves accurate 3D face shape and texture recovery. For multi-object scene parsing, Picture programs can infer the number, identity, position, and orientation of multiple objects in a scene, demonstrating the compositional nature of the approach. Across all tasks, the same inference engine is used, with only the generative program changing.

Picture demonstrates that probabilistic programming provides a powerful and flexible framework for scene perception. By expressing generative models as short programs and leveraging graphics engines for rendering within a probabilistic inference loop, we enable rapid prototyping of vision models that are interpretable, compositional, and data-efficient. The separation of modeling from inference allows researchers to focus on what the world looks like rather than how to compute with that knowledge. We envision Picture as a step toward "vision as inverse graphics" becoming a practical reality, complementing the strengths of discriminative deep learning approaches.