We address the problem of incrementally modeling and forecasting long-term goals of a first-person camera wearer. We present the Darko algorithm for Discovering Agent Rewards for K-futures Online. Darko learns and predicts semantic states, state transitions, rewards, and goals from streaming first-person video using Online Inverse Reinforcement Learning (IRL). In contrast to classical batch IRL approaches that require complete demonstrations, Darko discovers states, transitions, rewards, and goals from continuous observations. We provide a theoretical guarantee of no-regret performance and demonstrate the algorithm's effectiveness empirically on first-person activity datasets spanning extended temporal and spatial horizons.

Wearable cameras provide a continuous stream of first-person visual data that captures the wearer's activities, interactions, and environment over extended periods. A key challenge is to develop AI systems that learn human intent and goals through continuous behavioral observation. This extends beyond simple trajectory forecasting to include reasoning about object interactions, scene-based goals, and semantic understanding. We present the first application of ideas from online learning theory and inverse reinforcement learning to the task of continuously learning human behavior models with a wearable camera. Our approach addresses fundamental questions: where will this person go, what will they do, and what semantic goals are they pursuing?

Our work draws from three research domains: first-person vision (FPV), inverse reinforcement learning (IRL), and trajectory forecasting. In first-person vision, prior work focuses on activity recognition, object detection, and gaze prediction from egocentric video. In IRL, classical methods such as maximum entropy IRL learn reward functions from complete expert demonstrations in a batch setting, assuming access to full trajectories and known state spaces. Trajectory forecasting methods predict future physical positions but typically do not reason about object interactions or semantic goals. Our work distinguishes itself by going "beyond physical trajectory forecasting by reasoning over future object interactions and predicting future goals in terms of scene types".

We formulate the problem as a Markov Decision Process (MDP). States encode 3D position, held objects, and previous goal location: s = [x, y, z, o_1...o_|O|, h_1...h_|K|]. Actions include movement and object acquisition/release. State transitions are incrementally built from observed (s, a, s') triplets. The reward function is modeled as R(s, a; theta) = theta^T * phi(s, a), an inner product of features and learned weights. The Darko algorithm operates in three steps: (1) state tracking via SLAM for localization and action detection for hand-object interactions; (2) goal detection via velocity-based stop detection; (3) online IRL weight update after each episode using gradient descent on the maximum entropy IRL objective.

The gradient update uses the difference between expected feature counts and empirical feature counts of the current episode. We prove a regret bound of R_t <= 2B * sqrt(2td), where the average regret approaches zero as more episodes are observed, guaranteeing no-regret convergence. This means Darko's cumulative prediction performance asymptotically matches the best fixed reward function in hindsight, even though it learns incrementally without access to future data.

Given the learned reward function, we derive multiple forecasting capabilities. Goal prediction is formulated as posterior estimation: P(g | trajectory) is proportional to P(g) * exp(V_st(g) - V_s0(g)), encoding progress toward goals through value differences. This captures the intuition that if the agent has gotten closer to a goal (in terms of expected cumulative reward), that goal becomes more likely. Additional predictions include trajectory length forecasting and action-state subspace visitation, both derived from the core state visitation function. The state visitation function computes expected future state occupancies given partial trajectories, enabling probabilistic multi-modal forecasting of future behaviors.

We evaluate on a custom first-person continuous activity dataset across 5 environments (homes, offices, laboratory), featuring over 200 high-level activities, 250+ actions with 19 objects, and 17 scene types, spanning over 15 days of recording. Goal detection uses velocity-based stop detection, achieving 62-73% accuracy. Darko achieves mean goal prediction probability of 0.378-0.667 across environments using visual detections, and 0.683-0.880 with ground-truth labels. Darko outperforms baselines including logistic regression, max-margin event detection (MMED), and RNN classifiers. Trajectory length prediction achieves 6.3-34.8% median relative error. We also empirically validate sublinear regret convergence, confirming the theoretical guarantee. Notably, incorporating object interactions substantially improves goal prediction compared to position-only representations.

We presented Darko, the first method for continuous first-person semantic behavior modeling at extended temporal and spatial horizons. By combining online inverse reinforcement learning with first-person visual perception, Darko learns to predict goals, trajectories, and actions from streaming egocentric video with theoretical guarantees of no-regret convergence. Our results demonstrate that reasoning beyond physical trajectories — incorporating object interactions and scene semantics — significantly enhances long-term activity forecasting. This work opens new directions at the intersection of online learning, decision-theoretic modeling, and first-person vision.