A Framework For Recognizing Multi-Agent Action From Visual Evidence

Abstract

A framework for representing and visually recognizing complex multi-agent action is presented. Motivated by work in model-based object recognition and designed for the recognition of action from uncertain visual evidence, the representation has three components: (1) temporal structure descriptions representing the temporal relationships between agent goals, (2) belief networks for representing and recognizing individual agent goals from visual evidence, and (3) belief networks automatically generated from the temporal structure descriptions that use low-order temporal relationships between detected goals to support the recognition of the complex action in the presence of uncertainty. We describe our current work on recognizing American football plays from noisy trajectory data.