MSGFusion: Multimodal Scene Graph-Guided Infrared and Visible Image Fusion

ARXIV  ·  2025

Guihui Li; Bowei Dong; Kaizhi Dong; Jiayi Li; Haiyong Zheng;

RELATED

Papers      Patents     Grants     Venues     Experts    Cite    Share    Save

ABSTRACT

Infrared and visible image fusion has garnered considerable attention owingto the strong complementarity of these two modalities in complex, harshenvironments. While deep learning-based fusion methods have made remarkableadvances in feature extraction, alignment, fusion, and reconstruction, theystill depend largely on low-level visual cues, such as texture and contrast,and struggle to capture the high-level semantic information embedded in images.Recent attempts to incorporate text as a source of semantic guidance haverelied on unstructured descriptions that neither explicitly model entities,attributes, and relationships nor provide spatial localization, therebylimiting fine-grained fusion performance. To overcome these challenges, weintroduce MSGFusion, a multimodal scene graph-guided fusion framework forinfrared and visible imagery. By deeply coupling structured scene graphsderived from text and vision, MSGFusion explicitly represents entities,attributes, and spatial relations, and then synchronously refines high-levelsemantics and low-level details through successive modules for scene graphrepresentation, hierarchical aggregation, and graph-driven fusion. Extensiveexperiments on multiple public benchmarks show that MSGFusion significantlyoutperforms state-of-the-art approaches, particularly in detail preservationand structural clarity, and delivers superior semantic consistency andgeneralizability in downstream tasks such as low-light object detection,semantic segmentation, and medical image fusion.

 Paraphrase     Questionfy     Wikify     Simplify     Summarize

Download