GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy Memory

Hu Zhu1,2, Bohan Li2,3, Xianda Guo4, Hongsi Liu2, Baorui Peng6
Mingqi Yuan5, Xin Jin2, Wenjun Zeng2,*, and Chang Wen Chen1
1The Hong Kong Polytechnic University   2Eastern Institute of Technology   3Shanghai Jiao Tong University
4Wuhan University   5The University of Hong Kong   6Georgia Institute of Technology
* Corresponding author
PDF BibTeX Code Coming Soon Data
Overview of HI-Occ and GEM-Occ.

HI-Occ provides hierarchical indoor semantic occupancy annotations across local views, rooms, and connected buildings. GEM-Occ builds persistent semantic Gaussian memory for embodied occupancy mapping.

Abstract

Semantic occupancy provides a structured spatial memory for embodied indoor agents by jointly representing occupied regions, observed free space, unknown areas, and object semantics. However, existing indoor occupancy benchmarks and methods mainly focus on single-view prediction or room-level online perception, leaving long-horizon semantic mapping across connected indoor spaces underexplored.

We introduce HI-Occ, a hierarchical indoor occupancy benchmark that unifies ScanNet, ScanNet++, and Matterport3D under a common sparse semantic occupancy format while preserving their native observation geometries, including perspective RGB-D frames and pano-centric observation groups. HI-Occ supports local semantic occupancy prediction, room-level online occupancy mapping, and building-level mapping across connected panoramic environments.

We further propose GEM-Occ, a Gaussian Evidence Memory framework for semantic occupancy mapping. Rather than using pointmaps as persistent map states, GEM-Occ treats local visual geometry predictions as transient evidence, converts them into semantic Gaussian occupancy evidence and free-space ray evidence, and fuses them into a persistent hierarchical memory through visibility- and uncertainty-aware causal updates.

HI-Occ Benchmark

HI-Occ examples spanning perspective observations, panoramic observations, room-level occupancy, and building-level occupancy.
Hierarchical indoor occupancy data. HI-Occ unifies perspective RGB-D observations from ScanNet and ScanNet++ with panoramic, connected-space observations from Matterport3D, supporting local, room-level, and building-level occupancy evaluation.
HI-Occ annotation pipeline from reconstructed semantic scenes to visibility-filtered sparse semantic occupancy.
HI-Occ annotation pipeline. Calibrated observations and annotated meshes are converted into scene-level semantic voxels, cropped around each frame or panorama, visibility-filtered, and saved as sparse semantic occupancy annotations.

GEM-Occ Method

GEM-Occ method pipeline.

GEM-Occ converts local visual geometry predictions into semantic Gaussian occupancy evidence and explicit free-space ray evidence, then updates local caches, room submaps, and a building-level graph.

Qualitative Results

Local semantic occupancy predictions from GEM-Occ and five comparison methods alongside images and ground truth.
Local semantic occupancy prediction. GEM-Occ produces cleaner semantic structure and fewer spurious occupied regions than EmbodiedOcc, SplatSSC, GPOcc, and ISO.
Building-level occupancy mapping across connected panoramic environments.
Building-level mapping. Hierarchical Gaussian memory scales occupancy mapping from local observations to connected multi-room environments.

BibTeX

@misc{zhu2026gemoccvisualgeometryevidence,
  title         = {GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy Memory},
  author        = {Hu Zhu and Bohan Li and Xianda Guo and Hongsi Liu and Baorui Peng and Mingqi Yuan and Xin Jin and Wenjun Zeng and Chang Wen Chen},
  year          = {2026},
  eprint        = {2607.05543},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2607.05543}
}