GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy Memory

Hu Zhu1,5 Bohan Li2,5 Xianda Guo3 Yanlun Peng4 Hongsi Liu5 Baorui Peng6
Xiaofeng Wang7 Mingqi Yuan8 Xin Jin5 Wenjun Zeng5 Chang Wen Chen1
1The Hong Kong Polytechnic University 2Shanghai Jiao Tong University
3Wuhan University 4Great Wall Motor 5Eastern Institute of Technology, Ningbo
6Georgia Institute of Technology 7Tsinghua University 8The University of Hong Kong
PDF BibTeX Code Data
Overview of HIOcc and GEM-Occ.

HIOcc provides hierarchical indoor semantic occupancy annotations across local views, rooms, and connected buildings. GEM-Occ builds persistent semantic Gaussian memory for embodied occupancy mapping.

Abstract

Embodied agents exploring indoor environments require reliable semantic occupancy memory that persists across observations and revisits. Building such memory is challenging because each observation provides incomplete and uncertain geometric and semantic evidence.

We introduce GEM-Occ, a Gaussian Evidence Memory framework that consolidates evidence accumulated over time into persistent semantic occupancy memory. Local predictions are converted into occupied semantic Gaussians and free-space ray evidence. Confidence- and visibility-aware causal updates integrate supporting observations, suppress occupancy contradicted by observed free space, and preserve previously observed structures through occlusion. A hierarchical memory organization supports continued mapping and efficient queries across connected indoor spaces.

To evaluate this capability, we introduce HIOcc, a unified benchmark for embodied semantic occupancy memory. HIOcc establishes a shared semantic label space and evaluation framework spanning local prediction, room-level online mapping, and building-level mapping, while accommodating perspective and panoramic observations.

Experiments on HIOcc demonstrate that GEM-Occ outperforms existing methods, enabling accurate semantic occupancy prediction and consistent online mapping across spatial scales with efficient memory usage and fast occupancy queries.

HIOcc Benchmark

HIOcc annotations compared with Occ-ScanNet, with additional ScanNet++ and Matterport3D examples.
HIOcc annotation quality. Comparisons with Occ-ScanNet highlight preserved object structures on ScanNet. Additional ScanNet++ and Matterport3D examples show perspective and panoramic observations with their occupancy targets.
HIOcc annotation pipeline from annotated scene geometry to sparse semantic occupancy targets.
HIOcc annotation pipeline. Annotated scene geometry from ScanNet, ScanNet++, and Matterport3D is mapped to shared semantic labels and converted into sparse occupancy targets through viewpoint-dependent cropping and filtering.

GEM-Occ Method

GEM-Occ evidence construction, causal memory updates, and semantic occupancy queries.

GEM-Occ converts streaming perspective and panoramic observations into semantic Gaussian evidence and free-space rays. Confidence-weighted fusion reinforces supported structures, free-space evidence revises contradicted occupancy, and visibility-aware updates retain occluded memory. Occupancy queries read out local, room-level, and building-level predictions.

Qualitative Results

Local semantic occupancy predictions from GEM-Occ and four comparison methods alongside images and ground truth.
Local semantic occupancy prediction. GEM-Occ produces cleaner semantic structure and fewer spurious occupied regions than EmbodiedOcc, SplatSSC, GPOcc, and ISO.
Building-level occupancy mapping across connected panoramic environments.
Building-level mapping. Two sequences show semantic occupancy memory evolving with incoming panoramic observations. The upper sequence includes the final prediction and ground-truth map; the lower sequence ends with a larger view of the accumulated map.

BibTeX

@misc{zhu2026gemoccvisualgeometryevidence,
  title         = {From Visual Geometry Evidence to Embodied Semantic Occupancy Memory},
  author        = {Hu Zhu and Bohan Li and Xianda Guo and Yanlun Peng and Hongsi Liu and Baorui Peng and Xiaofeng Wang and Mingqi Yuan and Xin Jin and Wenjun Zeng and Chang Wen Chen},
  year          = {2026},
  eprint        = {2607.05543},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2607.05543}
}