MomentDrive
CVPR 2027 Submission
Anonymous Authors
Author information will be released after the review process.
Most existing Autonomous Valet Parking (AVP) methods rely on pre-built parking maps, which limits their generalization to open-world parking environments and diverse linguistic contexts. Meanwhile, parking instructions in real-world scenarios are often transiently grounded: the referenced entity or region is unambiguous when the instruction is issued, yet may become unresolvable as the scene or viewpoint changes. This characteristic introduces additional challenges for goal localization and task execution.
To address these challenges, we propose MomentDrive, a natural-language task navigation framework for AVP that integrates Bird's-Eye-View (BEV) representations, the general reasoning capabilities of vision-language models (VLMs), and world modeling. MomentDrive accommodates transiently grounded instructions, localizes goals specified by open-vocabulary language, and supports both parking-in and parking-out tasks.
MomentDrive preserves token-level multimodal instruction features and injects them through layer-wise cross-attention into a joint flow-matching model of future BEV features and continuous controls. This formulation supports both direct action generation and joint world–action rollout in a shared latent space. We further introduce a benchmark with two task categories, over 100 distinct scenes, and more than 1,000 navigation episodes. Quantitative results will be added after the evaluation is finalized.
A language command can be clear now—and ambiguous one moment later.
The user refers to a uniquely identifiable entity or region in the surround-view observation at the instant the command is issued.
Directional, ordinal, and visibility-based expressions may change meaning or become invalid as the vehicle moves.
The controller must preserve the grounded intent and produce coherent controls after the original description is no longer directly verifiable.
Visual examples coming soon
Anchor observations and transient-reference examples will appear here.
Token-level instruction grounding meets joint world–action modeling.
Surround-view images and the natural-language command.
A frozen VLM retains fine-grained multimodal instruction features.
Cross-attention injects anchored tokens into the flow transformer.
Future BEV features and continuous controls are modeled jointly.
Method figure coming soon
The final architecture diagram will replace this placeholder.
Instruction representation. MomentDrive grounds the command once at its anchor observation and retains variable-length token-level features instead of reducing the instruction to a single pooled vector.
Visual world state. Surround-view observations are lifted into an ego-centric semantic BEV feature space, providing spatially aligned history without reconstructing RGB pixels.
Joint control. A flow-matching transformer predicts future scene features and vehicle actions in a shared latent space. The model supports direct action generation as well as diagnostic joint rollout.
Designed around instruction anchor frames rather than persistent route descriptions.
Task categories
2
Parking-in and parking-out
Distinct scenes
100+
Diverse parking layouts
Episodes
1,000+
Closed-loop navigation tasks
Each instruction is paired with the anchor observation at which its referent is uniquely identifiable. The benchmark is intended to evaluate task success and safety together with paraphrase consistency, contrastive grounding, distractor robustness, and sensitivity to instruction timing.
Dataset release coming soon
Coming soon
Closed-loop comparisons, grounding diagnostics, ablations, and qualitative rollouts will be published after the evaluation is finalized.
MomentDrive introduces transient language-conditioned vehicle control, where a command is grounded in the current visual field but may not remain resolvable as the vehicle moves. The benchmark makes anchor time and instruction validity explicit, while the model carries token-level multimodal grounding into a joint latent world–action model. This perspective shifts language-conditioned driving from persistent destination specification toward interaction that is grounded in the moment.
@inproceedings{momentdrive2027,
title = {Grounded in the Moment: Transient Language-Conditioned Control for Vision-Only Vehicle Maneuvering},
author = {Anonymous Authors},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2027},
note = {Submission; citation details coming soon}
}