Six-view perception
Use front, lateral, and rear visual context rather than relying solely on the front-facing camera.
SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
Research project page
Learning from future visual dynamics during training, planning efficiently with six-camera surround-view observations at inference.
Abstract
World models offer a promising direction for end-to-end autonomous driving, but generating future videos at inference time can be computationally expensive. Using only a front-facing camera also limits access to side and rear traffic, which can be critical for turns, merges, and lane changes. SV-WAM learns from historical six-view camera observations, action tokens, and future-video latent supervision during training, while retaining efficient action-only inference at deployment. An action-centered causal attention mask prevents future-video information from leaking into action prediction, and a differentiable drivable-area compliance regularizer guides road-aware trajectories. The project evaluates the approach with driving benchmarks and qualitative comparisons, including cases illustrating the value of full surround-view perception.
Method Overview
Learning from future video does not require generating future video when the planner is deployed.
Joint world-action learning
Train with action and future-video flow matching under action-centered causal attention.
Action-only inference
Remove the future-video branch at deployment and regularize the training trajectory with drivable-area compliance.
Qualitative Video Demonstrations
All nine demonstration videos from the SV-WAM repository are included below. Every clip is displayed in its own full-width player without side-by-side compression or cropping. Use fullscreen for finer trajectory inspection.
Comparison with DriveLaW and DriveVLA-W0
Four synchronized multi-camera and bird's-eye-view planning comparisons. Purple: DriveVLA-W0; blue: DriveLaW; red: SV-WAM; green: ground truth.
SV-WAM Planning Examples
Four further full-width videos showing the SV-WAM trajectory (red) and ground truth (green).
Experimental Results & Visual Analysis
Figures are displayed one per row at the full available content width. Click any image to inspect the original high-resolution file.
Additional Qualitative Visualizations
Supplementary materials in the repository include fifteen further comparison scenarios, success and failure cases, and an additional visualization. All are shown below at full width, rather than as tiny thumbnails.
15 Comparison Scenarios
Five high-resolution panels, covering cases 01–15 in groups of three.
Success Cases
Failure Cases
Additional Visualization
Paper and Code
Explore the paper, visual demonstrations, and repository release status.