Google's light-field VR app resamples rays, it builds no mesh

What you see

Welcome to Light Fields is a free SteamVR application, released by Google, that puts a headset wearer inside a still real-world location, such as the Gamble House or the Space Shuttle Discovery, and lets them lean and shift their head to look around objects with correct changing reflections, depth, and parallax, an effect Google's own March 2018 blog post calls unlike any VR experience shown before. The scenes are described on the app's own Steam store page as navigable stills rather than fully walkable environments; a viewer can move within a limited volume, not walk freely through the whole space.

How it works

Google's post describes building a light-field camera rig by bending a ring of 16 GoPro cameras, normally used for 360 video, into a vertical arc mounted on a rotating platform. Over roughly a minute, the rig sweeps through about a thousand outward-facing viewpoints across a 70-centimeter sphere at each real location. The captured images are aligned, compressed into a custom dataset, and played back through rendering software, built as a Unity plugin, that synthesizes new in-between views by sampling and interpolating the light rays recorded by the physical cameras, rather than by building a 3D mesh of the scene.

Viewing conditions and limits

The Steam listing states the release date as 14 March 2018 and requires a positional-tracking VR headset, listing HTC Vive, Oculus Rift, and Windows Mixed Reality as supported hardware; there is no version for viewing without a headset. The usable viewing volume is bounded by the roughly two-foot-diameter sphere the camera rig swept through during capture, so the experience supports leaning and small repositioning within that volume rather than walking through the scene freely, and each location is a fixed capture from one point in time, not a live or interactive feed.

What it is not

This is not a hologram, free-space or otherwise; the image only exists inside a headset display and is invisible to anyone standing in the room. It is also not standard photogrammetry or a 3D reconstructed mesh, since Google's post is explicit that new views come from resampling recorded light rays rather than from a built geometric model, which is what makes reflections and fine depth cues in the scene look physically correct rather than approximated.

• Ask whether a light-field VR experience requires a headset or claims to be visible without one.
• Ask how large the navigable volume actually is relative to the full scene being described.
• Ask whether new views are synthesized from recorded light rays or from a reconstructed 3D mesh.

Welcome to Light Fields remains a clear, headset-bound demonstration of light-field capture and playback, useful precisely because Google's own materials are specific about the camera count, capture volume, and rendering method rather than reaching for looser display language.