Dataset card · PermaVid-sourced · static-scene multi-trajectory video

78 real Unreal-Engine scenes, twenty diverse camera paths each.

A curated corpus of 1,527 photorealistic video clips drawn from PermaVid. Each Unreal-Engine environment supplies dozens of real captured camera trajectories; we keep the twenty most diverse per scene by farthest-point sampling, standardised to PermaVid’s exact training pose convention. Every clip carries ground-truth per-frame camera poses, intrinsics and depth — the long-horizon 1000-frame path is the raw asset.

scenes

UE environments

clips

1,527

≈ 20 trajectories / scene

frames

1.53M

1000 per clip

footage

21.2 h

50 s per clip @ 20 fps

resolution

832×480

90° pinhole · fx=fy=416

poses

bit-exact

world→camera, GT depth

Companion to the AI2-THOR–rendered spear_scene_dataset (150 scenes). SPEAR itself would not build on this cluster, so this version curates real PermaVid trajectories rather than rendering new ones — same 832×480 / 90° intrinsic, co-trainable.

at a glance

One frame from every scene

The middle frame of each scene’s first curated clip — a single sweep across the whole corpus.

Composition

Every scene is tagged with one of six coarse types. Clips are unevenly distributed because scenes differ in how many trajectory episodes PermaVid captured.

scene types

Clips per scene type

UrbanNatureInteriorHistoricalIndustrialOther

Urban · 19 scenes · 379 clips

Streets, plazas, cities, transit, waterfront towns.

Nature · 19 scenes · 362 clips

Terrain, deserts, forests, gardens, caves, snow.

Interior · 15 scenes · 286 clips

Indoor spaces: halls, corridors, garages, rooms.

Historical · 12 scenes · 240 clips

Ruins, castles, temples, medieval and period sites.

Industrial · 11 scenes · 220 clips

Factories, plants, hangars, yards, sci-fi facilities.

Other · 2 scenes · 40 clips

Mixed / hard-to-bucket modular environments.

Statistics

Curation depth and the geometric character of the selected camera paths, computed from manifest.parquet (1,527 clips). Bars are hoverable.

curation

How deep was each pool?

For every scene we start from all of PermaVid’s captured trajectory episodes, then keep the T = 20 most diverse by farthest-point sampling. Median pool is 46 episodes.

honesty

7 scenes fall short of T = 20

The target is 20 trajectories per scene, but a handful of environments simply do not have 20 usable episodes. We keep the full pool rather than pad — the dataset is capped by what PermaVid captured.

scene	type	pool	T	min-div
Hospital	Interior	15	15	0.68
InteriorDemo_NEW	Interior	19	19	1.90
OperaHouse	Interior	19	19	2.55
OperatingRoom	Interior	13	13	1.57
Eastern_Garden	Nature	7	7	1.26
RainMap	Nature	15	15	2.00
TemplePlaza	Urban	19	19	1.50

71 / 78 scenes reach the full T = 20 · 0 clips failed QA (all 1,527 pass).

trajectory descriptors

What the camera paths look like

Distribution across all 1,527 clips of the geometric descriptors used to score diversity. Heavy-tailed quantities (path length, footprint, height, speed) are shown on a log axis; the amber line marks the median. A few near-aerial fly-throughs stretch the tails to kilometre scale.

selection quality

Per-scene trajectory diversity

Each dot is one scene, placed by its minimum pairwise diversity — the closest any two of its kept trajectories get in descriptor space. Higher is better. Hollow rings are the 7 reduced-T scenes. Hover a dot for the scene.

UrbanNatureInteriorHistoricalIndustrialOther

Anatomy of a clip

Every clip is a folder: an RGB video plus per-frame ground-truth geometry, in PermaVid’s own verified convention. Poses were validated bit-identical (round-trip error 2.4e-7) against PermaVid’s training reader.

example

AncientRuins · traj_00_ep13

frame 500 / 1000

RGB frame — video.mp4 RGB · 832×480 · 20 fps · lossless stream-copy of PermaVid rgb.mp4

depth frame (colourised) — depth.mp4 depth · 8-bit relative, colourised here (Inferno) — **ordinal, not metric**

poses.npy (1000, 7)

world→camera · row = [tx ty tz | qx qy qz qw]

frame 0  t = [+33.402, +217.552, +52.170]
         q = [+0.670, +0.227, +0.227, -0.670]  (xyzw)

R_w2c = Rotation.from_quat(q).as_matrix() · camera centre = −Rᵀt

intrinsics.npy (1000, 4)

normalised · [fx/W, fy/H, cx/W, cy/H] · constant per clip

norm = [0.5000, 0.8667, 0.5000, 0.5000]
K    = fx=416  fy=416  cx=416  cy=240

832×480 → hFOV 90.0° · vFOV 60.0° (PermaVid CineCamera default; pose.json carries no focal length)

Verified camera convention

poses.npy = world→camera. Bit-identical to what PermaVid’s reader (quaternion_to_w2c) consumes from the raw pose.json.
c2w axis frame is [right, up, forward] in a UE left-handed world (X-fwd, Y-right, Z-up) — up → camera +y. Not OpenCV.
For a right-handed OpenCV c2w: c2w_cv = diag(1,1,−1,1)·c2w·diag(1,−1,1,1) (flip world-Z on the point cloud too).
Windowing is a pure slice: an N-frame window from frame i uses poses[i:i+N], intrinsics[i:i+N] and decodes frames [i, i+N) — no re-encode, poses stay exact. PermaVid trains on 81-frame windows.

PermaVid camera-path render — campose.png PermaVid’s own camera-trajectory render, shipped per clip

All 78 scenes

The complete corpus, grouped by type. Each thumbnail is the first curated trajectory of that scene; hover for its numbers. Featured scenes below get a full trajectory breakdown.

Urban

19 scenes · 379 clips

JapanTrainStation_Optimised20× · div 2.41

Science_Fiction_valley_town20× · div 2.22

Nature

19 scenes · 362 clips

Interior

15 scenes · 286 clips

InteriorDemo_NEW19× · div 1.90 reduced T

Historical

12 scenes · 240 clips

Industrial

11 scenes · 220 clips

Other

2 scenes · 40 clips

Featured scenes

Sixteen representative scenes across all six types. For each: the twenty preview frames (one per clip), the same twenty camera paths reconstructed from poses.npy (top-down + side), and the numbers.

scene

AncientRuins

Historicalval

Overgrown highland ruins under a dynamic sky — wide exploratory sweeps and slow orbits.

trajectories

from 50 in pool

2.50

min diversity

closest kept pair

24–89

path length m

per clip

829–5970

yaw sweep °

cumulative turn

0.0–8.9

height range m

vertical extent

0.5–1.8

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

Demonstration_Castle

Historicaltrain

Fortified castle grounds — the most diverse path set in the corpus (min-div 3.72).

trajectories

from 50 in pool

3.72

min diversity

closest kept pair

69–101

path length m

per clip

650–1633

yaw sweep °

cumulative turn

0.1–0.2

height range m

vertical extent

1.4–2.0

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

Demonstration_Castle trajectory diversity

scene

Brass_Palace

Historicaltrain

Ornate palace interiors and courtyards — measured architectural walk-throughs.

trajectories

from 50 in pool

3.08

min diversity

closest kept pair

52–98

path length m

per clip

1002–2978

yaw sweep °

cumulative turn

0.0–4.6

height range m

vertical extent

1.0–2.0

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

Greek_Island

Naturetrain

Whitewashed cliffside village over the Aegean — coastal approaches and ridgelines.

trajectories

from 50 in pool

2.54

min diversity

closest kept pair

22–88

path length m

per clip

780–4838

yaw sweep °

cumulative turn

0.0–7.4

height range m

vertical extent

0.4–1.8

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

Grass_Hills

Naturetrain

Vast open terrain — kilometre-scale, near-aerial camera flights.

trajectories

from 50 in pool

3.24

min diversity

closest kept pair

68–93

path length m

per clip

714–2104

yaw sweep °

cumulative turn

1.0–5.0

height range m

vertical extent

1.4–1.9

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

SnowMap

Naturetrain

Snow-covered landscape — the widest trajectory spread of any nature scene.

trajectories

from 50 in pool

3.69

min diversity

closest kept pair

71–89

path length m

per clip

778–2130

yaw sweep °

cumulative turn

0.3–4.2

height range m

vertical extent

1.4–1.8

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

Colosseum_Desert

Naturetrain

A colosseum amid dunes — long approaches and full-perimeter orbits.

trajectories

from 47 in pool

3.04

min diversity

closest kept pair

61–92

path length m

per clip

799–2288

yaw sweep °

cumulative turn

0.0–0.8

height range m

vertical extent

1.2–1.8

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

DowntownWest

Urbantrain

Modern downtown blocks — street canyons, intersections, rooftop passes.

trajectories

from 46 in pool

2.59

min diversity

closest kept pair

25–95

path length m

per clip

739–6239

yaw sweep °

cumulative turn

0.0–0.6

height range m

vertical extent

0.5–1.9

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

Venice

Urbantrain

Venetian canals and palazzi — boat-level glides and bridge crossings.

trajectories

from 44 in pool

2.98

min diversity

closest kept pair

37–90

path length m

per clip

746–3103

yaw sweep °

cumulative turn

0.0–0.1

height range m

vertical extent

0.7–1.8

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

ChineseWaterTown_Ver1

Urbanval

Canal-side old town — bridges, streets and waterline paths.

trajectories

from 50 in pool

2.46

min diversity

closest kept pair

26–83

path length m

per clip

898–6253

yaw sweep °

cumulative turn

0.0–0.7

height range m

vertical extent

0.5–1.7

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

ChineseWaterTown_Ver1 trajectory diversity

scene

racing_track

Urbantrain

Circuit and grandstands — sweeping high-speed follow paths.

trajectories

from 50 in pool

3.37

min diversity

closest kept pair

62–91

path length m

per clip

704–1708

yaw sweep °

cumulative turn

0.0–0.6

height range m

vertical extent

1.2–1.8

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

Museum

Interiortrain

Gallery interior — deliberate walk-throughs past exhibits (min-div 3.32).

trajectories

from 25 in pool

3.32

min diversity

closest kept pair

69–87

path length m

per clip

879–1809

yaw sweep °

cumulative turn

0.0–0.4

height range m

vertical extent

1.4–1.7

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

CommandCenter

Interiorval

Interior control room — tight, constrained indoor trajectories.

trajectories

from 34 in pool

2.31

min diversity

closest kept pair

26–80

path length m

per clip

1665–5994

yaw sweep °

cumulative turn

0.0–1.0

height range m

vertical extent

0.5–1.6

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

Old_Factory_01

Industrialtrain

Derelict industrial hall — machinery, gantries and catwalks.

trajectories

from 50 in pool

2.87

min diversity

closest kept pair

21–94

path length m

per clip

886–3678

yaw sweep °

cumulative turn

0.0–2.5

height range m

vertical extent

0.4–1.9

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

LookoutTower

Industrialtrain

Tower and surrounds — vertical climbs and long panoramic reveals.

trajectories

from 50 in pool

3.47

min diversity

closest kept pair

66–90

path length m

per clip

687–1558

yaw sweep °

cumulative turn

0.4–13.0

height range m

vertical extent

1.3–1.8

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

scene

ModularBuilding

Othertrain

Modular mixed environment — varied framing across a compact site.

trajectories

from 50 in pool

3.26

min diversity

closest kept pair

69–87

path length m

per clip

685–1573

yaw sweep °

cumulative turn

0.0–0.6

height range m

vertical extent

1.4–1.7

mean speed m/s

avg over path

Twenty preview frames — one per curated clip (middle frame of each 50-second path)

$ModularBuilding contact sheet$

The same twenty camera paths, reconstructed from world→camera poses — coloured by selection order

Provenance & layout

What ships, where it lives, and what to be careful about.

deliverable layout

Directory tree

dataset/
  <Scene>/traj_<ii>_ep<k>/
      video.mp4        # RGB · 1000f @ 20fps · 832×480
      depth.mp4        # 8-bit relative depth (non-metric)
      poses.npy        # (nf,7) world→camera  [t | quat xyzw]
      intrinsics.npy   # (nf,4) normalised, constant per clip
      poses.json       # K, FOV, descriptor, prompt, provenance
      preview.jpg      # middle frame
      campose.png      # PermaVid camera-path render
  <Scene>/diversity.json         # selection report + marginal gains
  manifest.parquet / .csv   # one row / clip · 47 columns
  scenes.parquet            # one row / scene
  splits.json               # scene-level train / val
  contact_sheets/           # per-scene grids + overview
  README.md · qa_report.md

honesty notes

Read before use

Depth is relative. depth.mp4 is 8-bit, ~15–20 grey levels/frame, lossy H.264 — ordinal, not metric, not 16-bit. depth_format = mp4_8bit_grayscale_nonmetric.
78-scene ceiling. PermaVid ships ~78 usable UE environments with archives; that is the hard cap. 7 scenes have fewer than 20 usable trajectories.
Intrinsics are a reader default. 90° hFOV / 60° vFOV is PermaVid’s default CineCamera pinhole — pose.json carries no focal length, so this is the convention, not a measured value.
Native 20 fps. Relabelling to 16 fps is an apparent-speed change — document it, don’t resample.
QA-clean. All 1,527 clips pass the QA gates (pose finiteness, frame count, decodability).

Source & attribution. Curated from the PermaVid dataset (ysmikey/PermaVid_datasets, arXiv:2606.16449, gated). RGB is a lossless stream-copy of PermaVid’s rgb.mp4; poses/intrinsics are re-expressed in PermaVid’s own training convention, validated bit-identical to its reader. Redistribute under PermaVid’s terms.