What 5.8 Million Gaussians Taught Me About Building a Visual Twin

The final scene looked more complete than it was.

Two people and a beacon stood inside a cleaned Garden reconstruction. The splat had become an OpenUSD Gaussian particle field. The stage reopened successfully, its relative references resolved, and the viewport read the final result back from the USD file rather than from the editor’s PLY.

The unresolved scale from Part 1 remained visible. The stage also contained zero colliders, and nothing outside SplatStage had rendered the package.

All of those statements are part of the result.

I use one complete project here to test where SplatStage crosses the readiness boundaries from Part 1, and where it stops. Part 2 covers the application itself.

Why I used the Garden scene

For this run I used the garden scene from the Mip-NeRF 360 dataset through the pretrained model distributed by the official 3D Gaussian Splatting project, not a local camera capture from ReconStudio. I am naming both sources because the images and the trained artifact do not have the same provenance. The 3DGS repository’s project license limits its software to non-commercial research and evaluation. The model gave me a sharp, complex scene with foliage, a sloped site, background reconstruction debris and enough Gaussians to expose scaling problems in the application.

That choice makes this article related to ReconStudio but independent of it, because the input is the same kind of artifact ReconStudio can produce—a Gaussian PLY—while the editing and composition workflow does not depend on which reconstruction system created it.

The source contained 5,834,784 Gaussians, spherical harmonics through degree 3, and a 1,447,027,964-byte PLY. The recorded walkthrough took just over nine minutes across four screen recordings. It covered all seven application steps and produced twelve self-contained HTML evidence snapshots across those steps.

The twelve files form a record with branches rather than a twelve-frame filmstrip. Clean and Remove have multiple before, transition and after states; the Asset step also has a later revisit after the stage existed. I use the recorded creation times, version fields and cumulative activity log to establish chronology.

The timing is demonstration time, not a benchmark. The evidence is in the lineage, exported files, snapshots and package manifest.

The run used the tested topology documented in Part 2: an Ubuntu x86_64 host with 16 EPYC vCPUs, 62 GB of RAM and an L40S, reached from a browser on my Mac. The browser handled the live WebGL view; the L40S handled the full server render. That division matters when reading the timings or estimating what another SplatStage host would need.

The complete run in numbers

The edit history contains three immutable PLY versions:

Version	Operation	Gaussians removed	Gaussians remaining	PLY bytes	What changed
v0	imported	—	5,834,784	1,447,027,964	original Garden reconstruction
v1	isolate robust core	1,990,933 (34.12%)	3,843,851	953,276,580	distant reconstruction debris removed
v2	crop a selected cylinder	78,860 (2.05% of v1)	3,764,991	933,719,300	one intentionally selected object removed

The final PLY retained 64.5% of the original Gaussians. That number combines two edits with different meanings. The first tried to preserve the useful reconstruction while removing statistical outliers. The second deliberately changed the captured scene.

I kept that distinction in lineage so the intentional removal would not look like an algorithmic correction.

Robust bounds turned a vague defect into a measurable one

Before the first edit, the raw scene radius was 60.30 in the reconstruction’s uncalibrated units. A radius based on the second-to-ninety-eighth percentile range was 46.25. After isolating the core, those values fell to 14.21 and 11.33.

Diagnostic	Original v0	Cleaned v1	Change
Raw radius	60.295	14.213	-76.4%
Robust p98 radius	46.247	11.329	-75.5%
Mean opacity	0.339	0.330	similar overall distribution

The deletion count alone is incomplete evidence. The large contraction in both raw and robust extent says the operation removed a substantial outer region. The similar mean opacity is consistent with a filter that did more than delete every faint Gaussian.

Because a smaller radius could still hide a bad edit—clipping the house and foliage would produce an even more compact statistic—I used bounds and counts to reject obvious problems, then inspected the result before treating it as fidelity-preserving cleanup.

This operation also exposed an incomplete lineage record. The version stored the operation name and its result, but not the exact defaults used by isolate_core. With those values left implicit, replaying the same named operation could produce a different result after a code change. The crop in the next step stored its cylinder center, radius, height and axis, which is the level of detail I now expect from reproducible history.

Object removal was geometrically exact and semantically manual

For the second edit, I selected a cylinder around a vase-like object and previewed the affected count before committing the crop. SplatStage removed 78,860 Gaussians and wrote v2.

I gave the application an exact cylinder, not the label vase. The meaning came from my placement of the selector.

The scar is equally important because camera reconstruction estimates the visible scene, and when an object hides the surface behind it, deleting the object’s Gaussians cannot reveal information the cameras never captured. A clean fill would need another source: additional views, an image or scene completion method, or a replacement asset. SplatStage currently performs none of those.

Intentional editing is difficult to score because a PSNR comparison against the original images would reward keeping the object that was present when those images were captured. The proper acceptance test depends on intent: was the selected object removed, were unrelated regions preserved, and is the exposed region acceptable for the next use?

The crop was recorded as fidelity_preserving_lossy, the operation’s default label, even though I used it to remove an object intentionally. I got that wrong in this run. The comparison path then scored versions across an edit it should have refused to treat as a fidelity test. Next time I would ask for the intent of each edit instead of attaching one permanent label to the operation.

The export count connected v2 to USD

Version 2 exported to an 888.5 MB OpenUSD asset containing one ParticleField3DGaussianSplat with 3,764,991 Gaussians.

The count connects the final file to the selected edit version. A valid ParticleField with 5,834,784 records would still be the wrong product. That was the original export seam: a technically valid exporter was reading the reconstruction checkpoint instead of the edited PLY.

I used five checks, each aimed at a different risk:

Check	What it can catch	What it cannot prove
Gaussian count matches v2	stale version, missing or extra records	correct field meanings
Schema field types and array lengths	scalar/vector layout mistakes, missing attributes	correct convention or visual quality
USDZ → PLY → USDZ round trip	packing and read/write asymmetry	external renderer support
Reopen wrapper and resolve references	missing or host-specific dependencies	target-runtime behavior
Read final viewport from USD	live-view/export divergence	another application’s rendering

One early assumption failed even though its arithmetic was perfect. The expected fields added up to roughly 236 bytes per Gaussian, and an incorrect writer produced that width. It used the wrong arrangement for a schema field. I had matched the width without establishing the layout; only a typed readback could distinguish the two.

Gaussian count has a similar limit. It linked this export to v2 because the count was distinctive within this project. A count alone cannot establish universal identity: many trained models are capped at the same round number. Content hashes and explicit parent references provide stronger lineage than a filename or count that happens to match.

Up was measured; scale remained unknown

I fitted a plane from selected ground points. The first normal pointed the wrong way on the sloped garden, so I used the application’s flip control. The lineage records both the resulting normal—approximately [0.0383, -0.6123, -0.7897] in the pre-alignment frame—and flipped: true.

I retained the fitted normal and the manual correction because the correction is part of the measurement. I do not present either one as ground truth.

I left the two-point scale calibration for a later run. The stage still declares:

(
    defaultPrim = "World"
    metersPerUnit = 1
    upAxis = "Z"
)

upAxis = "Z" describes the aligned stage, while metersPerUnit = 1 says that one authored unit should be interpreted as one meter; neither line proves that the source reconstruction was measured in meters.

The asset dimensions make the missing evidence measurable. The person asset is about 3.64 authored units tall, while the final scene extent is about 17.9. Without one known real-world distance, the file cannot tell whether that is a garden, a miniature or a giant figure. Any collider size, mass or gravity-based behavior authored on top of this scale would inherit the uncertainty.

I could have resized the people until the image looked plausible. That would improve the screenshot and weaken the artifact. SplatStage instead labels the scale unknown and preserves world scale at 1.0. In the next pass, I need to perform the measurement and keep the gap visible until then.

The wrapper stage made composition inspectable

After export, I added three existing USD assets: two references to a person asset and one beacon. They are simple engineered stand-ins already expressed as USD. Since SplatStage has no native CAD translator in this build, this run tested the placement and composition of existing assets rather than translation from a CAD format.

The packaged wrapper has the relevant structure:

def Xform "World"
{
    def Xform "Capture" (
        prepend references = @./inbox_garden-30k_v2_z.usdz@
    ) {}

    def Scope "Assets"
    {
        def Xform "Part0183" (
            prepend references = @./person.usda@
        ) { ... }

        def Xform "Part4712" (
            prepend references = @./person.usda@
        ) { ... }

        def Xform "Part4025" (
            prepend references = @./beacon.usda@
        ) { ... }
    }
}

The two person prims reference one asset file at different transforms. The capture remains one independently replaceable USDZ. No 888.5 MB field is rewritten when a person moves.

The final view had to come from the file

The final step opened scene.usda, followed its capture reference, read the particle field and displayed a 400,000-Gaussian interactive sample alongside the part meshes.

The stage inspection reported:

Stage property	Verified value
Up axis	Z
Unit declaration	1 meter per unit; source scale unverified
Default prim	/World
Total prims	25
Gaussian fields	1 ParticleField3DGaussianSplat
Engineered geometry	17 Mesh prims across the referenced stand-ins
Structure	5 Xform, 2 Scope
Physical readiness	0 colliders, 0 physical materials, simulatable: false

This readback proves that SplatStage can consume its own composed artifact, but because I have not yet observed the packaged result in usdview, Isaac Sim or another intended downstream runtime, it says nothing yet about external renderer support. The schema structure and round trip pass; external render-delegate behavior remains a separate test.

The dependency graph made the package relocatable

The authored working stage originally referenced files by their paths on the editing machine. Those references would fail as soon as the stage moved.

The package operation walks the composition dependencies, copies each unique file, rewrites the wrapper to use relative references, reopens the result on the authoring host and writes a manifest. The delivered folder contains:

scene.usda
inbox_garden-30k_v2_z.usdz
person.usda
beacon.usda
MANIFEST.json

There are four reference arcs over three unique dependency files because the person asset is referenced twice. The capture is 888.5 MB; the compressed delivery was about 821 MB. The manifest found no missing references.

I kept this as an inspectable folder and zip so the wrapper and replaceable layers remain obvious. The OpenUSD USDZ specification supports nested packages. The manifest proves that this copy is dependency-complete and relocatable by path. It does not prove that a second runtime can render its Gaussian field.

The manifest also caught a small but revealing trust problem. Its computed fields reported 17 meshes and zero colliders, while an early hard-coded note said “0 meshes / 0 colliders.” The counts were correct; the prose had gone stale. After that, I generated the sentence from the same computed values.

A snapshot became a second evidence layer

The twelve HTML files gave me more than a set of screenshots. Each one kept several kinds of evidence together:

Layer	Question it answers
Embedded render	What was visible at that moment?
Human-readable state	Which asset, version, operation, view source and renderer produced it?
Counts and stage facts	How many Gaussians were drawn versus present, and which parts, meshes and colliders existed?
Cumulative activity	What sequence of actions led here, and how long did important operations take?
Embedded JSON	Can a tool parse the same record without scraping the page?

The snapshots were useful, but imperfect. One Remove file caught the interface during a full-Gaussian download and preserved the transitional warning; a later Asset snapshot shows version 0 while retaining project-level ground and stage facts established after version 2; and two generated summary strings lost their separators even though the underlying JSON remained structured. The records remain usable, but the next version needs a clearer separation between project state, view state and chronology.

For publication I use one rendered page and the derived facts. The raw twelve-file bundle stays in the working evidence set: its HTML contains internal paths and identifiers and ranges from roughly 0.9 to 4.2 MB per page because the render is embedded. A curated public set should sanitize those details, fix the summary formatting and preserve the source snapshots separately as the audit record.

What changed how I think about the next build

Five things from the run changed either the architecture, a test, or what I was willing to claim. The smaller implementation discoveries stay in the build record.

The seam needs its own test

The editor and checkpoint exporter both worked. The result was wrong because the exporter never consumed the editor’s output. An end-to-end lineage assertion—selected PLY version in, same record count in the particle field—was more valuable than another isolated component test.

Store the reason with the result

The final ground normal is less useful without the selected points and a record of the flip. A scale factor is less useful without the two points and entered distance. I cannot reproduce an edit without its selection parameters and operation defaults.

Invariants can reject an answer; typed readback has to verify it

Byte width, Gaussian count and file size are strong alarms when they change unexpectedly. A wrong result can still satisfy all three. Typed round trips, hashes and explicit parentage are needed to say two artifacts are connected.

What each view was actually showing

“Final” was too vague as a view label. Live PLY, server render, USD readback and geometry diagnostic answer different questions, so the source and renderer now travel with the image and screenshot.

Testing the stage after moving it

Once the package moved, I used three practical checks: could it still resolve its dependencies, expose its structure and carry enough evidence for another tool to inspect it?

That includes its unresolved state. I kept scene scale unknown, navigation proxy, and simulatable: false on the artifact so a downstream user would not assume facts the project had not established.

A few quirks I did not chase further

I am keeping these separate from the lessons above. I observed each one, made enough of a local correction to keep the work moving, and stopped. I would not treat them as general findings yet, but they are worth leaving in the record.

• A black viewport did not always mean a broken asset. On a remote connection, the first WebGL proxy for a large scene could take roughly ten seconds to appear while about 141 MB arrived through the SSH tunnel. An earlier recording also contains one black viewport after loading; I could not reproduce its cause. The interface needs a clearer loading and error state before I treat remote viewing as dependable.
• The first gsplat call behaved differently from later calls. Two simultaneous before-and-after render requests could enter lazy CUDA initialization together and return an error. Warming the renderer during startup and serializing render calls solved it for this application. Multi-process and multi-GPU behavior remain untested.
• A saved selection is a predicate rather than frozen membership. Deleting Gaussians renumbers the remaining records, so saving indices could make a later edit target the wrong data. SplatStage reevaluates a selection against the chosen version. That is safer, but a neighborhood or percentile rule can legitimately select different Gaussians after an earlier crop.
• Part placement has two kinds of state. The first “seat on ground” calculation used the part’s current translated position, so a second click moved it again. Anchoring the calculation to the original placedAt value made repeated clicks return the same transform. The final USD preserves that transform, but not a replayable rule such as “seat this part on the measured floor relative to that machine.” Applying the same composition to a new reconstruction remains manual; relational placement belongs in a later build.

What I will build next

The next project starts with the gaps this run measured: establish scale, replace the stand-ins with real CAD-derived assets, add collision and physical properties, and validate the package in the target simulation runtime. Until those checks pass, the result remains a visual-ready foundation rather than a simulation-ready twin.

The three articles in this series now form one progression: the mental model, the SplatStage build, and this evidence record. Together with the earlier ReconStudio series, they cover the path from camera evidence to reconstruction and from reconstruction to a composable visual scene.

The application code used for this run is at pr9868/splatstage. The public repository includes generated-fixture tests but intentionally excludes the Garden model, raw project files and evidence snapshots described here.

Disclaimer: The views and opinions expressed in this account are those of my own and do not represent those of my employer, NVIDIA.