From Camera Media to Gaussian Splats: Building ReconStudio

I ended Part 1 with a practical question: could someone submit images or video and receive a Gaussian splat, an OpenUSD file and a useful report without operating the GPU pipeline themselves?

I built ReconStudio to answer it. In the browser, a user can choose the media and run settings, start the job, watch the camera solve and GPU training, then inspect the result or open the report. The same application keeps the settings and outcome for later comparison.

The reconstruction methods already existed. COLMAP could recover camera poses, and 3DGRUT, an open-source library from NVIDIA’s Spatial Intelligence Lab, could train the Gaussians and export them. What I wanted to test was the product surface around those steps: one job contract, visible evidence as the job progresses, and a predictable set of outputs at the end.

Camera media is useful here because it is a common input. A team may not have current CAD, may not be able to exchange a supplier’s native format, or may only need the visual surroundings around the asset it is modeling precisely. A useful reconstruction still needs overlap, sharp frames and changes in viewpoint. In the environments I had in mind, access to a phone or camera was a smaller starting barrier than finding a current 3D model. Camera reconstruction offers a low-friction way to create the visual layer when that layer fits the use case.

What I built

ReconStudio accepts a video or a zip of photographs, creates a job, and carries it through eight stages:

Stage	Work	Evidence exposed to the user
Ingest	Read the media and its basic properties	source type, image count, resolution
Frame selection	Keep sharp frames spread across the capture	frames considered, selected and rejected
Feature extraction	Find distinctive points in each image	median and minimum feature counts
Feature matching	Link the same points across views	matcher and completed work blocks
Sparse reconstruction	Solve camera poses and sparse 3D points	registration rate, model count, points and reprojection error
Gaussian training	Optimize the splat representation on a GPU	quality metrics, Gaussian count and GPU use
Export	Write the result into two useful forms	PLY and OpenUSD sizes, prim structure
Ready	Package the result and its evidence	downloads, report, viewer and replay

The dashboard is a view over that job record. It does not reduce a forty-minute reconstruction to a spinning progress bar. Each stage explains what it is doing and puts its current measure beside it.

The distinction matters because the camera-solve failures in my test runs were visible before the final scene loaded. If registration has split the capture into several disconnected models, or only half the images have registered, waiting for more training will not repair the camera solve. The current POC does not expose cancellation, but it turns a poor final asset into a diagnosable run and gives a hosted implementation a concrete signal it could act on.

What the project covers beyond reconstruction

The eight stages describe the compute path. ReconStudio also covers the work around that path, because a sequence of commands by itself is hard for another user to operate or evaluate.

Area	Capabilities included
Run setup	Accept video, a zip of JPEG or PNG photographs, or pre-staged data; control frame count, image width, training iterations and Gaussian budget
Observable jobs	Create and start jobs separately; stream stage changes, logs, timings, camera-solve metrics, quality metrics and GPU use
Inspection and outputs	Preview the splat, inspect the OpenUSD scene graph, open a self-contained report, and download PLY and USDZ artifacts
Experiment support	Replay recorded events, retain failed and completed runs, track metric provenance, and compare budget sweeps within the same scene
Local evaluation and extension	Exercise the workflow without a GPU in simulation mode; replace stage adapters later; apply local upload limits and lifecycle tests

The experiment ledger came from a mistake I nearly made while running the pipeline. I compared scores from two different scenes and almost treated the difference as evidence about Gaussian budget. It was measuring the scenes. The ledger now groups budget sweeps within one scene, records where each metric came from and excludes untrusted values from ranking. Failed runs remain in the count because they were part of the work.

Simulation mode serves a different purpose. It runs real ingest and frame selection, then exercises the API, dashboard, events and report path without fabricating a PLY or USDZ. That makes the user experience and job lifecycle testable on a laptop while keeping simulated execution distinct from a reconstruction.

Together, the ledger, replay and simulation mode let me use the same application as a user experience and as a testbed for the runs in Part 3.

This is the scope I mean when I call ReconStudio a reference implementation. GPU fleet provisioning, identity, tenant policy and cost control sit outside the repository. The job contract and executor boundary give a service builder a clear place to add them.

The API is a job contract

A reconstruction is a poor fit for one long HTTP request. Real runs in this project took tens of minutes, used a GPU for the expensive stage, and produced files that could approach a gigabyte. The API therefore separates upload, execution, observation and download:

POST /api/jobs                         create a job from video or photographs
POST /api/jobs/{id}/start              place it on the local worker
GET  /api/jobs/{id}                    read current state and metrics
GET  /api/jobs/{id}/events             stream live events
GET  /api/jobs/{id}/replay             replay the recorded event stream
GET  /api/jobs/{id}/artifacts/{name}   download PLY, USDZ, report or snapshot

Creating and starting are separate on purpose. Finishing an upload should not mean seizing a GPU. A hosted version can validate the request, apply a quota, wait in a queue and start when a matching worker is available without changing the client contract.

Every event is also written to events.jsonl. Replay sends those recorded events back with compressed timing. It is the same log, stage transition and metric that the real run produced, which lets a completed forty-minute job be demonstrated in a few minutes without pretending that a simulation is the original computation.

The final job record is equally plain. job.json holds the state and metrics, while the artifact directory holds the files. The dashboard, report generator and API all read that same record.

This local state is deliberately limited. Completed jobs remain readable and replayable after a process restart, but the active-job registry lives in memory. A created job that has not started cannot be started after that restart. A hosted version needs a transactional job record or durable queue for that transition. The client can keep the same job model when local disk moves to a database and object storage.

One job, several useful outputs

I did not want the service to end with a single opaque download. The same reconstruction is packaged for different consumers:

Output	Why it exists
scene.ply	Direct Gaussian data for browser and dedicated splat viewers
scene.usdz	An OpenUSD ParticleField3DGaussianSplat asset that can be composed with other scene layers
report.html	A small, self-contained record of settings, timings, metrics, source-frame samples, USD structure and process output
events.jsonl	The recorded event history used to inspect and replay a run
job.json	A machine-readable summary for another service or client

The report became more important than I expected. A three- or four-million-Gaussian PLY takes time to parse and render in a browser. During that wait, a black viewport can mean loading, framing trouble or a broken file. The report opens immediately after the job completes and answers the first questions without needing to render the scene: did the images register, how long did each stage take, what did the USD contain, and how large are the exports?

The PLY and USD also have different jobs. The PLY is the easiest way to inspect the appearance in a tool such as SuperSplat. The USD package carries the scene structure and the standard Gaussian particle-field prim. It still has no mesh surfaces, collision geometry, semantics or known physical scale. Those belong in other layers when the use case requires them, as described in the fidelity article.

The architecture I tested

The reference implementation runs on one GPU machine. FastAPI owns the control plane, a background worker owns the GPU, and each job gets a directory containing its inputs, intermediate data, events and outputs. The worker calls ordinary tools behind adapters: OpenCV and FFmpeg for media, COLMAP for camera poses, 3DGRUT for training and export, and OpenUSD for inspection.

That local shape was enough to test the whole contract. It was also the right place to stop for a public reference implementation. Authentication, tenant isolation and fleet management would have added a large amount of code without making the camera-to-output path any clearer.

The hosted shape follows directly from the boundaries already in the app:

1. The client uploads media, ideally through a signed object-storage URL.
2. The API validates the request and writes a durable job record.
3. A queue holds the job until a worker with the required GPU profile claims it.
4. The worker downloads the inputs, runs the same adapters, uploads the artifacts and emits status events.
5. The worker can terminate when the job is complete. The API and artifacts remain available without keeping the GPU alive.

This is where GPU-on-demand belongs in the design. The API should express the outcome and job state. Provisioning is an executor concern behind the queue. A service builder can scale workers from queue depth and wait time, then route work using requested budget, image properties and measured memory headroom. The pool can scale to zero when no reconstruction is waiting. My later runs showed that Gaussian budget alone was not enough to predict peak GPU memory.

The repository stops at the upper half of the diagram. The lower half shows how I would host it. ReconStudio leaves authentication, quotas, retries, cancellation, signed downloads, observability, cost controls and retention to the team turning the reference into a service. Those are substantial responsibilities. They are separate from proving that one accepted job can move through the reconstruction worker and produce the promised outputs.

A measured run through the contract

The public repository includes one compact sample from a real playroom run. The source photographs come from the Deep Blending scene distributed with the official 3D Gaussian Splatting evaluation data. The run used 225 photographs, registered all 225, trained to a four-million-Gaussian budget and completed in 41.5 minutes on one NVIDIA L40S. The OpenUSD package was 944.1 MB.

I chose this sample for the repository because it exercises the full path and is small enough to inspect as a report, a short recording and a few images. The 944 MB export is also a useful systems result. Large artifacts make object storage, resumable download and retention policy part of the service design long before the API sees heavy request traffic.

The number is not a recommendation to use four million Gaussians. The later one-million and four-million outputs showed no paired-frame PSNR separation, while the larger job took longer and produced four times the output. The complete comparison and its limitations belong in Part 3.

What the build established

The build answers the question that started it. Once an operator installs the toolchain on a GPU machine, a user can submit new camera media through one batch interface, follow visible stage evidence and receive a predictable artifact set. The user does not have to operate COLMAP or the trainer by hand for each run.

The build does not establish that arbitrary media will reconstruct well. The interface can expose a weak camera solve; it cannot recover missing views, remove motion blur or reinterpret a capture rig that the pose configuration does not represent. Those limits showed up repeatedly in the experiments behind Part 3.

ReconStudio is a reference implementation and a working POC. It runs the real tools, records real jobs and produces the real artifacts. It binds to localhost by default, applies upload and expanded-archive limits, provides a simulation mode and has tests around the job lifecycle. It is intentionally single-node, single-worker and unauthenticated.

That is enough for someone evaluating the service shape, reproducing the pipeline or replacing one adapter with another. A production owner still has to decide how identities, tenants, failures, spending and data retention work. I would rather make that boundary explicit than bury a small amount of incomplete production code inside the demo and imply the hard parts are handled.

The code, setup instructions, architecture notes, sample report, images and the 58-second recorded replay are in github.com/pr9868/reconstudio. In Part 3, I cover the experiments behind it: 24 job records across six scenes, the measurements I initially got wrong, and what the comparisons said about capture, Gaussian budget, GPU memory and the limits of the output.

The next project starts from that exported splat. SplatStage asks what it takes to clean, scale, compose and package it as a visual-ready twin foundation.

Disclaimer: The views and opinions expressed in this account are those of my own and do not represent those of my employer, NVIDIA.