I Built a Live USD Viewer That Never Launches Kit

I wanted to know if the library-first pitch — build real things on Omniverse without ever touching Kit — actually holds up for something with genuine interactivity, not just a static render. So I built one from scratch: a browser-based USD viewer that renders server-side at full RTX quality and streams the frames live over WebRTC, with no Omniverse install and no GPU required on the client. Took a few hours, start to finish.

Demo screenshot: the browser-based USD viewer showing a 3D scene in the viewport with the prim hierarchy and inspector panels

▶ Demo video — on the GitHub release page.

The stack

Everything runs as plain Python and a normal npm frontend — no .kit file, no extension, no app to launch:

ovrtx — the RTX renderer. Loads a USD stage internally and renders a frame every time you call step().
ovstream — the WebRTC server. Handles signaling, the data channel for camera/input events, and the actual video pipeline.
warp-lang — a small GPU kernel to convert the renderer’s RGBA output to BGRA before it hits the stream encoder, entirely on-GPU, no CPU round-trip.
pxr (usd-core) — the open-source USD library, for every read/write the HTTP server itself needs: walking the prim hierarchy, editing attributes, building session layers, generating baked animation.
React + TypeScript + Vite on the frontend, talking to the server over a plain REST API.

pxr and ovrtx are not the same stage — that turned out to matter a lot (more below).

What it actually does

By the time I stopped adding to it, the viewer had grown well past “stream a frame to a browser”: a scene browser that lists USD files and loads them on demand, a lazy-loaded prim hierarchy tree, GPU picking (click anywhere in the viewport, get back the prim under the cursor), a prim inspector for type/transform/visibility/variants, transform editing that writes straight back to the stage with a live reload, session-layer authoring (create Sphere/Cube/Cylinder/Xform/DomeLight prims, deactivate prims, undo/redo), camera bookmarks, timeline playback controls, PNG snapshot download, and a telemetry mode that binds prims to motion channels (oscillate, rotate, alert pulse, conveyor) and plays back live animation on the stage. Every one of those is also exposed over a REST API, so none of it is locked to the bundled frontend — any script or client can drive the same server.

That’s basically USD Composer’s core editing feature set, minus Kit, running as a Python process that starts in seconds.

Architecture

Architecture diagram: Python server (ovrtx renderer, Warp color-conversion kernel, ovstream WebRTC server, scene_loader) on the left; React/TypeScript browser (Viewport, Inspector, Telemetry, SceneList) on the right; WebRTC carries frames and input events one way, REST carries commands the other way.

The Python server is the only thing that touches the USD stage and the only thing that renders. The browser is pure UI — it sends commands over REST (load scene, pick a pixel, edit a prim) and receives rendered frames plus server events over WebRTC. Nothing runs client-side except the React app.

Where I actually got stuck

The single biggest architectural cost of skipping Kit: pxr and ovrtx hold separate stages. pxr opens the USD file on disk; ovrtx has its own in-memory copy for rendering. They don’t share state — every edit has to go through a file round-trip:

# pxr writes to its own copy of the stage, not ovrtx's
stage = pxr.Usd.Stage.Open("/path/to/scene.usda")
prim = stage.GetPrimAtPath("/World/Table")
prim.GetAttribute("xformOp:translate").Set(Gf.Vec3f(1.0, 0.0, 0.0))
stage.GetRootLayer().Save()

# ovrtx has no idea anything changed until you force it to reload
renderer.open_inline_root(usda_string)   # ~2 second blink in the stream

That reload is a visible black flash on every single edit — drag a transform slider, see the stream blink. Session-layer authoring (the create/deactivate/undo-redo feature) makes this worse: since there’s no proper session-layer API available outside Kit, I ended up hand-generating USDA strings and injecting them as a sublayer. That’s fragile in a specific, annoying way — a single formatting error in the generated string doesn’t throw, it just silently drops the edit. The telemetry feature made this sharper still: baking 720 frames of animation into a timeSamples block is mostly string templating, and a missing comma anywhere in those 720 entries quietly kills the animation with no error at all. I lost real time to exactly this — the fix was writing a small validator that re-parses the generated USDA before injecting it, specifically so a bad template fails loudly instead of just not animating.

Here’s the actual cost of that choice, side by side with how a Kit app avoids it entirely:

	`pxr` (what I used)	Shared-stage model (how Kit does it today)
Stage model	Separate copy from the renderer	One shared object — Kit, the renderer, and your extension all hold the same `pxr.Usd.Stage`
Edit → visible	Save to disk, then force a full reload (~2s blink)	Next frame — no save, no reload
Session layer	Hand-built USDA string, injected as a sublayer	`stage.GetSessionLayer()` — a real API
`timeSamples` write	String templating, silently breaks on a formatting error	Direct `attr.Set(value, time_code)` call
Failure mode	Silent — a malformed string is just skipped	Typed exception at the call site
Available today	✅ Yes, standalone, no Kit needed	✅ Yes, but only inside a full Kit app

Kit gets this for free because omni.usd.get_context().get_stage() hands your code the exact same Python object the renderer is already holding — there’s nothing to synchronize. Outside Kit, the library-equivalent of that is still a work in progress — the same extraction that already happened for ovrtx and ovphysx just hasn’t landed for stage management yet. So the file-round-trip cost above is an interim tax, not a permanent one: it’s what you pay today, before that piece reaches parity with what Kit already does natively. I’m hoping that lands very soon, and I’ll be first in line to rebuild this against it once it does.

The other mistakes were smaller but cost real debugging time:

Two ovrtx renderers on one GPU deadlock. The CUDA driver serializes access and neither process can proceed. I hit this by accident restarting the server without killing the old process first — now step one of every restart is confirming the old process is actually dead.
Cold start burns about 90 seconds compiling shaders the first time ovrtx renders a frame. Warm restarts are ~15 seconds. Not a bug, just something you have to know to wait for before assuming the server is broken.
WebRTC reconnect after a disconnect can come back as a black frame — the underlying stream handle corrupts and doesn’t recover on its own. Fix is a server restart plus a hard client refresh, not anything fixable from the browser side.
On Python 3.13, UsdTokens isn’t exported from pxr anymore. Cost me an import error before I found UsdGeom.Tokens as the replacement.

What I actually took away from it

None of the above is a knock on the pattern — it’s the normal cost of building on libraries that are still finding their edges, and every one of these was a real debugging session that taught me something, not a dead end. The core claim held up completely: a fully interactive, Composer-grade editing tool, built and running in a few hours, with zero Kit install anywhere in the stack. If someone asked “so what’s the catch,” the dual-stage problem is it — but it’s the one piece of this stack I’m actually excited is still in motion.

Code, full setup instructions, and the complete REST API reference: github.com/pr9868/omniverse-realtime-viewer. Status: working, and still my own testbed for this stack.

Disclaimer: This is my own individual opinion, formed from my personal experience working with this technology. It does not represent an official NVIDIA position. I may have gotten something wrong or missed a detail — always happy to be corrected.