Accepting Agent-Generated 3D: What Needs to Change?

When a check or reviewer flags a problem in a 3D scene an agent built for me, I first ask what that finding changes for the intended use. Then I want to understand its cause before assigning a repair. A missing material needs a scene repair. An unstated delivery rule needs a decision about the brief. A checker that cannot measure valid geometry needs a different response again. If I send all three back as “fix the scene,” I can get a greener report without resolving the reason it failed.

The dairy and distribution-centre builds made that concrete for me. Broken material bindings in the dairy pointed objects at materials that did not exist and needed repair. Missing authored normals, the surface-direction data used for shading, came from a policy I had given the checker and left out of the distribution-centre brief. Adding normals fixed that delivery. Stating the policy in the next brief fixes my part of the problem.

My tool, Scene Acceptance, runs chosen checks against saved OpenUSD (USD) scenes and retains the findings and evidence. It can identify some technical causes, but it does not yet make this whole classification. A scripted failure still asks for a scene repair, even when the rule itself needs review. I use the categories below to review that request before sending it. The code on GitHub that assigns each finding a next action makes this limitation inspectable.

The categories I use

I sort findings into six causes, grouped by whose work changes first: the delivery, the brief, the evidence or the evaluator. A valid variation is an outcome reached after that review. The cause is separate from who found the issue, whether the requirement is mandatory, and its low, medium or high human-review risk. The harness article explains the roles.

Examples and next actions for each category

Scroll sideways for more columns.

Finding type	Example from this work	What should happen next
Scene defect: the saved scene is internally broken	Dairy bindings refer to materials that do not exist.	Producer repairs the references and rechecks the saved scene.
Requirement miss: the delivery misses an agreed outcome, or one that follows from the stated purpose	A route indicator contradicts the parcel’s path, despite correct arrival timing.	Confirm the intended behavior, repair the scene, and check the relevant crossing interval.
Scope gap: a requirement is unclear, conflicting, or unstated and not implied by the stated purpose	Authored normals were required by the checks but absent from the brief.	Caller or owner resolves the requirement, communicates it, then requests any needed repair.
Evidence gap: there is too little evidence to decide	Application captures and interaction measurements were missing from the first large-scene delivery.	The evidence provider supplies records from the delivery under review. Then assess whether a repair is needed.
Checker error: the checker gave a wrong answer	A dependency reader falsely accepted three faulty material test cases.	The maintainer fixes the checker and reruns it before any producer repair is requested.
Measurement limit: the checker cannot measure what was delivered	After its correction, the clearance check reports a skinned mesh as unknown until baked geometry is supplied.	Choose an evaluator that can measure it, or agree a supported representation in the brief. Do not ask for a repair by default.
Valid variation (an outcome after judgment): the flagged behavior is allowed or requested	A requested low-friction variant slides beyond an advisory comparison threshold.	Keep the variant and its measurement. The owner decides whether it is useful within the stated limits.

Classify each finding, rather than the whole scene. If the expected behavior is disputed, I resolve that scope gap before calling it a requirement miss. A valid variation is never where I start; I only reach it after checking the result against what the brief asked for.

Example: one distribution-centre delivery

The distribution centre is a snapshot of how I would use the framework. Its first delivery drew eight feedback items, including scene repairs, missing evidence and a delivery rule I had left out of the brief.

Four items asked for scene repairs: a route indicator that showed the wrong branch while a parcel passed, overhead services hanging below the 14 m clear height, an unsupported gate sign and a status tower missing its connecting hardware. Three asked for evidence: an overview camera blocked by an overhead tray, a wide video too coarse to read, and the application captures the producer had disclosed as missing. One, the normals rule, belonged in the brief; it covered more than 121,000 meshes but counts as one item. The material and clearance controls below cover the checker branch.

Supplying evidence can expose a decision as well. The producer measured about 4.5 draw callbacks per second while orbiting the scene in Blender, which I did not rerun, and the brief had no performance target, so I still had to decide what this use needs. The two-scene case study shows the route-indicator repair, and the feedback record keeps each finding with its recheck.

How I counted the feedback

The original eight-item ledger did not use these categories; I grouped it retrospectively for this article. One item came from an automated rule, five from separate agent review, and two from producer disclosures confirmed by review. This was not a blinded trial. All 24 scripted brief checks passed before and after repair.

The unit is a recorded feedback item, counted once by its primary acceptance problem. It is neither a complete defect inventory nor a measure of severity, affected assets or time saved. Other builds, planted test faults and reruns are excluded. The saved-camera ambiguity changes the totals, but either grouping still calls for scene work, evidence and a policy decision.

A passing check still has a boundary

Another view from the distribution centre shows where a passing check stops. The maintenance gate passed its scripted checks for sampled open and closed holds and rotation about its hinge. The sign beside it floated in space.

First delivery: the gate check passes; its sign has no support.

Repaired delivery: a post, base plate and fasteners support the sign.

The brief asked for a realistic, editable facility, and review turned that broad expectation into a concrete repair request: give the sign plausible mounting or move it onto supported geometry. A separate review agent found it; I don’t know whether the harness’s optional judge would have.

The added post and base now occupy space beside a moving gate. If the scene is used to assess passage or interference, that hardware could affect the decision, so the repair needs evidence for the relevant clearance as well as its appearance. The matched stills show the mounting, not clearance throughout the motion.

The gate checker had answered its own narrow question correctly. For the next delivery I would keep its motion checks and assign equipment mounting to review or a suitable new check, because a report that says only “gate checked” hides that difference.

Building for the purpose

Nobody wrote “mount the sign” into the distribution-centre brief. A brief never lists everything a scene has to get right. When I accept a digital twin, what I care about is whether it serves the purpose it was built for, and a good producer brings a sense of the domain to that instead of building only what the project team or customer happened to write down.

Those assumptions shift with the purpose. If the twin is only for a layout review, a status tower without its connecting hardware may be fine. If maintenance staff will train on it, that hardware is part of the job. A conveyor modelled as a simple box can support a throughput study, but a robot learning to pick from it needs real dimensions and collision geometry. The same scene can be complete for one use and wrong for the next, which is why I would rather have a producer build for the purpose and tell me which domain assumptions it made than one that only meets the spec. Anticipating where the use case will go makes a producer better, but I can only accept against the purpose I stated; if the next use matters now, it belongs in the brief.

The distribution-centre brief asked for a realistic facility, so the floating sign and the status tower without its hardware missed that purpose even though no line named them. I count those as requirement misses. The normals did not follow from any stated use. They came from my checker’s convention, so they were a scope gap. Where the producer had to guess, I want the guess declared, so the owner can confirm it or change the requirement before acceptance. The harness can’t judge domain sense, but it can keep those declared assumptions beside the result and route the open ones to a person.

I want to preserve that room for judgment. An exhaustive list of production instructions would remove much of the reason to delegate. Useful approximations should come with enough context to decide whether they affect the job, without making every routine choice a review item. The floating sign shows an omission; it does not tell me whether the agent deliberately chose a shortcut. If I accept a declared approximation, I would keep its use limits with the decision, and agree a change first if it conflicts with an explicit requirement.

What the wider evaluations showed

The taxonomy draws on a wider record than these two builds. The 68 delivered scene artifacts came from 47 completed AI production attempts, checked with progressively improved versions of the harness. That includes early layout studies, other agents’ multi-artifact submissions and a four-scene adopter exercise, as well as larger builds such as the roughly 125,000-prim distribution centre. Two groups recur below: 30 small brief-study scenes from two studies of how much detail the brief gave, and five later workflow scenes, the adopter exercise plus a new inspection-and-palletizing cell. A submission sometimes contained several planned scenes or parameter variants. Repairs, repeat checks and deliberately broken test copies do not increase these counts.

What is included in the 68 scenes and 47 production attempts?

Scroll sideways for more columns.

Creation exercise	Production attempts	Scene artifacts
Early layout edits	6	6
Original animated assembly	1	1
Three submissions of trays, panels, mechanisms and physics variants	3	24
Simple prompt versus detailed brief	6	6
Four brief depths across gantry, rotary and pump/valve scenes	24	24
Dairy and distribution-centre builds	2	2
Other-agent adopter exercise: packaging, conveyor, skid and storage aisle	4	4
Inspection-and-palletizing cell	1	1
Total	47	68

The 36 layout and brief-study deliveries used constrained direct generation, with much smaller assignments than the facility builds. Each of the three multi-artifact submissions supplied eight planned scenes, including paired files and requested variants; those are correlated outputs from one attempt. The last cell was built and evaluated by the same coordinating assistant. The adopter exercise used separate agents but one model family across builders and reviewers. These are exploratory exercises, not 47 independent production deployments.

Two conflicting early requests were appropriately refused and produced no scene, so they are outside the completed-delivery count. Older saved-demo audits are also excluded. The dairy’s initial failure is documented in its run records, but the failing scene itself was not preserved. Checks and policies changed between exercises. The observation data, brief-study tooling, brief-depth study, two-build evidence and later replay summary keep those conditions separate.

Across the 68 delivered scenes I have checked so far, my read is that agents can often produce usable USD structure. The five later workflow scenes all passed their selected general checks, but 27 of the 30 small brief-study scenes had normals or extent-policy findings (an extent is an object’s stored bounding box), and the dairy had broken material bindings. I keep those checks. What I now watch more closely is the gap between a valid scene and the result I meant to receive: an unstated policy, a behavior checked too narrowly, or a decision I cannot inspect from the supplied evidence.

Some deliveries missed supplied requirements. Some were correct within the selected checks but still lacked evidence for their intended use. Others exposed restrictions in my evaluator. The chart keeps those observations beside the group and test that produced them.

The image result changed what I expect from a detailed brief. Six of the nine scenes given an exact image reference still missed its source pixels, although their supplied geometry and motion checks passed. That gives me a reason to check the delivered content even when the agent received the reference. It does not establish a general image-failure rate: these were small palette images produced through a constrained interface.

The later workflow tests changed what I expect from my own checker. Four briefed scenes encountered limits in the clearance check, which measures the gap between objects; all five scenes in that group passed general validation. In the new inspection cell, which my coordinating assistant built and evaluated, the assistant’s own review of the source and previews, outside the frozen checks, found a 185 mm gap between the lift bar and carriage. Both those checks and the visual judge, an AI model that reviews rendered views, had missed it. A clean general report, a failed measurement and a real scene defect can coexist in the same delivery.

How the broader chart was assembled

The two small brief studies contribute 6 + 24 scenes. Their general-profile findings occur in 6 + 21 scenes. Only 3 + 6 received the exact reference image during creation; 1 + 5 missed it. Requirements withheld from the producer are excluded from that miss count. The texture article explains what the pixel comparison can establish.

The later workflow group combines the other agents’ four-scene adopter exercise with the inspection cell. The unbriefed packaging scene had no clearance request and is excluded from the four-scene clearance denominator. The first-delivery measurement limits are retained even though later harness changes addressed them.

The 24-scene submission result is the final selected artifact assessment after evaluator corrections, with producer scenes unchanged. All three mechanisms still lacked proof of continuous attachment. Likewise, two of the six early layouts passed geometry checks while lacking the requested physical evidence. A missing proof is not a confirmed defect.

The original assembly is counted once even though several articles and checks reuse it. Both facility builds had general validation findings, but one was a broken binding and the other an unstated normals policy. Separate reviews and producer disclosures supply additional observations; speculative audit questions and model risk recommendations are not counted as confirmed defects. Planted fault tests remain separate evidence about the checker.

The chart data and counting notes preserve these denominators. I have not assigned every historical finding to a taxonomy bucket: the coverage and detail of those reviews differ too much to claim where most defects lie across the whole body of work.

Sometimes the checker is the problem

A checker can be wrong, or it can be right that it cannot measure what it was given. I keep those apart because they lead to different work.

The material checks once accepted three faulty test cases because their dependency reader, which lists the files a scene references, missed a texture reference. That was a checker error. The reader needed fixing, and the cases needed checking again before those acceptance results were worth anything. A later constructed clearance control found the same kind of wrong pass: the rest-pose mesh had a 2 m gap, but skinning (deformation driven by a skeleton) moved its surface onto the other one. The correction turned that checker error into a measurement limit, and the check now reports skinning as unknown, so the caller has to supply baked, already-deformed geometry or pick an evaluator that handles deformation.

A measurement limit should not become a repair request by default. Asking a builder to rebuild a valid scene so that my checker can read it moves my problem onto their side of the handoff.

The opposite risk is rejecting a result the task allowed. In the friction-variant build, a requested low-friction block slid 664 mm down its incline, far beyond an advisory 10 mm comparison taken from an earlier stay-put test. Only after judging it against the brief could I call it a valid variation, and raising the friction to turn the comparison green would have undone the difference I had asked for.

The case for just retrying

The strongest objection is that retries are cheap. An agent can regenerate a scene without complaint, so sending everything back and letting it iterate until the checks go green looks cheaper than my time spent classifying findings.

For a scene defect, I agree. The dairy bindings needed no classification beyond the validator’s message, and a repair was the right response. But the agent can also supply missing evidence or revise a generator if I ask it to. The problem is assigning the wrong job: remodeling valid geometry to suit my checker, following a requirement I wrote incorrectly, or removing a variation I requested.

A later self-test, the four-scene adopter exercise, showed builders spending repair rounds triangulating meshes and closing surfaces so the evaluator could measure them. The original deliveries subsequently passed under a more capable evaluator and revised policies. Those policies changed object selection, allowed open surfaces, accepted boundary contact within 1 mm and reconciled the port vocabulary. They also added checks the old evaluator could not run, so the conveyor went from 22 to 24 checks and the skid from 9 to 13.

The retest record separates those runs. The new policies were written after the earlier failures were known, so this is not a code-only improvement. All builders and reviewers in this synthetic exercise used Claude. I take a narrower lesson from it: settle measurement support and delivery policy before asking the builder to remodel a scene for the checker.

For now, this is how I am approaching acceptance in my own work. As I try more scenarios and run into new problems, I will deepen the harness where the intended use calls for it. I want each extension to answer a concrete question I could not settle with the checks and evidence I had.

Whether this approach saves work still needs a comparison that counts correct outcomes and reviewer effort, as well as repair rounds.

What I would send back to the builder

Before a finding goes back, I classify it by whose work changes first. Only a scene defect or a requirement miss starts as a repair request to the producer; resolving a scope, evidence or checker problem may reveal a repair or show that the original result is valid.

When the owner changes a requirement, the next report should show that decision, rerun the checks it could disturb and leave any dropped check marked unassessed. The harness keeps a delivery’s requirements, evidence and next actions together, but resolving the requirements, supplying the captures and deciding whether the scope covers my intended use stay with the owner, the calling application and me.

A useful finding names the object or behavior, the observation, the requirement or policy behind it, who acts next, and what evidence would close it. For the gate sign, that means a supported mounting, views showing it, and a check that the new hardware leaves room for the gate to move. “Improve the scene” gives the builder much less to work with.

Disclaimer: The views and opinions expressed in this account are those of my own and do not represent those of my employer, NVIDIA.