Counting
Atomic claim“Three white candles arranged in a single row on the round table.”
Preference: A. Counting evidence identifies three candles in A and two in B.
Spatial Preference Examples
Layout
Atomic claim“Three sunflowers arranged in a single row on a wooden bench.”
Preference: B. Both contain three flowers, but A is a cluster and B is a single row.
Relative position
Atomic claim“A turtle is to the viewer’s right of a diver, with a coral reef.”
Preference: B. Object coordinates place the turtle left of the diver in A and right of the diver in B.
Preference-verification examples, not before/after generator outputs. Evidence notes summarize source cases, not live inference.
Abstract
Reinforcement learning (RL) is promising for spatial text-to-image generation, but existing work largely focuses on policy optimization, assuming accurate rewards and effective online rollouts. We observe in-domain gains with out-of-domain degradation, exposing two limitations: unreliable spatial rewards in open-world settings and rollouts lacking basic-to-complex progression. We propose a curriculum-guided RL framework. First, we introduce SpatialCritic, an agentic reward model equipped with a spatial perception harness that combines reusable verification guidance with specialized vision tools. We then construct spatial preference datasets through targeted counterfactual perturbations and perform agent–harness alignment to improve tool selection and evidence integration efficiently. Second, curriculum learning organizes shared foundational capabilities and their compositional dependencies in a capability graph for mastery-guided rollouts, periodic review, and targeted remediation. On SpaRW-Eval, SpatialCritic achieves 95.08% verification accuracy, outperforming 320B GLM-5.3-Flash with a 40× smaller 8B backbone and raising average baseline scores from 78.4 to 83.9 on SD3.5-Medium and 83.2 to 86.5 on BAGEL.
Motivation
RL effectiveness depends on rollout organization, reward accuracy, and policy optimization. Existing work primarily advances the third, often assuming the first two are reliable. In spatial generation, benchmark-derived rewards can improve in-domain scores while out-of-domain performance deteriorates.
Generalization collapse. ID accuracy improves while OOD performance eventually falls.
Two Questions
Q1Are reward models accurate and robust enough in open-world spatial scenarios?
Q2Can rollouts support progressive learning from basic capabilities to complex compositions?
SpatialCritic
Reward-modeling paradigms. SpatialCritic connects prompt decomposition, tool use, and visual evidence.
Spatial Perception Harness
Two routes to grounded verification. A fixed routine handles a familiar depth relation; composed tools handle a multi-object orientation constraint.
Agent–Harness Alignment
The agent decomposes a prompt into atomic claims, plans tool calls under uncertainty, integrates observations, and produces a preference verdict. Verifiable outcome rewards and a token-budget penalty align tool selection and evidence integration. With the same 8B backbone, accuracy improves from 92.91% to 95.08%.
Spatial Preference Dataset
Constraint-targeted preference construction. Generated images must exhibit the intended visual contrast.
- Scene settings
- 36
- Task categories
- 13
- Training pairs
- 4,389
- SpaRW-Eval pairs
- 5,572
Curriculum Learning
Compositional capability graph. Foundational skills unlock compositions; review identifies weaknesses for remediation.
Experiments
Spatial Preference Verification
Qwen3-VL-8B84.94
+ spatial harness92.91
+ agent–harness alignment95.08
0%100%
Same backbone, better verification. The 30B harness-equipped variant reaches 95.53%.
Best-of-10 Selection
70.76Selection-free baseline
→
74.44SpatialCritic
SpatialCritic selection improves overall accuracy by 3.68 points on SD3.5-Medium.
RL Post-Training
SD3.5-Medium78.4 → 83.9Average score
BAGEL83.2 → 86.5Average score
SpatialGenEval spatial accuracy reaches 69.5% and 77.8%, respectively.
| Method / reward | TIIF BR | TIIF AR | TIIF RR | UniGen 2D | UniGen 3D | SGE Basic | SGE Spatial | Avg. |
|---|---|---|---|---|---|---|---|---|
| Base model | 82.2 | 77.6 | 68.0 | 82.9 | 81.4 | 92.0 | 65.0 | 78.4 |
| Flow-GRPO / SpatialScore | 83.4 | 80.9 | 69.4 | 92.1 | 78.7 | 93.7 | 67.3 | 80.8 |
| DiffusionNFT / GenEval-2 | 85.1 | 83.7 | 73.4 | 89.0 | 82.8 | 94.0 | 68.3 | 82.3 |
| Flow-GRPO / SpatialCritic † | 83.3 | 80.3 | 69.8 | 93.3 | 85.3 | 94.4 | 67.0 | 81.9 |
| SpatialCritic + curriculum | 86.2 | 83.9 | 74.1 | 93.3 | 85.3 | 95.0 | 69.5 | 83.9 |
| Method / reward | TIIF BR | TIIF AR | TIIF RR | UniGen 2D | UniGen 3D | SGE Basic | SGE Spatial | Avg. |
|---|---|---|---|---|---|---|---|---|
| Base model | 86.4 | 82.5 | 73.8 | 91.8 | 82.8 | 94.2 | 70.9 | 83.2 |
| DiffusionNFT / GenEval-2 | 87.6 | 80.7 | 77.3 | 90.6 | 87.3 | 92.2 | 70.0 | 83.7 |
| Flow-GRPO / GenEval-2 | 86.9 | 80.9 | 74.9 | 90.8 | 84.0 | 80.5 | 65.6 | 80.5 |
| Flow-GRPO / SpatialCritic | 86.1 | 82.2 | 74.8 | 93.2 | 82.7 | 93.6 | 73.1 | 83.7 |
| SpatialCritic + curriculum | 88.3 | 83.9 | 77.1 | 94.6 | 89.3 | 94.7 | 77.8 | 86.5 |
Avg. is the unweighted mean of the seven benchmark scores shown, excluding human-preference metrics. TIIF = TIIF-Bench; UniGen = UniGenBench++; SGE = SpatialGenEval.
† The SD3.5-Medium non-curriculum setting uses GenEval-2 prompts without its benchmark-specific questions. The manuscript reports same-pool, matched-budget controls.
Paper
The manuscript includes the spatial harness, dataset construction, optimization details, extended comparisons, and curriculum controls.
Author information, publication metadata, and public code / dataset links will be added when confirmed.