Brief
Study probes when LLM draft-verify-revise pipelines disagree on 'previous'
A preprint abstract reports that in draft-verify-revise LLM pipelines, a context-dependent expression such as 'previous' can be resolved differently by different stages, producing a deictic shift. The study used a synthetic dataset of 10 base examples in three conditions and tested six models across 21 reasoning effort configurations.
Balanced accuracy ranged from 0.156, below chance, to near-perfect. GPT-5.2 rose from 0.156 without reasoning to 0.942 at its highest reasoning effort level, while Gemini 3 Pro stayed above 0.94 at every level. Gemini 3 Pro at low reasoning effort outscored GPT-5.2 at xhigh reasoning effort for roughly 5% of the cost per trial. When the meta-evaluator erred, it tended to rely on surface cues rather than operational reasoning. The abstract advises context engineers to make the intended referent explicit at each stage.
Our reading
Our reading is that this is a controlled synthetic demonstration, so it shows a possible failure mode rather than a settled rate for real-world pipelines.
What to do or watch
For now, treat this as a possible failure mode in draft-verify-revise pipelines: when a context-dependent term like "previous" must be resolved, make the intended referent explicit at each stage. The unresolved question is whether the same deictic-shift pattern appears in real-world pipelines beyond this synthetic 10-example, three-condition test, and whether that explicit-referent fix actually reduces errors.
Source details and supporting facts
Each line is stated by the page named above it.
Stated by arXiv
- Draft-verify-revise is a common LLM orchestration pattern for scaling inference-time compute.
- The phenomenon was studied with a synthetic dataset of 10 base examples, each rendered in three conditions.
- Six models from three providers were tested across 21 reasoning effort configurations using e-values for sequential testing.
- Balanced accuracy ranged from 0.156, below chance, to near-perfect.
- GPT-5.2 rose from 0.156 without reasoning to 0.942 at its highest reasoning effort level.
Sources
- arXivText stored 14 September 2026
How this story was checked. Written from the 1 page listed above, stored 14 September 2026; claims checked against that stored text on 14 September 2026.
What that means
- 5 of 5 reported statements were confirmed against the page that carries them; the rest were removed rather than published.
- Figures in the text were required to appear in the stored source text: yes. Identifiers: yes.
- The check reads stored text only: no claim rests on a fresh look that did not happen.
- Where the reporting was silent, the text says so instead of filling the gap.