Brief
Preregistered test finds evidence masking improved held-out compositional accuracy, with attribution unresolved
A preregistered study on arXiv reports that evidence masking improved held-out compositional accuracy in sixty four-cell systems, but key attribution questions remain unresolved.
The study compared five conditions that varied evidence masking, ownership markers, and replacement of foreign evidence with neutral filler. It used sixty four-cell systems with a frozen language-model backbone, six initialization clusters, two data orders each, and one fresh task world. With markers available, masking improved accuracy on held-out two- and three-operation compositions by median paired differences of 0.846 and 0.859; all twelve pairs cleared the required margins, and the preregistered behavioral criterion passed. The unmarked replication also passed. But no globally visible system passed the marker-following check, and the filler condition's decomposition criteria were inconclusive. The packet audits, while consistent with predicted intermediate-value changes, do not establish mediation.
Our reading
This is an arXiv preprint with public protocols, results, and checkpoints, so the measured effect is clear but not yet a settled finding. The useful separation is between a large masking advantage and the authors' proposed attribution to usable role information, which the marker check did not resolve. Readers who follow AI generalization should watch for peer review, independent replication, and…
What to do or watch
Watch for independent replication and a follow-up that isolates ownership markers and packet-level attribution; the precise unresolved question is whether the masking advantage depends on usable role information or on the masking regime itself.
Source details and supporting facts
Each line is stated by the page named above it.
Stated by arXiv
- The study tested sixty four-cell systems sharing a frozen language-model backbone and communicating through learned continuous packets.
- Five conditions vary evidence masking, ownership markers, and replacement of foreign evidence with neutral filler, across six initialization clusters, each with two data orders, on one fresh task world.
- With markers available in both regimes, masking improved accuracy on held-out two- and three-operation compositions by median paired differences of 0.846 and 0.859.
- All twelve pairs cleared the required margins, and the full preregistered behavioral criterion passed.
- No globally visible system passed the marker-following check, so the effect of usable role information remains unresolved.
- The filler condition yielded seven full generalizers, but its decomposition criteria were inconclusive.
- Packet interventions in all eighteen audited masked systems followed the predicted intermediate-value changes on eligible cases; these finite, success-conditioned audits do not establish mediation.
- Protocols, results, and checkpoints are public.
Sources
- arXivText stored 17 September 2026
How this story was checked. Written from the 1 page listed above, stored 17 September 2026; claims checked against that stored text on 17 September 2026.
What that means
- 8 of 8 reported statements were confirmed against the page that carries them; the rest were removed rather than published.
- Figures in the text were required to appear in the stored source text: yes. Identifiers: yes.
- The check reads stored text only: no claim rests on a fresh look that did not happen.
- Where the reporting was silent, the text says so instead of filling the gap.