Fabricated references in AI-first-authored research: a manually verified audit of all Agents4Science 2025 submissions

We are pleased to accept your paper. The independent, label-free coding of the random sample of 45 manual decisions (Appendix A.8) gives a directly estimated fabricated-versus-not agreement of 97.8% (kappa 0.94). The relabelling sensitivity analysis shows that the detected figures remain lower bounds and that the associations keep their sign. You propagated the coder's labels through Table 2b and Appendix A.3, and you revised the §4 independence wording and the §6 causal wording as asked. Please make the editorial corrections listed in the notes before publication. In particular, reconcile the remaining stale corruption-rate phrases with the 6.2% coder-anchored rate. Please also deposit the Zenodo-archived release and report its DOI, so that the repository contents described in your response letter can be checked against the paper.

Verdict: accept. Every round-5 condition is addressed in the manuscript. The remaining points are editorial and do not change the validity of the claims. Round-5 concerns: (1) Independent coding of random sheet B: resolved. §4 describes a coder with no role in the study, working from the reference strings alone. §5.1 and Appendix A.8 report 75.6% category agreement (kappa 0.63, 0.42–0.81) and 97.8% fabricated-versus-not agreement (kappa 0.94, 0.78–1.00), superseding the stratum-weighted 73%/94%. (2) Sheet A coded by the same coder, with agreement against the first adjudicator, the blind agent and the author: resolved (Appendix A.8; the coder's role is recorded in §4). (3) Bounds on detected counts and robustness: resolved. The A.8 relabelling table gives 549 (493–638) fabricated references and 45.2% of reviewed submissions affected; §5.3 gives flag sensitivity 0.83; §5.2 gives 4 (0–13) accepted papers with an invented reference. Table 2b and Appendix A.3 are recomputed with the coder's labels (13.5%, 6.2%, 7.8%). (4) Qualification of the §4 independence claim: resolved in §4, §5.1, §7 and the Appendix A.7 heading. (5) The §6 'kept out' sentence: resolved; it now states an association. (6) Repository contents: stated in the response letter but unverifiable from the supplied evidence (not_enough_evidence). This is reproducibility documentation, not a validity defect in the reported analysis. New issues: residual stale corruption figures remain in §§5.1, 5.2, 6 and 7, and the Data availability section omits the new coder files. All are editorial; the abstract and Table 2b are correct. The competing interest (the author operates the venue) remains disclosed.

Published with the paper; the authors were not required to address them before publication.

• §7 'Undetected errors' says 'about one accepted citation in twelve' for 11 of 180 (6.1%); correct it to about one in sixteen to match §5.1 and Table 2b.
• §6 'What the audit adds' says title matching passes 'about one citation in ten' with wrong metadata; update this to the coder-anchored 6.2% (author-anchored 8.2%, blind-agent-only 9.7%).
• §5.1 calls about 370 undetected corrupted references 'about twice' the 227 adjudicated corruptions, and §5.2 cites 'Nine of the 14 human-confirmed' author-list corruptions; restate both under the coder-anchored counts (about 1.6 times; out of 11).
• Data and code availability: list the independent coder's sheets, CODER_STATEMENT.md, coder_overrides.json, independent_agreement outputs and the README mapping, and add the Zenodo DOI of the archived release.
• §4: state that the coder's independence rests on the coder's signed statement, since the coder was recruited from the author's personal acquaintance.