Rationale
Verdict: advance to full review. The submission is structurally complete and makes a specific, falsifiable contribution: a measured test of an analytical constraint model against EZKL 23.0.5, with baselines, leave-one-axis-out and out-of-range tests (Section 4, Tables 1 and 4), 24 verified proofs (Section 5), a mask-fill/domain-size finding (Section 6), and a multi-seed perplexity study (Section 7). Limitations are stated in detail (Section 10), artifacts are released, and all six arXiv identifiers in the REFERENCE CHECK resolve. The NANOZK entry differs only in subtitle between the paper and the registry, and EVIDENCE [8] shows the paper's title for v1, so it is not a mismatch. On novelty, EVIDENCE [1] (parameter-constraint correlation and proof runtime for nanoGPT) and EVIDENCE [2] (EZKL circuit work scaling with depth, an activation-related separation principle) overlap with Sections 4–5. Neither is cited in Section 2, and the excerpts do not show that the paper's core results (the baseline comparison, the FFN/softmax decomposition, the mask-fill effect) are already published. Because novelty is uncertain rather than refuted, the panel should judge it. Reviewers should weigh three further points: the analytical model is the authors' own formalization (Section 3), which risks a strawman; cost and quality are measured on different models (Section 8); and proving times are noisy (Section 5).
Concerns
- Section 2 does not engage with directly overlapping prior measurements of EZKL constraint/cost scaling for small transformers (EVIDENCE [1], [2]), so the novelty of Sections 4–5 relative to them is unestablished.
- Section 3 tests an analytical model the authors formalized themselves rather than a published one, so the headline finding that it loses to the dimension product risks refuting a strawman.
- Section 5 proving times come from one or two runs on a machine at load average 20–40 with run-to-run ratios of 0.82–1.91x, which leaves the activation-level proving-time comparisons uninformative.
- Section 8 / Table 9 joins row counts at width 128/2 layers/s=16 with perplexity from 384d/6L models that were never compiled, so the cost–quality conclusion rests on an untested transfer of ordering.
- Sections 4 and 10 limit circuits to width ≤512, ≤4 layers and s≤64, and proofs to domain ≤2^21, far from the GPT-2-scale regime the analytical argument targets.
- Section 10(6) leaves quantized accuracy of the compiled circuits unmeasured, so the Section 7 perplexity results do not establish the quality of the circuits actually proved.
decision: advance_to_review · confidence: 4/5
References checked (6): 6 found · 0 not found · 0 unreachable