Response to the major-revision decision (Online Quotient Learning)
We thank the editor and reviewers. Every change is listed per concern. One of
our own checks also exposed a seeding defect in the collection code (see
Concern 3); we report it in full.
Concern 1 -- repository, commit, artifact directories, manifest
Section 5 and Appendix F: supply the actual repository URL, commit hash, artifact directories and SHA-256 manifest location that are currently blank, so that Table 1 and every additional check can be audited.
Changed (end of Section 5; Appendix F). The paths were not blank in our
source: the conversion to the submission format silently dropped every
\path{...} macro, which is why the reviewers saw an empty list. All 19
occurrences are now plain code text. Section 5 now gives the public repository
https://github.com/npow/quotient-learning-papers, directory oql/, states that
all paths are relative to it, lists every result directory
(experiments/main_table/results_*), the new per-seed partition CSV, and the
SHA-256 manifest experiments/ARTIFACTS_MANIFEST.json (verified by
python scripts/gen_manifest.py --check). Appendix F repeats this.
Commit: we cite the release tag v1, not a hash. The paper says so and
explains that the commit field inside the JSON sidecars is the hash of our
development repository at generation time and need not resolve in the public
one. The manifest was regenerated and now also covers the new scripts and
results.
Concern 2 -- environment specifications and per-seed partitions
Section 5: give full transition and observation specifications for every custom environment (8-cycle, depth-2 chain, position-flip chain, misspecification chain, Dirichlet HMM) and report per-seed partition assignments or partition error rather than class counts alone.
Changed (new Appendix G; Table 1 caption). Appendix G gives the complete
generative specification of Tiger, the 8-cycle, the noisy depth-2 chain (all
rows, the stationary reset, the r=0 and nested r=1 laws), the position-flip
chain, the misspecification chain, and the Dirichlet HMM (pi_0, A, B to four
decimals, instance seed, scan rule, context masses, laws and TVs), with the
derivation of each analytic gap. New script
experiments/main_table/run_partitions.py re-runs every reported configuration
with its original seeds, captures the partition the estimator returns, and
writes results_partitions/partitions_per_seed.csv (recovered partition, true
partition, pair-level error, exact-match flag, per seed). It also checks that
the re-run class count equals the committed CSV on every row. Appendix G,
Table 3 summarizes it (generated from the CSV and checked by
scripts/check_paper_numbers.py). Result: the recovered partition equals the exact truth on every seed of every certified row (Tiger 20/20, 8-cycle 20/20, depth-2 20/20, position-flip 10/10, two endpoints 10/10 at each endpoint, nested L=2 20/20, stochastic Tiger 20/20, Dirichlet HMM 20/20; zero pair errors) and the partition is identical across seeds within each row. For the misspecification probe the reported partition is {0,1}|{2} on 10/10 seeds, i.e. exactly one wrong pair of three (the engineered sub-gap pair merged). The re-run class counts match the committed CSVs on every row that has a committed counterpart (the misspec summary has no committed class-count column). Re-running the full set took about 2 hours on a heavily loaded machine.
Concern 3 -- stochastic-policy Tiger derivation
Section 5, stochastic-policy Tiger check: state which actions produce recorded observations before termination, and derive the survival-conditioned gap p_listen(2a-1)^2 = 0.392 explicitly, reconciling it with conditioning on reaching r+k.
Changed (Section 5, "A nondegenerate stochastic policy"). The paragraph now
states the timeline (O_0 from reset; A_0 emits O_1, the context; A_1 emits O_2),
that a door opening ends the episode but its uniform terminal draw is recorded
as that step's observation, that the trajectory reaches r+k=2 iff A_0 is
listen (survival factor p_listen), and that A_1 is unrestricted. It derives
P(O_2=L | O_1=L)=p(a^2+(1-a)^2)+(1-p)/2=0.696 and P(O_2=L | O_1=R)=0.304, shows
the survival factor cancels in the conditioning (it multiplies numerator and
mass alike, mass 0.4 per context), and concludes TV = p_listen(2a-1)^2 = 0.392,
where the p_listen is that of the second action, not of surviving. We verified
this against the code three ways: the closed form, the exhaustive enumeration in
run_nondegenerate.py, and a 400,000-episode Monte Carlo through the real
collection and block-extraction code (check_tiger_stochastic.py, output in
results_nondegenerate/tiger_gap_check.json): reach fraction 0.8006,
conditional laws 0.698/0.305, TV 0.393.
Defect found and fixed. The Monte Carlo first disagreed with the closed
form (conditional laws 0.78/0.22, TV 0.56). Cause: collect_trajectories
seeded the policy generator with the same per-episode seed as the environment
reset, coupling action draws to Tiger's noise. This affected only policies that
use their generator, i.e. only this stochastic-Tiger check (the other rows use a
single action or a constant policy). We made the generators independent
(quotient_learning/collection.py), added a regression test
(tests/unit/test_policy_rng_independence.py), regenerated the
results_nondegenerate artifacts, and the result is unchanged in substance:
exact recovery on 20/20 seeds, zero abstention, 1,548-1,619 inference blocks
per rare context against m*=468. Two legacy tests that depended on the coupled
streams were corrected, and the paper says so in a parenthetical. Without
this fix the earlier artifact sampled a different process from the one the
enumerated ground truth describes.
Concern 4 -- data-driven epsilon check
Section 5, data-driven epsilon check: either provide a certification argument that does not rely on post-hoc access to the true minimum TV, or reframe the check explicitly as a diagnostic that carries no certificate.
Reframed as a diagnostic (Section 5, retitled "A data-driven eps: a
diagnostic, not a certificate"). We chose the honest reframe. The paragraph
now states that the check carries no certificate and is not evidence for
Theorem 2. It explains what does hold (eps-hat and the candidate set depend on
the selection half only, so conditionally on it the inference-half test is the
fixed-eps test of Theorem 1) and what does not (nothing bounds the probability
that eps-hat exceeds the true gap; a certificate would need a selection-half
lower confidence bound on the minimum gap, which we do not provide; the check
eps-hat <= 0.3872 uses the true minimum TV, which a user does not have). The
reported numbers (no fallback fired, eps-hat in [0.189, 0.199], 20/20 exact
recovery) are unchanged and presented as a single-instance observation.
Related Work: position the quotient against the bounded-indistinguishability quotient of arXiv 2603.21399, and sharpen the significance argument relative to Palmer & Goldberg (2007) and CSSR beyond conditioning differences.
Changed (Related work; references.bib). Added the Nixon paper (read from the
arXiv abstract and HTML text). We describe it as a model-based, exact
construction for known finite POMDPs: a probe family of finite-state
controllers, a closed-loop Wasserstein pseudometric on histories, canonical /
minimal / unique, exactly decision-sufficient for observation-action objectives
under clock-aware probes. As far as the text we read states, it gives no
finite-sample guarantee for learning the quotient from trajectories. We position
ours as complementary: single behavior policy (a one-element probe family),
total variation on a finite alphabet, indexed by episode position with
conditioning on reaching r+k, and a finite-sample certificate for estimation.
We state explicitly that we prove no refinement relation between the two and
inherit neither its canonicity nor its decision-sufficiency results.
The PDFA/CSSR paragraph now concedes that the statistic and proof techniques
are standard and that PDFA algorithms recover the whole automaton when the data
are PDFA-generated, and argues the delta as three concrete differences:
(i) realizability (a bounded context of a POMDP/HMM carries a mixture over
histories and is generally not a PDFA state, so those guarantees are silent on
such data; CSSR is asymptotic for stationary finite-causal-state processes);
(ii) closed loop with a stated policy, actions, and conditioning on reaching
r+k; (iii) context selection, sample splitting, an executable gate m* with
constants, and abstention inside the theorem, plus a two-point lower bound. We
say we have not verified that no PDFA result has some of these features, and
make no dominance claim. The remaining significance is the estimand plus the
executable certificate; we did not add a downstream planner.
Concern 6 -- Appendix E, routing by the longest context
Appendix E: state what continuation law is guaranteed for the class routed by the longest-context induced state map, given that routing restricts events defining P_h.
Changed (Appendix E, new Proposition 10 "Routed laws" with proof). The
reviewers are right that P_h conditions on the suffix event E_h, while routing
sends h only the histories in R_h (no longer ℋ-context matches). The
proposition gives the exact decomposition P_h = (λ_h Q_h + Σ_{g in G_h} λ_g Q_g)
/ (λ_h + Σ λ_g) for the routed laws Q. Consequences: (1) when ℋ is the full
eligible universe (the event of the sample-complexity corollary) every history
routes to a maximal-length context and the continuation law of the histories
routed to a class is exactly the class law P_c, and the map is total; (2) for a
strict subset ℋ, a class's routed law for a non-maximal member is a residual
law, equal to P_h only if its ℋ-extensions already carry P_h; the certificate
covers equalities P_h=P_g, not residual laws, and we claim nothing more. We note
that no experiment exercises case 2 (the nested check uses all 12 contexts in
every seed).
Other
Zhang et al. 2019 DOI. Replaced the unresolvable DOI by an arXiv eprint
(1906.10437) and url https://arxiv.org/abs/1906.10437 (entry type misc).
Verification. Test suite: see the final message. scripts/check_paper_numbers.py
now also checks the per-seed partition table against the CSV.
