Response to the editor and reviewers
We thank the editor and reviewers for identifying gaps in the validation and comparisons. We agree that the earlier Safety-Gymnasium non-violation count was not a clean test of the selection procedure, that the 32 cells shared baseline pools, and that the approximate cost-sufficiency assumption prevents a formal guarantee for those tasks. We revised the claims accordingly and added the requested analyses.
1. Frozen pilot rule on an unseen training seed
The original 24-cell split (PointGoal1 seeds 1 and 2 and CarGoal1 seed 0) is now explicitly labeled retrospective: those rollouts were already available while the window, onset and margin choices were developed. We report it only as a robustness check and report its independent truth pools, certificate pools, coverage counts and bound-to-cost ratio separately from seed 0 (Section 3, “Validation split”).
We additionally trained the baseline (λ=5) and candidate (λ=0.1) on a new PointGoal1 training seed (seed 3), then applied the frozen selection rule. Fifty baseline episodes per condition selected window 28, onset {3,4}, and (\eta=0.0563). Certificate and independent truth pools used 400 and 300 episodes per policy, respectively, with disjoint seeds; no episode terminated before the 1,000-step horizon. Baseline and candidate truth means were 33.76 and 70.96, and the candidate episode-bootstrap 95% interval was [49.47, 94.23]. The empirical-Bernstein upper bound on excess cost was 273.80, above both the candidate mean and interval. We report one held-out seed and candidate as a case study, not as a general coverage validation. The manuscript and release include the selected alphabet, threshold sensitivity, rollout costs and lengths, training checkpoints, and the analysis script.
2. Power to detect excess cost
We added a CartPole screening analysis where cost sufficiency holds exactly. Unsafe status is based on independent 10,000-episode reference pools; screening rates are estimated from 3,000 episode-level bootstrap resamples of separate rollout pools at 100, 500 and 2,000 episodes per policy. For excess-cost margins of 1, 5 and 10, estimated power was 1.00 at each sample size. This came with false-rejection rates among safe policies of 1.00 at n=100 and 500; at n=2,000 the rates were 1.00, 0.75 and 0.25 for margins 1, 5 and 10, respectively. We now state that the certificate has useful screening power in this constructed family but is too loose for many smaller margins; we do not present non-violation alone as evidence of practical power.
3. Controlled failure of cost sufficiency
We added a one-symbol, T=200 construction with identical occupancy under both policies, baseline cost zero and candidate cost one at every step. With 400 episodes per policy, empirical TV is zero and the finite-sample TV upper bound is 42.8, below the candidate's true cost 200. The hidden policy-dependent cost makes Assumption 1 false, so this is a counterexample to applying the certificate outside its assumptions, not a counterexample to the theorem. It demonstrates the undercoverage mechanism directly (Section 3, “Failure when cost sufficiency is violated”).
4. Variance-adaptive importance sampling
We added the Maurer–Pontil empirical-Bernstein upper bound for ordinary per-trajectory importance-weighted returns, using the known maximum return T times the maximum trajectory likelihood ratio and no weight truncation. At n=2,000 and 95% confidence, its upper bounds are 7.1e11, 3.9e9 and 7.9e4 for target noise levels 0.05, 0.1 and 0.2. All exceed the cost ceiling 200. We limit the conclusion to these policies and this horizon, and note that truncating weights does not itself give a valid upper bound. The table and reproducibility script are updated.
We now discuss Vincent et al.’s 2023 preprint and 2024 journal version on distribution-free post-design rollout bounds for expected cost, VaR, CVaR and sparse failure probability, including candidate-selection correction. Those bounds certify absolute performance from a policy’s own simulator cost rollouts; our relative occupancy certificate instead compares a candidate with a trusted baseline without candidate action probabilities and requires cost sufficiency. We describe the methods as complementary and acknowledge that direct rollout certification is simpler when its inputs and assumptions are available.
We also discuss RS-GAIL as risk-profile occupancy matching for training an imitation policy, distinct from a post-deployment upper bound on expected additive cost relative to a baseline. The desk-screen concern about SPoRt is addressed as well: it bounds temporal-property violation probability from a safe base policy and a maximum policy ratio, and uses projection during training, whereas our target is expected additive cost and a post hoc occupancy comparison.
We have narrowed the novelty claim to the finite-sample occupancy certificates under their stated assumptions and the empirical study. We continue to state that the TV/IPM inequality and the lower-bound reduction are elementary, that the TV lower bound is for T=1 and leaves a sqrt(K) gap, and that the JSD form is never tighter than TV. We added an explicit decision interpretation: the certificate can screen against an expected-cost budget only when its upper bound is below that budget; otherwise it is inconclusive. It cannot certify that a candidate is safer than the baseline or provide a per-episode safety guarantee.
6. Sensitivity to pilot size and onset threshold
We report episode-bootstrap fits at total pilot sizes 10, 25, 50 and 100, using a fresh CarGoal1 pilot with 50 episodes per condition. Across 20 bootstrap resamples per size, the observation window matched the 100-episode fit in 17/20, 20/20, 20/20 and 20/20 resamples, respectively. At size 50 and the 0.05 threshold, onset {2,3} was selected in 13/20 resamples, {3,4} in 5/20, and another onset in the remaining two. These fits quantify selection variability; the 50-episode evaluation pools yielded bounds above the task ceiling and are not a coverage test. On the fresh seed-3 pilot, the window matched in 19/20 fits at total pilot size 10 and in 20/20 fits at sizes 25, 50 and 100; median edge errors were 0.0197, 0.0143, 0.0078 and 0.0047. At size 50 the onset was {3,4} in 8/20 fits, {4,5} in 8/20, and another set in 4/20. The separately reported seed-3 threshold sensitivity was stable from 0.025 through 0.10, but changed at 0.20. This supports window stability in that sample while making the onset-selection variability explicit.
Holding the released alphabets and evaluation pools fixed, thresholds from 0.025 through 0.20 selected the same onset set on each of the three validation runs and produced unchanged bounds. A zero threshold selected all seven thresholds and raised median bounds (Section 3, “Sensitivity to the onset threshold”). We distinguish this retrospective sensitivity analysis from the newly collected seed-3 held-out case.
7. Assumption and validity language
The paper now states directly that approximate cost sufficiency on Safety-Gymnasium means those non-violations are empirical diagnostics, not formal coverage evidence. Cost sufficiency is not testable from baseline data alone in general; a structural justification, such as a known cost function of the discretised observation, is needed for the theorem. We retain the observed drift analysis as a diagnostic and do not present it as proving the assumption.