Public research and engineering by Satoshi Kawasaki, MIKOTO’s representative. Each entry separates local findings, validation limits and upstream responses.
Published input audit · scope guidance from PR author
Checked 24 stored backdoor inputs across 10 sampled APPS problems against their stated constraints. One unresolved specification/data conflict was reported. The package also records 202 synthetic boundary tests and 30 adapter tests; these counts are distinct from the 24 stored inputs.
The PR author replied that #899 is limited to validator infrastructure; coverage and format policy remain maintainer decisions. A separate corpus PR or follow-up on #755 was suggested if maintainers want it. This is scope guidance, not maintainer approval or adoption.
Scope & limits
An offline constraint audit, not a demonstrated error in the full safety score. No live model attacks, sandbox-isolation validation or end-to-end evaluation run were performed.
Published side-test probes · upstream response pending
Probed the swe_django tasks 15103 and 16661 using reconstructed base/gold function bodies checked against normalized upstream ASTs, monkeypatched into Django 4.2.16 and run in pytest subprocesses. In 15103, an or-combined escaping condition accepted both escaped and unescaped output. For 16661, the literal prompt mutation and an alternative mutation were tested separately.
Scope & limits
SWE-bench Docker, the full main-task suite and ControlArena’s side_task_correctness() were not run. The default safety aggregation requires both main-task and side-task success; a passing side test alone does not establish a false positive in the overall evaluation.
Published scorer probe · upstream response pending
Exercised the standalone Inspect scorer (eval/inspect/judge.py at commit 5f764cbf) in a standard-library AST-extracted harness across eight cases with stubbed judge responses. With a valid, cue-aligned answer, missing reasoning (has_reasoning=False) or invalid reasoning-judge JSON (r_parse_ok=False) returns VCR=0.0 and UAR=1.0. The no-trace VCR result conflicts with the integration README’s documented NaN value. The intended UAR and parse-failure conventions, and their eligible populations, have been raised upstream for clarification.
Scope & limits
A source-level probe with stubbed judge responses; Inspect’s full task orchestration, metric aggregation and live model API calls were not run. Because the conversion bridge (to_results.py) preserves parse flags and role_register.load_clean excludes affected rows via PARSE_OK, no effect on published paper results has been demonstrated.
Published methods note · upstream response pending
Checked the Phase 2 mirror reanalysis script and archived JSON at commit 34898fb in selvarath-debate. Separated the erratum’s +4.56 pp side-standardized estimate (which collapses same-side transcripts within questions and weights worlds equally) from the +3.94 pp realized-schedule estimate under strict scoring (which retains all scheduled slots and counts INVALID judgments as wrong), and derived conditional sensitivity bounds for unobserved opposite-polarity outcomes (a fixed-sample outer bound of [+2.06, +5.94] pp for support-feasible homogeneous κ; for heterogeneous |κ_u|≤K, the coarse outer envelope’s lower endpoint reaches zero at K=7.876 pp, while expanding the existing question-cluster CI by K/2 makes its lower endpoint reach zero at K=4.4715 pp).
Scope & limits
Conditional sensitivity calculations on archived Phase 2 outputs; no new debates, judging runs or bootstrap resamples were executed. These are outer bounds, not exact feasible ranges or newly estimated confidence intervals; the crossings do not establish when a negative effect becomes feasible. The CI expansion inherits the existing interval’s assumptions. The note separates the two estimands rather than claiming the reported +4.56 pp is an arithmetic error, and leaves the missing-slot weight question under valid_only separate.
Proposed an offline sensitivity example and unit tests for how proposal attribution and scoring windows affect a synthetic Timaeus prediction. It uses the existing scoring functions and separates missing records from explicit zero values.
Scope & limits
Production scoring and ground-truth data are unchanged. Synthetic scenario results do not establish which specification is correct, nor demonstrate improved model rankings, AI safety or real funding decisions.
Reported bug confirmed · upstream fix merged 16 October 2026
Reported a small mixed-integer model with a feasible objective of 1 and a matching exact upper bound, while the recorded HiGHS run reported Optimal 0. A standalone rational checker supports the report. A maintainer confirmed the issue on latest. The fix in PR #3408, authored by upstream contributor Opt-Mucca, was merged on 16 October 2026. Our contribution was the report, reproducer and follow-up verification.
Scope & limits
Numerical-software verification experience, not an AI safety result. The reproducer does not establish that every instance of this bug class has been fixed.
Proposed a fix and regression tests for history-shard keys after unwind in op-reth. An upstream maintainer independently reproduced the bug and reviewed the fix in a comment. The original root cause and reproduction were reported by omerfirmak.
Scope & limits
Local validation: just lint and scoped tests across 17 packages (822 passed, 3 skipped); no live-node RPC validation. The unwind path normalizes affected legacy tails but does not repair untouched malformed intermediate shards.
Our representative’s wider contributions include SGLang, Tellurium and Diffrax. Implementation, independent review and upstream adoption are recorded separately on GitHub.
SGLang #42850 ↗ — the PR author incorporated a proposed branch guard; local kernel measurements do not establish end-to-end performance.
AI tools assisted implementation, experiments, analysis and writing. The linked records distinguish local checks from upstream confirmation. Neither these studies nor the services constitute certification of a model’s overall safety.
