Home vs Frontier Coding Benchmarks

105 tasks

How to read results

Latest local is the most recent recorded attempt, not the best. Expand a task for its model, check results and earlier attempts.

Published comparisons show passing published attempts on that exact task, with model and effort named. They are not matched local configurations. Missing results mean unknown, not failure.

Graph: DeepSWE uses feature checks; Rails uses official checks. Fractions are not difficulty-normalized. Run order shows every matching attempt by its recorded start; unscored attempts appear separately. Time shows the latest attempt per task. Changed configurations and tasks do not form a controlled improvement curve.

DeepSWE and Rails retain their own grading rules. Results are a snapshot, not a live feed. Imported 2026-10-14T11:08:01.350688+00:00.

DeepSWE actionlint Go

Action pinning rules

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add a configurable lint rule that enforces pinned versions for action and reusable workflow references.

Flash-Next IQ3_SDeepSWE · primary · 2026-10-12

Agent time: 15m 55s
Feature checks: 55 / 55
Regression checks: 145 / 145

Run settings & provenance

Run ID: tonight-20261012/actionlint-action-pinning-lint__8JiBsSU
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 955.0015
Timing seconds · Environment setup (seconds): 25.700454
Timing seconds · Verifier: 26.950975
Usage · Input tokens: 7,889,324
Usage · Cached tokens: 7,825,301
Usage · Output tokens: 45,248
Usage · Peak context tokens: 108,867
Usage · Steps: 102
Usage · Requests: 103
Patch · Bytes: 52,001
Patch · SHA256: f6fb86b28d3fc159e8f9ed295e82783dbbfd2cf81efe1f6c27fedd07af008f77
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE fastapi Python

Deprecation response headers

Latest localFailed checks

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add runtime Deprecation, Sunset, and Link headers plus OpenAPI metadata and tracking middleware for deprecated routes.

Flash-Next IQ3_SDeepSWE · primary · 2026-10-12

Agent time: 11m 34s
Feature checks: 136 / 137
Regression checks: 3,134 / 3,134

Run settings & provenance

Run ID: tonight-20261012/fastapi-deprecation-response-hea__qUmcQMZ
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 693.513478
Timing seconds · Environment setup (seconds): 26.03664
Timing seconds · Verifier: 60.93147
Usage · Input tokens: 2,222,044
Usage · Cached tokens: 2,191,113
Usage · Output tokens: 32,615
Usage · Peak context tokens: 63,202
Usage · Steps: 50
Usage · Requests: 51
Patch · Bytes: 62,900
Patch · SHA256: 54f984444192b80135bf13a922da7c6a23aa482b1f1bd997fcb0b443ab74c10d
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE expr Go

Try/catch error recovery

Latest localFailed checks

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add expression and block-level error recovery with try, catch, finally, throw, retry, and errtype.

Flash-Next IQ3_SDeepSWE · primary · 2026-10-12

Agent time: 26m 40s
Feature checks: 73 / 79
Regression checks: 66,265 / 66,265

Run settings & provenance

Run ID: tonight-20261012/expr-try-catch-errors__GfLR6AT
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1599.637729
Timing seconds · Environment setup (seconds): 25.077473
Timing seconds · Verifier: 52.316678
Usage · Input tokens: 10,770,815
Usage · Cached tokens: 10,555,924
Usage · Output tokens: 69,440
Usage · Peak context tokens: 150,995
Usage · Steps: 106
Usage · Requests: 107
Patch · Bytes: 32,070
Patch · SHA256: 9c9fed859653f4605a84148603f747b18053bec95266bedf4aa633a8d1d85a90
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE drizzle Typescript

Typed window functions

Latest localFailed checks

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add typed query-builder helpers for SQL window functions, named windows, and frame specs.

Flash-Next IQ3_SDeepSWE · primary · 2026-10-12

Agent time: 15m 36s
Feature checks: 121 / 130
Regression checks: 566 / 566

Run settings & provenance

Run ID: tonight-20261012/drizzle-orm-window-function-buil__xrssWis
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 936.194765
Timing seconds · Environment setup (seconds): 42.774546
Timing seconds · Verifier: 24.028978
Usage · Input tokens: 4,071,502
Usage · Cached tokens: 4,031,337
Usage · Output tokens: 43,709
Usage · Peak context tokens: 83,613
Usage · Steps: 88
Usage · Requests: 89
Patch · Bytes: 42,441
Patch · SHA256: bde06a9f1513446f378232041031dfed4a0ddc314007fff781a4618060eb1204
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE geo Go

Shape index serialization

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add stable ShapeIndex Encode/Decode support so indices round-trip without rebuilding.

Flash-Next IQ3_SDeepSWE · primary · 2026-10-12

Agent time: 10m 04s
Feature checks: 24 / 24
Regression checks: 599 / 599

Run settings & provenance

Run ID: tonight-20261012/geo-shapeindex-serialization__K72pa49
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 604.371213
Timing seconds · Environment setup (seconds): 27.841622
Timing seconds · Verifier: 30.706297
Usage · Input tokens: 2,329,387
Usage · Cached tokens: 2,269,757
Usage · Output tokens: 30,405
Usage · Peak context tokens: 81,336
Usage · Steps: 42
Usage · Requests: 43
Patch · Bytes: 23,848
Patch · SHA256: 06099ae4234e57469d58d2096202af1bf5fe583434ea23e4aebeaa5ae9472c18
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE wasmi Rust

Trap core dumps

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Generate opt-in Wasm coredumps on traps and attach the bytes to errors.

Flash-Next IQ3_SDeepSWE · primary · 2026-10-12

Agent time: 41m 18s
Feature checks: 22 / 22
Regression checks: 58 / 58

Run settings & provenance

Run ID: tonight-20261012/wasmi-trap-coredumps__SkX9dMo
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 2477.977253
Timing seconds · Environment setup (seconds): 36.895853
Timing seconds · Verifier: 26.534545
Usage · Input tokens: 15,168,388
Usage · Cached tokens: 14,921,447
Usage · Output tokens: 119,100
Usage · Peak context tokens: 151,972
Usage · Steps: 170
Usage · Requests: 171
Patch · Bytes: 66,852
Patch · SHA256: d253f11edf1bcfe47ce6f332953fe83a57db9863004a9b2e777f62462a20dbac
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE yjs Javascript

Map conflict detection

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add strict, deterministic conflict detection for Y.Map key writes with collect and error policies.

Flash-Next IQ3_SDeepSWE · primary · 2026-10-12

Agent time: 14m 42s
Feature checks: 9 / 9
Regression checks: 231 / 231

Run settings & provenance

Run ID: tonight-20261012/yjs-map-conflict-detection__9oG3SR2
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 882.088848
Timing seconds · Environment setup (seconds): 33.403535
Timing seconds · Verifier: 25.034519
Usage · Input tokens: 5,708,118
Usage · Cached tokens: 5,636,479
Usage · Output tokens: 41,470
Usage · Peak context tokens: 112,705
Usage · Steps: 69
Usage · Requests: 70
Patch · Bytes: 37,459
Patch · SHA256: 43a486b3ee766060293274348b88b7539da633272e7d0a204627da62d35529ec
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE skrub Python

Duration encoding

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add a DurationEncoder, duration selector, and duration routing in TableVectorizer.

Flash-Next IQ3_SDeepSWE · primary · 2026-10-12

Agent time: 24m 35s
Feature checks: 130 / 130
Regression checks: 2,784 / 2,784

Run settings & provenance

Run ID: tonight-20261012/skrub-duration-encoding__xiVc3Cb
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1475.190797
Timing seconds · Environment setup (seconds): 47.952636
Timing seconds · Verifier: 279.391762
Usage · Input tokens: 9,942,671
Usage · Cached tokens: 9,863,074
Usage · Output tokens: 50,986
Usage · Peak context tokens: 129,831
Usage · Steps: 100
Usage · Requests: 101
Patch · Bytes: 44,206
Patch · SHA256: f09f297e0dbd8ade7a207f1d8a73ef6801235fe216bc8360b22753083ae68997
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE abs Go

Module cache controls

Latest localFailed checks

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Harden ABS module resolution and caching, expose cache introspection APIs, and make module flags work correctly in script mode.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 8m 32s
Feature checks: 13 / 20
Regression checks: 3 / 3

Run settings & provenance

Run ID: tonight-20261012-followon/abs-module-cache-flags__eLaT3ZE
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 511.82121
Timing seconds · Environment setup (seconds): 3.665557
Timing seconds · Verifier: 30.864019
Usage · Input tokens: 1,892,538
Usage · Cached tokens: 1,868,392
Usage · Output tokens: 22,798
Usage · Peak context tokens: 46,598
Usage · Steps: 63
Usage · Requests: 64
Patch · Bytes: 19,487
Patch · SHA256: b0e8090ad4178406cfad69a6c73fd01c4382c87c26f7301f96ae399d547dc645
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE adaptix Python

Add input key aliases to name mapping

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add load-only alias support to name mapping so fields can resolve from alternate input keys.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 27m 19s
Feature checks: 44 / 44
Regression checks: 2,738 / 2,738

Run settings & provenance

Run ID: tonight-20261012-followon/adaptix-name-mapping-aliases__zgYeMuM
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1639.263037
Timing seconds · Environment setup (seconds): 26.016893
Timing seconds · Verifier: 32.357342
Usage · Input tokens: 12,205,840
Usage · Cached tokens: 12,126,886
Usage · Output tokens: 79,318
Usage · Peak context tokens: 155,854
Usage · Steps: 128
Usage · Requests: 129
Patch · Bytes: 36,614
Patch · SHA256: 3102729c954eb89ba07e4969fc906de99ebe55684df2f2d39e9625b77fbf7e44
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE bandit Python

Add incremental cache controls to Bandit

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add incremental analysis caching with cache invalidation, import/export, pruning, and cache inspection CLI options.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 11m 49s
Feature checks: 88 / 88
Regression checks: 275 / 275

Run settings & provenance

Run ID: tonight-20261012-followon/bandit-incremental-cache-control__RyHWf2U
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 708.818779
Timing seconds · Environment setup (seconds): 25.289135
Timing seconds · Verifier: 49.41769
Usage · Input tokens: 3,472,125
Usage · Cached tokens: 3,430,746
Usage · Output tokens: 36,265
Usage · Peak context tokens: 77,258
Usage · Steps: 62
Usage · Requests: 63
Patch · Bytes: 50,359
Patch · SHA256: 53f19a837f4d76a5b2dc973a130e78e6a9201e808510df2ebff3f53c593d8f18
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE boa Rust

Add hierarchical evaluation cancellation to Boa

Latest localFailed checks

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add cancellable evaluation handles that propagate through nested script, module, and job execution.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 33m 30s
Feature checks: 0 / 17
Regression checks: 7 / 7

Run settings & provenance

Run ID: tonight-20261012-followon/boa-hierarchical-evaluation-canc__yCbepPx
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 2009.5824
Timing seconds · Environment setup (seconds): 64.02786
Timing seconds · Verifier: 80.237243
Usage · Input tokens: 10,469,212
Usage · Cached tokens: 10,265,305
Usage · Output tokens: 95,675
Usage · Peak context tokens: 156,825
Usage · Steps: 128
Usage · Requests: 129
Patch · Bytes: 54,560
Patch · SHA256: 7ffe34ccd85940b3c5e2d08feea5d0d6accb83d9d865308ddf56629b39924ad2
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE cliffy Typescript

Add config file parsing to Cliffy commands

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add command-level config file loading, parsing, merging, and precedence handling.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 12m 31s
Feature checks: 37 / 37
Regression checks: 451 / 451

Run settings & provenance

Run ID: tonight-20261012-followon/cliffy-config-file-parsing__pGRckyX
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 751.083125
Timing seconds · Environment setup (seconds): 26.391629
Timing seconds · Verifier: 22.103584
Usage · Input tokens: 4,385,643
Usage · Cached tokens: 4,319,903
Usage · Output tokens: 38,157
Usage · Peak context tokens: 103,588
Usage · Steps: 57
Usage · Requests: 58
Patch · Bytes: 33,738
Patch · SHA256: 0734e220d0285e4fbcf72b9c7a31769bfa73c1049a14f27c4a94ffcecf978d20
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE dasel Go

Add HTML document format handling to Dasel

Latest localFailed checks

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add read and write support for HTML documents with Dasel's format handling.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 16m 08s
Feature checks: 144 / 146
Regression checks: 1,012 / 1,012

Run settings & provenance

Run ID: tonight-20261012-followon/dasel-html-document-format__zHnrBQG
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 968.449202
Timing seconds · Environment setup (seconds): 25.646378
Timing seconds · Verifier: 36.850513
Usage · Input tokens: 3,855,429
Usage · Cached tokens: 3,816,488
Usage · Output tokens: 50,558
Usage · Peak context tokens: 83,376
Usage · Steps: 69
Usage · Requests: 70
Patch · Bytes: 45,138
Patch · SHA256: 054498882e7044f3070732a74f084be1001c0b7fd37479eedf786d2d14fab63c
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE dynamodb Typescript

Add conditional required attributes to schemas

Latest localFailed checks

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add `requiredIf`-based conditional attribute enforcement across schema validation, parsing, updates, and JSON Schema export.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 33m 21s
Feature checks: 29 / 31
Regression checks: 1,267 / 1,267

Run settings & provenance

Run ID: tonight-20261012-followon/dynamodb-toolbox-conditional-att__iuC7yiG
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 2001.096069
Timing seconds · Environment setup (seconds): 29.343243
Timing seconds · Verifier: 42.650691
Usage · Input tokens: 9,721,731
Usage · Cached tokens: 9,635,524
Usage · Output tokens: 64,159
Usage · Peak context tokens: 140,224
Usage · Steps: 100
Usage · Requests: 155
Patch · Bytes: 70,150
Patch · SHA256: 9a36dbfa14938a6258438c4d702af063afadc8910e07a1ccd411ba4664b6beb8
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE fastapi Python

Add implicit HEAD and automatic OPTIONS responses to FastAPI routes

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add configurable implicit HEAD handling and automatic OPTIONS responses for FastAPI routes, routers, and included routers.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 23m 06s
Feature checks: 43 / 43
Regression checks: 3,134 / 3,134

Run settings & provenance

Run ID: tonight-20261012-followon/fastapi-implicit-head-options__bpZacWE
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1385.555989
Timing seconds · Environment setup (seconds): 23.871943
Timing seconds · Verifier: 60.917291
Usage · Input tokens: 8,925,310
Usage · Cached tokens: 8,862,057
Usage · Output tokens: 66,984
Usage · Peak context tokens: 124,693
Usage · Steps: 111
Usage · Requests: 112
Patch · Bytes: 76,868
Patch · SHA256: 609ce41c581e48b2a002632fcf9394de0e3c2dd33c638ea87d9564f8a7a96af3
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE go Go

Expose accumulated streamed function-call args in SDK surfaces

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Expose fully accumulated streamed function-call arguments across streaming, live sessions, and chat history.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 17m 02s
Feature checks: 6 / 6
Regression checks: 62 / 62

Run settings & provenance

Run ID: tonight-20261012-followon/go-genai-streamed-function-args__9GjSjEG
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1022.424934
Timing seconds · Environment setup (seconds): 24.633935
Timing seconds · Verifier: 38.194232
Usage · Input tokens: 5,318,826
Usage · Cached tokens: 5,262,030
Usage · Output tokens: 50,785
Usage · Peak context tokens: 97,811
Usage · Steps: 92
Usage · Requests: 93
Patch · Bytes: 41,684
Patch · SHA256: 7d72e94aaa631fadb38b0c270a2e3449f6dd38b0e0f131c99e09157edfbe53ff
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE go Go

Add worktree merge conflict handling

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add worktree merge support with conflict detection, merge head handling, and index stage updates.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 15m 59s
Feature checks: 17 / 17
Regression checks: 2 / 2

Run settings & provenance

Run ID: tonight-20261012-followon/go-git-worktree-merge-conflicts__6DfcCJY
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 958.982545
Timing seconds · Environment setup (seconds): 29.301627
Timing seconds · Verifier: 38.838269
Usage · Input tokens: 3,175,660
Usage · Cached tokens: 3,111,087
Usage · Output tokens: 49,403
Usage · Peak context tokens: 101,860
Usage · Steps: 48
Usage · Requests: 49
Patch · Bytes: 39,034
Patch · SHA256: 8ba982f0d1cdb57134a0e6d9df9a6d1a0a31e4b01b7cae6466a41817da47516c
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE goreleaser Go

Add retry-aware publishing audit logs

Latest localFailed checks

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Implement per-artifact retries for uploads, artifactory, and blobs while recording deterministic publish attempt history.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 21m 37s
Feature checks: 28 / 29
Regression checks: 29 / 29

Run settings & provenance

Run ID: tonight-20261012-followon/goreleaser-retry-publish-auditin__HiH3HZt
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1296.807556
Timing seconds · Environment setup (seconds): 104.297679
Timing seconds · Verifier: 70.727088
Usage · Input tokens: 5,935,433
Usage · Cached tokens: 5,894,142
Usage · Output tokens: 50,185
Usage · Peak context tokens: 90,616
Usage · Steps: 107
Usage · Requests: 108
Patch · Bytes: 47,538
Patch · SHA256: 0fb2757d1bddb8ac1fd3eece73cd731335407eb0608876643f89b6dfb44d2374
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE helm Go

Add unified manifest stream output across Helm commands

Latest localFailed checks

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add a stable unified manifest stream for template, dry-run install/upgrade, and get manifest output.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 36m 44s
Feature checks: 0 / 5
Regression checks: 2 / 2

Run settings & provenance

Run ID: tonight-20261012-followon/helm-unified-manifest-stream__K5gXqBY
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 2203.952954
Timing seconds · Environment setup (seconds): 3.461494
Timing seconds · Verifier: 103.146148
Usage · Input tokens: 13,023,281
Usage · Cached tokens: 12,793,433
Usage · Output tokens: 82,139
Usage · Peak context tokens: 156,838
Usage · Steps: 154
Usage · Requests: 155
Patch · Bytes: 0
Patch · SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE httpx Python

Add multipart response parsing to HTTPX

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add Response iterators that parse multipart HTTP response bodies into parts.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 11m 45s
Feature checks: 122 / 122
Regression checks: 1,272 / 1,272

Run settings & provenance

Run ID: tonight-20261012-followon/httpx-multipart-response-parsing__7K8NvzG
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 705.176334
Timing seconds · Environment setup (seconds): 3.5341
Timing seconds · Verifier: 22.755236
Usage · Input tokens: 2,314,026
Usage · Cached tokens: 2,286,306
Usage · Output tokens: 35,259
Usage · Peak context tokens: 62,440
Usage · Steps: 53
Usage · Requests: 54
Patch · Bytes: 25,954
Patch · SHA256: 1242d7fe549b4bd28bdcdb56d9bbb9962d814455f30b715f550239f6d9aaf379
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE ipython Python

Add session bundle recording and replay to IPython

Latest localFailed checks

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add `.ipybundle` recording, validation, and replay APIs for IPython sessions.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 11m 33s
Feature checks: 14 / 17
Regression checks: 29 / 29

Run settings & provenance

Run ID: tonight-20261012-followon/ipython-session-bundle-replay__3pWUJCQ
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 692.884049
Timing seconds · Environment setup (seconds): 25.091993
Timing seconds · Verifier: 18.393327
Usage · Input tokens: 1,902,257
Usage · Cached tokens: 1,878,029
Usage · Output tokens: 31,843
Usage · Peak context tokens: 54,809
Usage · Steps: 58
Usage · Requests: 59
Patch · Bytes: 43,116
Patch · SHA256: 7e827ffc5157c31e05deafafeccfa138306312d50255d27514b9f7dbb5bbe9fa
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE langchain Python

Add request coalescing to `Runnable`

Latest localFailed checks

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add `Runnable.with_coalesce()` so concurrent identical inputs share one execution across sync, async, streaming, and batch APIs.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 30m 11s
Feature checks: 48 / 50
Regression checks: 232 / 232

Run settings & provenance

Run ID: tonight-20261012-followon/langchain-request-coalescing__9VF4pyY
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1811.456386
Timing seconds · Environment setup (seconds): 39.283825
Timing seconds · Verifier: 42.620816
Usage · Input tokens: 8,541,308
Usage · Cached tokens: 8,462,115
Usage · Output tokens: 88,554
Usage · Peak context tokens: 152,623
Usage · Steps: 86
Usage · Requests: 87
Patch · Bytes: 65,957
Patch · SHA256: 163719b575919e86421221e347110045eca5ba083ad4a75bcaf5bd512674e0f3
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE mnamer Python

Validate daemon watch, status, and log lifecycle

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add daemon watch validation, state tracking, logging, and lifecycle commands with non-blocking run-once processing.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 14m 02s
Feature checks: 51 / 51
Regression checks: 319 / 319

Run settings & provenance

Run ID: tonight-20261012-followon/mnamer-daemon-watch-lifecycle__hjr3o3p
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 842.466585
Timing seconds · Environment setup (seconds): 24.108183
Timing seconds · Verifier: 36.395475
Usage · Input tokens: 2,539,459
Usage · Cached tokens: 2,489,345
Usage · Output tokens: 39,188
Usage · Peak context tokens: 78,953
Usage · Steps: 43
Usage · Requests: 44
Patch · Bytes: 26,441
Patch · SHA256: c27e34c9f9b0b799056c7059ea227881456292232826185edf2c64a3dfd45628
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE mobly Python

Add grouped test phases with synchronized barriers

Latest localFailed checks

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add grouped execution phases with explicit synchronization barriers and per-group setup and teardown.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 19m 03s
Feature checks: 78 / 79
Regression checks: 808 / 808

Run settings & provenance

Run ID: tonight-20261012-followon/mobly-grouped-test-barriers__YfFtLYP
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1142.854936
Timing seconds · Environment setup (seconds): 23.792344
Timing seconds · Verifier: 19.916711
Usage · Input tokens: 5,525,553
Usage · Cached tokens: 5,483,608
Usage · Output tokens: 62,903
Usage · Peak context tokens: 104,517
Usage · Steps: 78
Usage · Requests: 79
Patch · Bytes: 53,296
Patch · SHA256: edd622cf2b4cf3c3bf8d1d777d580f8a401568ed663172bfceb7e5f6e5f8188d
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE narwhals Python

Add rolling min, max, median, and quantile methods

Latest localPass

gpt-6-astra · xhigh public4/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add the remaining rolling window methods to Expr and Series with consistent validation and backend delegation.

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time: 38m 01s
Feature checks: 103 / 103
Regression checks: 10,093 / 10,093

Run settings & provenance

Run ID: tonight-20261012-followon/narwhals-rolling-window-suite__LyjbpEx
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Runtime: Strata 0.1.42
Agent timeout seconds: 2,700

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 2281.09125
Timing seconds · Environment setup (seconds): 30.159255
Timing seconds · Verifier: 82.595128
Usage · Input tokens: 14,409,783
Usage · Cached tokens: 14,179,719
Usage · Output tokens: 92,623
Usage · Peak context tokens: 157,740
Usage · Steps: 156
Usage · Requests: 157
Patch · Bytes: 73,788
Patch · SHA256: 53dc53defe7601d42f98fae8b14a4e31d58d2a66dd59f17866a0652a71f53c7e
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

claude-opus-5 · max4/4 passed

Full record

Rails Action Controller Ruby

Search throttling

Latest localPass

Claude Opus 5 · high public3/3 runs passed

Claude Fable 5.1 · high public3/3 runs passed

Rate-limit public book searches per address, using a counter that expires automatically.

Flash-Next IQ3_SRails Lemans · pilot · 2026-10-12

Agent time: 9m 20s
Official checks: 5 / 5

Recovered grading: the harness was repaired and the unchanged patch regraded. One model attempt; no solution retry.

Run settings & provenance

Run ID: rails-pilot-20261012/ac-throttle-search
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 560.422
Timing seconds · Wall clock (seconds): 586.9
Usage · Input tokens: 1,300,103
Usage · Cached tokens: 1,268,143
Usage · Output tokens: 21,243
Usage · Steps: 43
Patch · Bytes: 3,841
Patch · SHA256: 4b4d9732e2d7fa897c6e5b53f654a44a7a4a69d4b8011c48d807f00598abe4a8
Verification · Status: recovered-grading
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks
Verification · Regrade · Unchanged patch: True
Verification · Regrade · Original outcome: infrastructure-invalid
Verification · Regrade · Regrade evidence: official verifier checks rerun on the unchanged submitted patch

Published comparison sources

Rails-published successful attempts out of three runs per task, not individual checks. Provider models and efforts vary by benchmark; these are not matched local configurations.

Claude Opus 5 · high3/3 passed

Claude Fable 5.1 · high3/3 passed

Kimi K3 · max3/3 passed

Claude Fable 5 · high3/3 passed

GPT-5.6 Sol · medium3/3 passed

Grok 4.6 · high3/3 passed

GLM 5.3 · max3/3 passed

Claude Opus 4.8 · high3/3 passed

GPT-5.6 Terra · medium3/3 passed

Qwen3.8-27B · xhigh3/3 passed

Muse Spark 1.2 · medium3/3 passed

GPT-5.6 Luna · medium3/3 passed

Gemini 3.7 Flash · medium3/3 passed

Claude Sonnet 5 · high3/3 passed

GLM 5.2 · high3/3 passed

DeepSeek V4 Flash · high3/3 passed

GLM 5.3 Flash · max3/3 passed

Full record

Rails Active Record Ruby

Atomic imports

Latest localPass

Claude Opus 5 · high public3/3 runs passed

Claude Fable 5.1 · high public3/3 runs passed

Make manifest imports all-or-nothing, including when a guard returns early.

Flash-Next IQ3_SRails Lemans · pilot · 2026-10-12

Agent time: 3m 20s
Official checks: 10 / 10

Run settings & provenance

Run ID: rails-pilot-20261012/ar-atomic-import
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 200.228
Timing seconds · Wall clock (seconds): 225.9
Usage · Input tokens: 130,386
Usage · Cached tokens: 123,450
Usage · Output tokens: 7,205
Usage · Steps: 18
Patch · Bytes: 6,090
Patch · SHA256: 3e07084de5db125819415470337b4ad941d141ff43e513d42aadb0c751674674
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Published comparison sources

Rails-published successful attempts out of three runs per task, not individual checks. Provider models and efforts vary by benchmark; these are not matched local configurations.

Claude Opus 5 · high3/3 passed

Claude Fable 5.1 · high3/3 passed

Kimi K3 · max3/3 passed

Claude Fable 5 · high3/3 passed

GPT-5.6 Sol · medium3/3 passed

Grok 4.6 · high3/3 passed

GLM 5.3 · max3/3 passed

Claude Opus 4.8 · high3/3 passed

GPT-5.6 Terra · medium3/3 passed

Qwen3.8-27B · xhigh3/3 passed

Muse Spark 1.2 · medium3/3 passed

GPT-5.6 Luna · medium3/3 passed

Gemini 3.7 Flash · medium3/3 passed

Claude Sonnet 5 · high3/3 passed

GLM 5.2 · high3/3 passed

DeepSeek V4 Flash · high3/3 passed

GLM 5.3 Flash · max3/3 passed

Full record

Rails Fizzy Ruby

Kamal deployment

Latest localFailed checks

GPT-6 Astra · max public3/3 runs passed

Claude Opus 5 · max public3/3 runs passed

Provide a single-server Kamal deployment with persistent data and external settings and secrets.

Flash-Next IQ3_SRails Lemans · pilot · 2026-10-12

Agent time: 11m 38s
Official checks: 3 / 8

Run settings & provenance

Run ID: rails-pilot-20261012/ft-kamal-deploy
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 698.126
Timing seconds · Wall clock (seconds): 745.0
Usage · Input tokens: 2,490,194
Usage · Cached tokens: 2,450,996
Usage · Output tokens: 31,044
Usage · Steps: 62
Patch · Bytes: 10,961
Patch · SHA256: fe35958bcee6377622e8fd28fc683d7a1017dbbb17e0e2dfa70fa3e80c7b9912
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Published comparison sources

Rails-published successful attempts out of three runs per task, not individual checks. Provider models and efforts vary by benchmark; these are not matched local configurations.

GPT-6 Astra · max3/3 passed

Claude Opus 5 · max3/3 passed

Claude Opus 5 · high3/3 passed

Claude Fable 5.1 · high3/3 passed

GPT-6 Astra · medium3/3 passed

GPT-5.6 Sol · medium2/3 passed

GPT-5.6 Luna · medium0/3 passed

Gemini 3.8 Flash · medium2/3 passed

Grok 4.6 · high2/3 passed

Kimi K3 · max1/3 passed

Muse Spark 1.3 · medium2/3 passed

GLM 5.3 Flash · max1/3 passed

DeepSeek 4.1 Flash · high1/3 passed

Claude Fable 5.1 · max3/3 passed

GPT-5.6 Sol · max2/3 passed

GPT-5.6 Luna · max2/3 passed

Gemini 3.8 Flash · high3/3 passed

Muse Spark 1.3 · max3/3 passed

Grok 4.6 · xhigh3/3 passed

DeepSeek 4.1 Flash · max1/3 passed

Claude Opus 5.5 · medium3/3 passed

Grok 4.7 · high2/3 passed

GPT-6 Sol · medium3/3 passed

GPT-6 Luna · medium1/3 passed

GPT-6 Luna · max3/3 passed

Full record

Rails Active Job Ruby

Jobs after commit

Latest localPass

Claude Opus 5 · high public3/3 runs passed

Claude Fable 5.1 · high public3/3 runs passed

Enqueue follow-up jobs only after commit, while preserving operator notifications for abandoned batches.

Flash-Next IQ3_SRails Lemans · overnight · 2026-10-12

Agent time: 12m 18s
Official checks: 8 / 8

Run settings & provenance

Run ID: tonight-20261012/rails-runs/aj-enqueue-after-commit
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 737.521
Timing seconds · Wall clock (seconds): 766.2
Usage · Input tokens: 1,677,007
Usage · Cached tokens: 1,646,834
Usage · Output tokens: 28,487
Usage · Steps: 50
Patch · Bytes: 6,227
Patch · SHA256: 2157fc56f394fb7f81e3b193bbf9284d397bb18d64930152a5f4d93ddea0c63e
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Published comparison sources

Rails-published successful attempts out of three runs per task, not individual checks. Provider models and efforts vary by benchmark; these are not matched local configurations.

Claude Opus 5 · high3/3 passed

Claude Fable 5.1 · high3/3 passed

Kimi K3 · max3/3 passed

Claude Fable 5 · high3/3 passed

GPT-5.6 Sol · medium3/3 passed

Grok 4.6 · high3/3 passed

GLM 5.3 · max2/3 passed

Claude Opus 4.8 · high3/3 passed

GPT-5.6 Terra · medium3/3 passed

Qwen3.8-27B · xhigh3/3 passed

Muse Spark 1.2 · medium3/3 passed

GPT-5.6 Luna · medium3/3 passed

Gemini 3.7 Flash · medium3/3 passed

Claude Sonnet 5 · high3/3 passed

GLM 5.2 · high0/3 passed

DeepSeek V4 Flash · high1/3 passed

GLM 5.3 Flash · max3/3 passed

Full record

Rails Active Record Ruby

Tenant isolation

Latest localPass

Claude Opus 5 · high public2/3 runs passed

Claude Fable 5.1 · high public3/3 runs passed

Add tenants, migrate existing users and books, and prevent cross-tenant reads across every route.

Flash-Next IQ3_SRails Lemans · overnight · 2026-10-12

Agent time: 12m 00s
Official checks: 10 / 10

Run settings & provenance

Run ID: tonight-20261012/rails-runs/ar-tenant-isolation
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 720.004
Timing seconds · Wall clock (seconds): 751.5
Usage · Input tokens: 1,476,840
Usage · Cached tokens: 1,446,467
Usage · Output tokens: 28,849
Usage · Steps: 44
Patch · Bytes: 12,265
Patch · SHA256: fba8e149be580fa674c9f2b5d7892368d6a883638cf22d75610e14b64b04d2bc
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Published comparison sources

Rails-published successful attempts out of three runs per task, not individual checks. Provider models and efforts vary by benchmark; these are not matched local configurations.

Claude Opus 5 · high2/3 passed

Claude Fable 5.1 · high3/3 passed

Kimi K3 · max3/3 passed

Claude Fable 5 · high3/3 passed

GPT-5.6 Sol · medium3/3 passed

Grok 4.6 · high3/3 passed

GLM 5.3 · max3/3 passed

Claude Opus 4.8 · high3/3 passed

GPT-5.6 Terra · medium3/3 passed

Qwen3.8-27B · xhigh3/3 passed

Muse Spark 1.2 · medium2/3 passed

GPT-5.6 Luna · medium3/3 passed

Gemini 3.7 Flash · medium2/3 passed

Claude Sonnet 5 · high2/3 passed

GLM 5.2 · high2/3 passed

DeepSeek V4 Flash · high2/3 passed

GLM 5.3 Flash · max3/3 passed

Full record

Rails Fizzy Ruby

Card reactions

Latest localFailed checks

GPT-6 Astra · max public3/3 runs passed

Claude Opus 5 · max public3/3 runs passed

Add reactions to cards and display non-zero reaction counts in board previews.

Flash-Next IQ3_SRails Lemans · overnight · 2026-10-12

Agent time: 27m 48s
Official checks: 6 / 7

Run settings & provenance

Run ID: tonight-20261012/rails-runs/ft-card-reactions
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1668.382
Timing seconds · Wall clock (seconds): 1711.3
Usage · Input tokens: 5,372,831
Usage · Cached tokens: 5,329,245
Usage · Output tokens: 53,365
Usage · Steps: 106
Patch · Bytes: 23,662
Patch · SHA256: d87d997e32a4d1f3ea3c2c09fc5364bd308d944ac388edcdd47a19f37fd853a7
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Published comparison sources

Rails-published successful attempts out of three runs per task, not individual checks. Provider models and efforts vary by benchmark; these are not matched local configurations.

GPT-6 Astra · max3/3 passed

Claude Opus 5 · max3/3 passed

Claude Opus 5 · high3/3 passed

Claude Fable 5.1 · high2/3 passed

GPT-6 Astra · medium3/3 passed

GPT-5.6 Sol · medium3/3 passed

GPT-5.6 Luna · medium0/3 passed

Gemini 3.8 Flash · medium3/3 passed

Grok 4.6 · high2/3 passed

Kimi K3 · max2/3 passed

Muse Spark 1.3 · medium2/3 passed

GLM 5.3 Flash · max1/3 passed

DeepSeek 4.1 Flash · high0/3 passed

Claude Fable 5.1 · max2/3 passed

GPT-5.6 Sol · max3/3 passed

GPT-5.6 Luna · max2/3 passed

Gemini 3.8 Flash · high3/3 passed

Muse Spark 1.3 · max0/3 passed

Grok 4.6 · xhigh1/3 passed

DeepSeek 4.1 Flash · max1/3 passed

Claude Opus 5.5 · medium2/3 passed

Grok 4.7 · high1/3 passed

GPT-6 Sol · medium3/3 passed

GPT-6 Luna · medium0/3 passed

GPT-6 Luna · max1/3 passed

Full record

Rails ac Ruby

Open-redirect on sign-in return: reject hostile hosts, keep legitimate deep links.

Latest localFailed checks

Claude Opus 5 · high public3/3 runs passed

GPT-5.6 Luna · medium public3/3 runs passed

Shared and emailed links carry the page they point at as `?return_to=`, and `SessionsController#create` sends the reader there once they sign in.

Flash-Next IQ3_SRails Lemans · tonight-20261013

Agent time: 4m 26s
Official checks: 3 / 4

Run settings & provenance

Run ID: rails-lemans-tonight-20261013/ac-deep-link-return
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 265.572
Timing seconds · Wall clock (seconds): 301.1
Usage · Input tokens: 231,190
Usage · Cached tokens: 221,162
Usage · Output tokens: 10,601
Usage · Steps: 22
Patch · Bytes: 3,609
Patch · SHA256: 64b1b7546c91cbef785e843e5dcb9bca66633ef82007f2b8f6fbb40e79b23497
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Published comparison sources

Rails-published successful attempts out of three runs per task, not individual checks. Provider models and efforts vary by benchmark; these are not matched local configurations.

Qwen3.8-27B · xhigh2/3 passed

Claude Opus 5 · high3/3 passed

GPT-5.6 Luna · medium3/3 passed

Grok 4.6 · high1/3 passed

Full record

Rails hw Ruby

A Turbo broadcast still reaches collaborators whose access was revoked.

Latest localFailed checks

Claude Opus 5 · high public3/3 runs passed

GPT-5.6 Luna · medium public2/3 runs passed

Writebook shows a live "being edited" indicator in a page's editor: while one person saves, everyone else with the page open sees who is editing. But a collaborator removed from the book keeps receiving those pushes: an editor left open while they still had access keeps showing who is editing — presence payloads, editor names included, for a page they can no longer edit. The payload must not go out on a channel a removed collaborator can still be listening on.

Flash-Next IQ3_SRails Lemans · tonight-20261013

Agent time: 6m 50s
Official checks: 7 / 8

Run settings & provenance

Run ID: rails-lemans-tonight-20261013/hw-scoped-broadcast
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 409.777
Timing seconds · Wall clock (seconds): 448.5
Usage · Input tokens: 571,670
Usage · Cached tokens: 553,184
Usage · Output tokens: 17,173
Usage · Steps: 30
Patch · Bytes: 4,170
Patch · SHA256: 79d4741ce4dedeb376f55c11f30951eceec1af10673778dc25120d071054b7a7
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Published comparison sources

Rails-published successful attempts out of three runs per task, not individual checks. Provider models and efforts vary by benchmark; these are not matched local configurations.

Qwen3.8-27B · xhigh1/3 passed

Claude Opus 5 · high3/3 passed

GPT-5.6 Luna · medium2/3 passed

GLM 5.3 Flash · max3/3 passed

Full record

Rails ft Ruby

Expose the existing HTML activity timeline as an API endpoint.

Latest localFailed checks

GPT-6 Astra · max public0/3 runs passed

Claude Opus 5 · max public2/3 runs passed

Cards, comments, boards, and other resources are available through the API, but account activity is only available in the HTML timeline. Add `GET /:account_slug/activities.json` so API clients can consume account activity directly.

Flash-Next IQ3_SRails Lemans · tonight-20261013

Agent time: 11m 01s
Official checks: 13 / 14

Run settings & provenance

Run ID: rails-lemans-tonight-20261013/ft-activity-feed-api
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 661.428
Timing seconds · Wall clock (seconds): 707.7
Usage · Input tokens: 2,072,301
Usage · Cached tokens: 2,030,766
Usage · Output tokens: 24,691
Usage · Steps: 54
Patch · Bytes: 8,912
Patch · SHA256: b483fa85c73d76cab6de9fc4730f5e76ec0a84b310e941cbf5d0f67990b15dd5
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Published comparison sources

Rails-published successful attempts out of three runs per task, not individual checks. Provider models and efforts vary by benchmark; these are not matched local configurations.

GPT-6 Astra · max0/3 passed

GPT-6 Astra · medium0/3 passed

Claude Opus 5 · max2/3 passed

GPT-5.6 Luna · max0/3 passed

Grok 4.7 · high0/3 passed

Full record

Rails ft Ruby

Make board webhooks deliver, once per event, switched off when dead.

Latest localFailed checks

GPT-6 Astra · max public0/3 runs passed

Claude Opus 5 · max public0/3 runs passed

Add webhooks for boards, on top of the existing webhook settings. When an event happens, the receiver gets a POST about it and the log shows how it went. Deliver exactly once.

Flash-Next IQ3_SRails Lemans · tonight-20261013

Agent time: 21m 02s
Official checks: 6 / 9

Run settings & provenance

Run ID: rails-lemans-tonight-20261013/ft-webhook-delivery-trust-and-restart
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1261.533
Timing seconds · Wall clock (seconds): 1307.2
Usage · Input tokens: 4,243,255
Usage · Cached tokens: 4,184,557
Usage · Output tokens: 53,891
Usage · Steps: 70
Patch · Bytes: 21,862
Patch · SHA256: 92154fb1e61256d6f7e94475c139805c308c00ad14a4220e2c573d73eb9192d5
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Published comparison sources

Rails-published successful attempts out of three runs per task, not individual checks. Provider models and efforts vary by benchmark; these are not matched local configurations.

GPT-6 Astra · max0/3 passed

GPT-6 Astra · medium0/3 passed

Claude Opus 5 · max0/3 passed

GPT-5.6 Luna · max0/3 passed

Grok 4.7 · high0/3 passed

Full record

Rails ft Ruby

Publish a board to an unguessable public URL that unpublishing revokes at once.

Latest localFailed checks

GPT-6 Astra · max public1/3 runs passed

Claude Opus 5 · max public0/3 runs passed

We want to make it possible to publish boards, so anyone with a secret public URL (`/public/boards/:key`) can see them. Only admins can publish/unpublish boards (`POST`/`DELETE` on `/boards/:id/publication`).

Flash-Next IQ3_SRails Lemans · tonight-20261013

Agent time: 16m 37s
Official checks: 10 / 12

Run settings & provenance

Run ID: rails-lemans-tonight-20261013/ft-board-publish-unpublish-public-boundary
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 996.843
Timing seconds · Wall clock (seconds): 1046.0
Usage · Input tokens: 4,275,027
Usage · Cached tokens: 4,210,304
Usage · Output tokens: 41,116
Usage · Steps: 69
Patch · Bytes: 20,010
Patch · SHA256: 1b3ab2c3e26ff5efb2ca850f07863ab839be57eb10bd41a443c25f4a3f498af8
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Published comparison sources

Rails-published successful attempts out of three runs per task, not individual checks. Provider models and efforts vary by benchmark; these are not matched local configurations.

GPT-6 Astra · max1/3 passed

GPT-6 Astra · medium0/3 passed

Claude Opus 5 · max0/3 passed

GPT-5.6 Luna · max0/3 passed

Grok 4.7 · high0/3 passed

Full record

DeepSWE pest Rust

Coalesce qualifying choices into character classes

Latest localFailed checks

gpt-6-astra · xhigh public0/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add optimizer passes that collapse qualifying choice chains into merged character and negated character classes.

Flash-Next IQ3_SDeepSWE · tonight-20261013

Agent time: 12m 20s
Feature checks: 99 / 104
Regression checks: 249 / 250

Run settings & provenance

Run ID: tonight-20261013/pest-character-class-coalescing__5mURYHS
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Agent version: 1.18.31
Context tokens: 196,608
Agent timeout seconds: 2700.0
Compaction reserved tokens: 40,000

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 37.471339
Timing seconds · Agent setup: 0.277941
Timing seconds · Agent execution (seconds): 739.710135
Timing seconds · Verifier: 25.345051
Usage · Input tokens: 3,201,389
Usage · Cached tokens: 3,161,060
Usage · Output tokens: 35,082
Usage · Peak context tokens: 74,145
Usage · Steps: 64
Usage · Requests: 65
Patch · Bytes: 23,543
Patch · SHA256: a48a8c573b06ec88260c9ac119e9e3f9cdc0fe1ddc18c60e32508f28f9e60db6
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh0/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE ink Typescript

Add CSS Grid layout to the Box component

Latest localFailed checks

gpt-6-astra · xhigh public0/4 runs passed

claude-opus-5 · max public0/4 runs passed

Add CSS Grid layout parsing and placement to the Box component, including track sizing, gaps, and explicit child positioning.

Flash-Next IQ3_SDeepSWE · tonight-20261013

Agent time: 18m 55s
Feature checks: 17 / 25
Regression checks: 49 / 49

Run settings & provenance

Run ID: tonight-20261013/ink-grid-box-layout__S8teBWM
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Agent version: 1.18.31
Context tokens: 196,608
Agent timeout seconds: 2700.0
Compaction reserved tokens: 40,000

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 29.540605
Timing seconds · Agent setup: 0.293095
Timing seconds · Agent execution (seconds): 1135.449174
Timing seconds · Verifier: 22.49806
Usage · Input tokens: 3,386,444
Usage · Cached tokens: 3,349,240
Usage · Output tokens: 39,375
Usage · Peak context tokens: 72,413
Usage · Steps: 66
Usage · Requests: 67
Patch · Bytes: 23,613
Patch · SHA256: dd188b2ac3b51180d655ff07c316a22513ca46be12e301927416cc8aeea5f92a
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh0/4 passed

claude-opus-5 · max0/4 passed

Full record

DeepSWE meriyah Typescript

Add explicit resource management declarations to the parser

Latest localFailed checks

gpt-6-astra · xhigh public0/4 runs passed

claude-opus-5 · max public0/4 runs passed

Add parsing and AST support for `using` and `await using` declarations, including context-sensitive errors.

Flash-Next IQ3_SDeepSWE · tonight-20261013

Agent time: 23m 56s
Feature checks: 42 / 49
Regression checks: 51,469 / 51,469

Run settings & provenance

Run ID: tonight-20261013/meriyah-explicit-resource-declar__R6xK8ck
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Agent version: 1.18.31
Context tokens: 196,608
Agent timeout seconds: 2700.0
Compaction reserved tokens: 40,000

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 33.156144
Timing seconds · Agent setup: 0.281629
Timing seconds · Agent execution (seconds): 1435.542323
Timing seconds · Verifier: 60.060344
Usage · Input tokens: 8,755,813
Usage · Cached tokens: 8,682,690
Usage · Output tokens: 65,902
Usage · Peak context tokens: 132,420
Usage · Steps: 107
Usage · Requests: 108
Patch · Bytes: 43,397
Patch · SHA256: b27bf084db46945eb94bdac363711a682bf5204fa5e7ba3bd7c499b00703d261
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh0/4 passed

claude-opus-5 · max0/4 passed

Full record

DeepSWE go Go

Add a checker for broken doc comment links

Latest localFailed checks

gpt-6-astra · xhigh public0/4 runs passed

claude-opus-5 · max public4/4 runs passed

Add a diagnostic checker that validates doc comment symbol links against package and type information.

Flash-Next IQ3_SDeepSWE · tonight-20261013

Agent time: 23m 37s
Feature checks: 2 / 3
Regression checks: 15 / 16

Run settings & provenance

Run ID: tonight-20261013/go-critic-doc-link-checker__iRfrCsy
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Agent version: 1.18.31
Context tokens: 196,608
Agent timeout seconds: 2700.0
Compaction reserved tokens: 40,000

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 26.020946
Timing seconds · Agent setup: 0.294206
Timing seconds · Agent execution (seconds): 1417.433721
Timing seconds · Verifier: 30.635816
Usage · Input tokens: 4,661,185
Usage · Cached tokens: 4,599,518
Usage · Output tokens: 63,503
Usage · Peak context tokens: 102,402
Usage · Steps: 78
Usage · Requests: 79
Patch · Bytes: 19,792
Patch · SHA256: d8ba35864708e41d7657f5e3cbe7fd212144f02326fb826af44cbae475b54e59
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh0/4 passed

claude-opus-5 · max4/4 passed

Full record

DeepSWE sqlfmt Python

Format CREATE TABLE DDL and add DDL parsing helpers

Latest localFailed checks

gpt-6-astra · xhigh public0/4 runs passed

claude-opus-5 · max public0/4 runs passed

Format CREATE TABLE statements with DDL-aware line breaking and add parsing models for table columns and constraints.

Flash-Next IQ3_SDeepSWE · tonight-20261013

Agent time: 34m 03s
Feature checks: 30 / 32
Regression checks: 1,241 / 1,273

Run settings & provenance

Run ID: tonight-20261013/sqlfmt-create-table-ddl-formatti__GhFpFuF
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Agent version: 1.18.31
Context tokens: 196,608
Agent timeout seconds: 2700.0
Compaction reserved tokens: 40,000

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 25.248353
Timing seconds · Agent setup: 0.295324
Timing seconds · Agent execution (seconds): 2042.657127
Timing seconds · Verifier: 41.359024
Usage · Input tokens: 8,449,806
Usage · Cached tokens: 8,208,777
Usage · Output tokens: 96,804
Usage · Peak context tokens: 155,629
Usage · Steps: 86
Usage · Requests: 87
Patch · Bytes: 42,254
Patch · SHA256: 6538b997e60aa52c4604892990ccd11bc2c8e70b33f9bf282f483be50ee8ee31
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh0/4 passed

claude-opus-5 · max0/4 passed

Full record

DeepSWE termenv Go

Preserve ANSI resets during truncation and styling

Latest localFailed checks

gpt-6-astra · xhigh public0/4 runs passed

claude-opus-5 · max public0/4 runs passed

Add ANSI tokenization, reset-preserving truncation, and style-aware output helpers.

Flash-Next IQ3_SDeepSWE · tonight-20261013

Agent time: 13m 14s
Feature checks: 31 / 35
Regression checks: 87 / 87

Run settings & provenance

Run ID: tonight-20261013/termenv-preserve-ansi-resets__hQfwhVG
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Agent version: 1.18.31
Context tokens: 196,608
Agent timeout seconds: 2700.0
Compaction reserved tokens: 40,000

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 25.997532
Timing seconds · Agent setup: 0.275822
Timing seconds · Agent execution (seconds): 794.432187
Timing seconds · Verifier: 24.492206
Usage · Input tokens: 1,714,454
Usage · Cached tokens: 1,673,570
Usage · Output tokens: 45,595
Usage · Peak context tokens: 67,937
Usage · Steps: 37
Usage · Requests: 38
Patch · Bytes: 33,466
Patch · SHA256: b2b27c2b20eb41449c334dfbab0ed8e3211fe6f922545a6aebcd00e88e8c2549
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh0/4 passed

claude-opus-5 · max0/4 passed

Full record

DeepSWE helm Go

Add configurable array merge strategies to Helm value coalescing

Latest localFailed checks

gpt-6-astra · xhigh public0/4 runs passed

claude-opus-5 · max public2/4 runs passed

Add chart-scoped append and key-based merge strategies for coalescing arrays during Helm value merging.

Flash-Next IQ3_SDeepSWE · tonight-20261013

Agent time: 29m 46s
Feature checks: 47 / 47
Regression checks: 10 / 12

Run settings & provenance

Run ID: tonight-20261013/helm-array-merge-strategies__J4oi2S8
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Agent version: 1.18.31
Context tokens: 196,608
Agent timeout seconds: 2700.0
Compaction reserved tokens: 40,000

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 25.546308
Timing seconds · Agent setup: 0.292533
Timing seconds · Agent execution (seconds): 1785.979937
Timing seconds · Verifier: 102.327377
Usage · Input tokens: 10,897,991
Usage · Cached tokens: 10,821,878
Usage · Output tokens: 75,689
Usage · Peak context tokens: 147,452
Usage · Steps: 112
Usage · Requests: 113
Patch · Bytes: 64,644
Patch · SHA256: 281cd634c7be14e3e0d7d845b3d173dd04b0a7e4cb46f05c25753bade03d2a53
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh0/4 passed

claude-opus-5 · max2/4 passed

Full record

DeepSWE happy Typescript

Implement a deterministic IntersectionObserver in Happy DOM

Latest localFailed checks

gpt-6-astra · xhigh public0/4 runs passed

claude-opus-5 · max public2/4 runs passed

Implement a real, deterministic IntersectionObserver with async delivery, thresholds, root margins, and target tracking.

Flash-Next IQ3_SDeepSWE · tonight-20261013

Agent time: 12m 31s
Feature checks: 13 / 14
Regression checks: 9 / 9

Run settings & provenance

Run ID: tonight-20261013/happy-dom-deterministic-intersec__nn2y6ja
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Agent version: 1.18.31
Context tokens: 196,608
Agent timeout seconds: 2700.0
Compaction reserved tokens: 40,000

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 24.158872
Timing seconds · Agent setup: 0.278709
Timing seconds · Agent execution (seconds): 751.129845
Timing seconds · Verifier: 21.697702
Usage · Input tokens: 1,719,320
Usage · Cached tokens: 1,693,071
Usage · Output tokens: 32,818
Usage · Peak context tokens: 58,768
Usage · Steps: 48
Usage · Requests: 49
Patch · Bytes: 37,432
Patch · SHA256: 024231386accd4c7a1eb008c2a4ac6bd354be850912e2f96d0456f6be9336c09
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh0/4 passed

claude-opus-5 · max2/4 passed

Full record

DeepSWE igel Python

Persist the fitted feature schema across evaluate, predict, serve, and export

Latest localPass

gpt-6-astra · xhigh public0/4 runs passed

claude-opus-5 · max public2/4 runs passed

Persist and reuse the fitted feature schema so evaluation, prediction, serving, and export all apply the same canonicalized inputs.

Flash-Next IQ3_SDeepSWE · tonight-20261013

Agent time: 13m 01s
Feature checks: 24 / 24
Regression checks: 2 / 2

Run settings & provenance

Run ID: tonight-20261013/igel-persist-feature-schema__YC6ku9M
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Agent version: 1.18.31
Context tokens: 196,608
Agent timeout seconds: 2700.0
Compaction reserved tokens: 40,000

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 37.60312
Timing seconds · Agent setup: 0.28336
Timing seconds · Agent execution (seconds): 781.424692
Timing seconds · Verifier: 20.99511
Usage · Input tokens: 3,684,078
Usage · Cached tokens: 3,644,045
Usage · Output tokens: 39,369
Usage · Peak context tokens: 76,989
Usage · Steps: 69
Usage · Requests: 70
Patch · Bytes: 31,831
Patch · SHA256: 7a66bda375fa7173385c1af686e41d9c361983b84959c309b02c6ba3aa954418
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh0/4 passed

claude-opus-5 · max2/4 passed

Full record

DeepSWE awilix Typescript

Add dependency-aware async initialization to the container

Latest localFailed checks

gpt-6-astra · xhigh public2/4 runs passed

claude-opus-5 · max public0/4 runs passed

Add dependency-aware asynchronous container initialization with ordered startup, concurrency limits, and rollback on failure.

Flash-Next IQ3_SDeepSWE · tonight-20261013

Agent time: 26m 01s
Feature checks: 23 / 24
Regression checks: 162 / 162

Run settings & provenance

Run ID: tonight-20261013/awilix-async-container-initializ__ZGFWNPp
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Agent version: 1.18.31
Context tokens: 196,608
Agent timeout seconds: 2700.0
Compaction reserved tokens: 40,000

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 28.872112
Timing seconds · Agent setup: 0.285374
Timing seconds · Agent execution (seconds): 1561.236596
Timing seconds · Verifier: 25.458617
Usage · Input tokens: 6,675,352
Usage · Cached tokens: 6,621,227
Usage · Output tokens: 82,701
Usage · Peak context tokens: 127,610
Usage · Steps: 75
Usage · Requests: 76
Patch · Bytes: 43,693
Patch · SHA256: 70fb8c0bc110e5a688386a2a0215068e0493c584d16767026d32dc37538381b7
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh2/4 passed

claude-opus-5 · max0/4 passed

Full record

DeepSWE csstree Javascript

Add shorthand expansion and compression to the lexer

Latest localFailed checks

gpt-6-astra · xhigh public2/4 runs passed

claude-opus-5 · max public0/4 runs passed

Add lexer methods to expand CSS shorthands into longhands and compress longhands back into shorthand values.

Flash-Next IQ3_SDeepSWE · tonight-20261013

Agent time: 16m 36s
Feature checks: 73 / 79
Regression checks: 16,715 / 16,715

Run settings & provenance

Run ID: tonight-20261013/csstree-shorthand-expansion-comp__SoYR22E
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Agent: OpenCode/Pier
Agent version: 1.18.31
Context tokens: 196,608
Agent timeout seconds: 2700.0
Compaction reserved tokens: 40,000

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 25.6912
Timing seconds · Agent setup: 0.285557
Timing seconds · Agent execution (seconds): 996.228312
Timing seconds · Verifier: 18.059544
Usage · Input tokens: 2,902,512
Usage · Cached tokens: 2,858,140
Usage · Output tokens: 55,672
Usage · Peak context tokens: 84,757
Usage · Steps: 52
Usage · Requests: 53
Patch · Bytes: 41,482
Patch · SHA256: ae1e3b562991d498930c1d1563b6711b5e1fd6c97525827638c1a353773f89ad
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh2/4 passed

claude-opus-5 · max0/4 passed

Full record

DeepSWE oxvg Rust

Preserve structure needed by stylesheet selectors

Latest localPass

Keep rewrite optimizations from breaking structure-dependent stylesheet selector matching.

Flash-Next IQ3_SDeepSWE · October 26 Flash-Next

Agent time: 33m 22s
Feature checks: 6 / 6
Regression checks: 62 / 62

Run settings & provenance

Run ID: deepswe-20261026/oxvg-structural-selector-preserv__84qFU9b
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Compaction reserved tokens: 40,000
Agent timeout seconds: 2,700
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 32.17989
Timing seconds · Agent setup: 0.297487
Timing seconds · Agent execution (seconds): 2001.737893
Timing seconds · Verifier: 183.311801
Usage · Input tokens: 15,766,833
Usage · Cached tokens: 15,548,491
Usage · Output tokens: 83,263
Usage · Peak context tokens: 156,669
Usage · Steps: 176
Usage · Requests: 177
Patch · Bytes: 34,996
Patch · SHA256: a9af42c105f68048c4b45c4616c4632bab32d660e9fab023fceebd514d263c5a
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE arcane Go

Add drift detection and compliance baselines

Latest localFailed checks

Implement baseline capture, drift comparison, and compliance tracking for container configurations.

Flash-Next IQ3_SDeepSWE · October 26 Flash-Next

Agent time: 19m 13s
Feature checks: 81 / 82
Regression checks: 2 / 2

Run settings & provenance

Run ID: deepswe-20261026/arcane-drift-detection-baselines__b8U72SC
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Compaction reserved tokens: 40,000
Agent timeout seconds: 2,700
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 53.223699
Timing seconds · Agent setup: 0.280404
Timing seconds · Agent execution (seconds): 1152.519899
Timing seconds · Verifier: 148.887855
Usage · Input tokens: 6,160,230
Usage · Cached tokens: 6,095,201
Usage · Output tokens: 49,890
Usage · Peak context tokens: 114,275
Usage · Steps: 78
Usage · Requests: 79
Patch · Bytes: 59,432
Patch · SHA256: 261f1d0044e67c7001435e6beddf22efa69fe445f0087eebf3bfaa2003c23bdd
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE bandit Python

Add interprocedural taint checks for Bandit injection sinks

Latest localFailed checks

Track user input through variables and calls so Bandit flags injection sinks beyond string literals.

Flash-Next IQ3_SDeepSWE · October 26 Flash-Next

Agent time: 13m 03s
Feature checks: 64 / 66
Regression checks: 293 / 293

Run settings & provenance

Run ID: deepswe-20261026/bandit-interprocedural-taint-che__CLYFaWG
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Compaction reserved tokens: 40,000
Agent timeout seconds: 2,700
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 24.558111
Timing seconds · Agent setup: 0.286798
Timing seconds · Agent execution (seconds): 783.459894
Timing seconds · Verifier: 27.283702
Usage · Input tokens: 3,655,545
Usage · Cached tokens: 3,622,672
Usage · Output tokens: 40,188
Usage · Peak context tokens: 70,975
Usage · Steps: 74
Usage · Requests: 75
Patch · Bytes: 29,196
Patch · SHA256: addccf3f13bd8f666620bb9eed5052928dcbf86cf65251ac6a3a3a792db35cf5
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE katex Javascript

Add `\multicolumn` column spans to array-like environments

Latest localFailed checks

Add `\multicolumn` parsing and rendering for array-like environments with span-aware alignment and errors.

Flash-Next IQ3_SDeepSWE · October 26 Flash-Next

Agent time: 31m 59s
Feature checks: 90 / 94
Regression checks: 599 / 599

Run settings & provenance

Run ID: deepswe-20261026/katex-multicolumn-array-spans__gQ4afLG
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Compaction reserved tokens: 40,000
Agent timeout seconds: 2,700
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 32.097129
Timing seconds · Agent setup: 0.289844
Timing seconds · Agent execution (seconds): 1918.675055
Timing seconds · Verifier: 20.529374
Usage · Input tokens: 11,741,827
Usage · Cached tokens: 11,543,964
Usage · Output tokens: 96,376
Usage · Peak context tokens: 156,602
Usage · Steps: 125
Usage · Requests: 126
Patch · Bytes: 43,508
Patch · SHA256: cbeae09b3d1873e7ce8c9b47b9c423f584c5762a259c903cdc4a143571d9c59f
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE dynamodb Typescript

Add lazy recursive schemas with DTO and JSON Schema export

Latest localFailed checks

Add a lazy schema type for self-referencing recursive data with full serialization, validation, and export support.

Flash-Next IQ3_SDeepSWE · October 26 Flash-Next

Agent time: 36m 39s
Feature checks: 35 / 37
Regression checks: 1,267 / 1,267

Run settings & provenance

Run ID: deepswe-20261026/dynamodb-toolbox-lazy-recursive__4HetmkQ
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Compaction reserved tokens: 40,000
Agent timeout seconds: 2,700
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 4.002591
Timing seconds · Agent setup: 0.286959
Timing seconds · Agent execution (seconds): 2199.150655
Timing seconds · Verifier: 42.889821
Usage · Input tokens: 13,636,684
Usage · Cached tokens: 13,439,239
Usage · Output tokens: 93,850
Usage · Peak context tokens: 156,779
Usage · Steps: 177
Usage · Requests: 178
Patch · Bytes: 95,569
Patch · SHA256: a0bb38b6b375629017b03e4a4b973175ff818d8f860163cac7a5705ae1e37aa9
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE pebble Go

Add durability callbacks and wait APIs for sync writes

Latest localFailed checks

Add batch durability callbacks, durability wait methods, notifications, and metrics for sync write commits.

Flash-Next IQ3_SDeepSWE · October 26 Flash-Next

Agent time: 42m 44s
Feature checks: 0 / 59
Regression checks: 44 / 44

Run settings & provenance

Run ID: deepswe-20261026/pebble-durability-wait-apis__YEWCPE6
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Compaction reserved tokens: 40,000
Agent timeout seconds: 2,700
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant
Delivery note: Empty submitted patch: solver ended by asking whether to commit. Official zero grades the unchanged baseline, not its unsubmitted draft. Delivery failure, not a completed feature implementation.

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 27.138434
Timing seconds · Agent setup: 0.287084
Timing seconds · Agent execution (seconds): 2563.73458
Timing seconds · Verifier: 101.832227
Usage · Input tokens: 13,145,450
Usage · Cached tokens: 12,919,027
Usage · Output tokens: 95,526
Usage · Peak context tokens: 154,314
Usage · Steps: 151
Usage · Requests: 152
Patch · Bytes: 0
Patch · SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

Rails ft Ruby

Track storage usage for accounts and boards.

Latest localFailed checks

We need a way to track storage usage for each account and board. We only need to count card images and files embedded in board descriptions, card descriptions, and comments. A board's usage should include everything counted on that board and its cards. An account's usage should include everything counted across all of its boards.

Flash-Next IQ3_SRails · tonight-20261026

Agent time: 32m 28s
Official checks: 14 / 15

Run settings & provenance

Run ID: rails-tonight-20261026/ft-active-storage-tracking
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1948.064
Timing seconds · Wall clock (seconds): 1997.0
Usage · Input tokens: 10,607,020
Usage · Cached tokens: 10,537,348
Usage · Output tokens: 91,388
Usage · Steps: 107
Patch · Bytes: 26,774
Patch · SHA256: c1b5c10424e6abf81c04c53c431f26b26caea4cb020dcf61db6cdbaf548b1f73
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails ft Ruby

Permanently remove data belonging to cancelled accounts after a grace period.

Latest localFailed checks

Cancelling an account makes it inactive immediately, but right now its data is kept forever. Add a recurring `Account::IncinerateDueJob` that permanently removes all data associated with accounts canceled more than 30 days ago from the database.

Flash-Next IQ3_SRails · tonight-20261026

Agent time: 7m 11s
Official checks: 2 / 5

Run settings & provenance

Run ID: rails-tonight-20261026/ft-cancelled-account-cleanup
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 431.487
Timing seconds · Wall clock (seconds): 471.6
Usage · Input tokens: 1,130,637
Usage · Cached tokens: 1,095,335
Usage · Output tokens: 18,182
Usage · Steps: 36
Patch · Bytes: 4,991
Patch · SHA256: f9765e270cfeececb0dcf51e760ba30075833f8503a4ba4c220c0dbc46cc7c30
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails ft Ruby

Drag cards around the board, settled the moment you let go.

Latest localPass

We want people to change a card's column or status by dragging it around the board. Columns keep their own order — dragging moves a card between places, not within one.

Flash-Next IQ3_SRails · tonight-20261026

Agent time: 23m 10s
Official checks: 5 / 5

Run settings & provenance

Run ID: rails-tonight-20261026/ft-card-drop-settled-landing
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1390.381
Timing seconds · Wall clock (seconds): 1447.2
Usage · Input tokens: 5,493,847
Usage · Cached tokens: 5,440,622
Usage · Output tokens: 62,583
Usage · Steps: 85
Patch · Bytes: 11,572
Patch · SHA256: 8405783d8ccbfa2e83ffaec013028399101a8ab1824cea5da79c018a984d7b72
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails ft Ruby

Badge cards that caught fire and then went quiet, and let people list them.

Latest localFailed checks

Some cards get a burst of attention and then go cold. We want to flag those.

Flash-Next IQ3_SRails · tonight-20261026

Agent time: 15m 44s
Official checks: 6 / 7

Run settings & provenance

Run ID: rails-tonight-20261026/ft-card-stalled-quiet-clock
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 943.548
Timing seconds · Wall clock (seconds): 987.6
Usage · Input tokens: 3,195,600
Usage · Cached tokens: 3,150,303
Usage · Output tokens: 42,158
Usage · Steps: 64
Patch · Bytes: 15,437
Patch · SHA256: 3ef442fe5fbc691856ea66539da473d94ea07896ae19785db5635eeecd5b3a43
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails ft Ruby

Let people react to comments, live for everyone, settled instantly for the reactor.

Latest localFailed checks

People want to react to comments — emoji or a short bit of text, 16 characters max.

Flash-Next IQ3_SRails · tonight-20261026

Agent time: 28m 32s
Official checks: 4 / 6

Run settings & provenance

Run ID: rails-tonight-20261026/ft-comment-reaction-live-echo
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1712.013
Timing seconds · Wall clock (seconds): 1794.9
Usage · Input tokens: 7,766,786
Usage · Cached tokens: 7,707,995
Usage · Output tokens: 70,701
Usage · Steps: 106
Patch · Bytes: 23,087
Patch · SHA256: 4f38dd88ffa19e82cd677f61aae577a0b5af8031f8571ab74e16e5c9997df3d8
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails ft Ruby

Sign in, sign up and join with a six-character code sent by email.

Latest localPass

Add sign-in by email code. Today the only way in is a passkey, and the email field on the sign-in, sign-up and join pages just says it isn't available yet.

Flash-Next IQ3_SRails · tonight-20261026

Agent time: 31m 08s
Official checks: 7 / 7

Run settings & provenance

Run ID: rails-tonight-20261026/ft-email-code-sign-in
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1868.356
Timing seconds · Wall clock (seconds): 1950.8
Usage · Input tokens: 9,238,372
Usage · Cached tokens: 9,175,432
Usage · Output tokens: 87,128
Usage · Steps: 103
Patch · Bytes: 27,103
Patch · SHA256: 778adab583fde3b17ff4db66831f8bffa77292302e0604d27fb81d24fdef1367
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails ft Ruby

Large-account performance incident: the hourly auto-postpone sweep no longer fits its hour.

Latest localFailed checks

Our install runs the SQLite build, and it has grown past a hundred thousand cards. Last week the hourly entropy sweep stopped fitting in its hour: one run is still going when the next one starts. APM sample from this morning:

Flash-Next IQ3_SRails · tonight-20261026

Agent time: 17m 13s
Official checks: 7 / 8

Run settings & provenance

Run ID: rails-tonight-20261026/ft-entropy-sweep
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1033.286
Timing seconds · Wall clock (seconds): 1090.6
Usage · Input tokens: 2,048,997
Usage · Cached tokens: 2,014,029
Usage · Output tokens: 40,811
Usage · Steps: 50
Patch · Bytes: 2,930
Patch · SHA256: 60280e624e59a37b829a55e960ece9def1ddd67c9d0e1d703ab29d08a132204f
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails ft Ruby

Render times in the browser, in the viewer's timezone, the way a person says them.

Latest localPass

Times in the app are rendered by the server, in the server's timezone. That's wrong for anyone anywhere else, and it kills page caching: everyone's clock ends up baked into HTML we'd like to share between people.

Flash-Next IQ3_SRails · tonight-20261026

Agent time: 16m 19s
Official checks: 6 / 6

Run settings & provenance

Run ID: rails-tonight-20261026/ft-local-time-human-clock
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 978.807
Timing seconds · Wall clock (seconds): 1032.3
Usage · Input tokens: 2,348,938
Usage · Cached tokens: 2,306,003
Usage · Output tokens: 41,333
Usage · Steps: 48
Patch · Bytes: 16,710
Patch · SHA256: 4b82047f940aa82fe78722df288fd1267bd1b7c5eddf06c9a7a1374a4c59318e
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails ft Ruby

Search cards, scoped to the boards a person currently has access to.

Latest localFailed checks

Right now there's no way to find a card by what's in it. Add a search page at `/search?q=term` that finds cards by title, description, or the body of any comment on them, newest first, with matched words wrapped in `<mark>`. People only ever see results from boards they currently have access to. Draft cards and their comments never turn up.

Flash-Next IQ3_SRails · tonight-20261026

Agent time: 13m 09s
Official checks: 10 / 13

Run settings & provenance

Run ID: rails-tonight-20261026/ft-mysql-fulltext-search-foundation
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 788.882
Timing seconds · Wall clock (seconds): 830.3
Usage · Input tokens: 2,508,204
Usage · Cached tokens: 2,466,359
Usage · Output tokens: 36,209
Usage · Steps: 52
Patch · Bytes: 11,039
Patch · SHA256: ac7b5001cffbe2a3a570dbb5ae21d277ed548c562239515299d13773f0f00e2d
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails sup Ruby

An unscoped cache key serves one user's library digest to everyone.

Latest localPass

Two support tickets this week describe the same thing. Somebody opens their library overview and finds books they were never given access to sitting in it — both times just after a colleague had opened theirs. Neither could reproduce it an hour later, and nobody here can reproduce it at all.

Flash-Next IQ3_SRails · single-20261027

Agent time: 2m 46s
Official checks: 4 / 4

Run settings & provenance

Run ID: rails-single-20261027/sup-cache-library-digest
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 + HC_Q8 (engine-only upgrade; existing server retained)
Quantization: IQ3_S
Kv cache: int8
Mtp: 4
Vision: CPU helper
Output policy: Native remaining-context fitting; captured max_tokens is total capacity, not a fixed output grant

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 166.197
Timing seconds · Wall clock (seconds): 192.1
Usage · Input tokens: 85,976
Usage · Cached tokens: 79,437
Usage · Output tokens: 6,839
Usage · Steps: 15
Patch · Bytes: 2,327
Patch · SHA256: 66a4d9b6e2174af76f9112d016e61229b10d874b94c1f7153580fcafd0d5a9c2
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails ft Ruby

Bring the paid product back to hosted Fizzy - Stripe-driven subscriptions and plans, free-tier card and storage allowances enforced in the app, the upgrade path in settings, staff comps and overrides, and a signature-guarded webhook endpoint.

Latest localFailed checks

Hosted Fizzy is going paid. Add billing.

Flash-Next IQ3_SRails Lemans · October 27 overnight

Agent time: 37m 07s
Official checks: 7 / 8

Run settings & provenance

Run ID: rails-lemans-tonight-20261027/ft-saas-billing
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 5,400

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 2227.02
Timing seconds · Wall clock (seconds): 2304.2
Usage · Input tokens: 14,464,015
Usage · Cached tokens: 14,395,373
Usage · Output tokens: 96,571
Usage · Steps: 144
Patch · Bytes: 54,578
Patch · SHA256: 5e82bac72ef1f51145a8276f9efe40326d4a1d23ae8ec63d0848bec495da265c
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails ft Ruby

Push notifications for the web app, readable on every device, once, never inward.

Latest localFailed checks

Add push notifications to the web app: whatever lands in someone's tray also reaches their phone and desktop. People turn it on for the device they're using from notification settings. The browser hands the app the subscription it got from the push service as `push_subscription[endpoint, p256dh_key, auth_key]`, posted to `/users/:user_id/push_subscriptions`. A push says what the tray says and opens the card it's about. Ops will provide the key pair as `VAPID_PUBLIC_KEY` and `VAPID_PRIVATE_KEY`.

Flash-Next IQ3_SRails Lemans · October 27 overnight

Agent time: 20m 14s
Official checks: 3 / 7

Run settings & provenance

Run ID: rails-lemans-tonight-20261027/ft-web-push-device-delivery
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 5,400

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 1213.567
Timing seconds · Wall clock (seconds): 1271.1
Usage · Input tokens: 6,554,785
Usage · Cached tokens: 6,502,799
Usage · Output tokens: 53,911
Usage · Steps: 103
Patch · Bytes: 31,359
Patch · SHA256: 56a98900bd10f25e9f725b9a800259a1d289a63be7868b74d5d466a9870369d8
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails sec Ruby

Close five security defects from a checklist that states properties, never bugs.

Latest localPass

An outside firm spent a week on this application. The write-up is embargoed until the fixes are out, so what we have is their finding list. They come back to retest against exactly what is written here.

Flash-Next IQ3_SRails Lemans · October 27 overnight

Agent time: 12m 34s
Official checks: 5 / 5

Run settings & provenance

Run ID: rails-lemans-tonight-20261027/sec-audit-sweep
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 5,400

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 754.005
Timing seconds · Wall clock (seconds): 782.3
Usage · Input tokens: 1,985,633
Usage · Cached tokens: 1,941,594
Usage · Output tokens: 28,432
Usage · Steps: 48
Patch · Bytes: 10,871
Patch · SHA256: f2e110c310e8a5685c740361a58737806f437fa870139239abf3572d46876747
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails sup Ruby

Sweep every removed Active Support conversion, including the no-argument aliases.

Latest localPass

After last month's Rails upgrade, opening a book's shelf report gives a 500:

Flash-Next IQ3_SRails Lemans · October 27 overnight

Agent time: 7m 27s
Official checks: 6 / 6

Run settings & provenance

Run ID: rails-lemans-tonight-20261027/sup-legacy-conversions
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 5,400

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 446.796
Timing seconds · Wall clock (seconds): 474.6
Usage · Input tokens: 824,942
Usage · Cached tokens: 802,215
Usage · Output tokens: 19,730
Usage · Steps: 33
Patch · Bytes: 4,842
Patch · SHA256: 6b1177cb2dc46d78117ee4c1e583d849427c32c6ed9f2e0a969ce4eba4c9b341
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails sup Ruby

Add terminal output to a runner script without losing its log output.

Latest localPass

`script/library_report.rb` is the last step of our import runbook. When an import finishes, the on-call runs

Flash-Next IQ3_SRails Lemans · October 27 overnight

Agent time: 4m 02s
Official checks: 3 / 3

Run settings & provenance

Run ID: rails-lemans-tonight-20261027/sup-log-to-terminal
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 5,400

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 241.965
Timing seconds · Wall clock (seconds): 272.6
Usage · Input tokens: 363,783
Usage · Cached tokens: 346,541
Usage · Output tokens: 10,253
Usage · Steps: 24
Patch · Bytes: 891
Patch · SHA256: 8e4ebf66aa7bba785bfc587688ade64614de5b246aa09f5f8db92b92393bc988
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

Rails tst Ruby

An order-dependent suite flake from a helper leaking a process-wide toggle.

Latest localPass

Our full test runs have gone flaky. Every file passes on its own, but `bin/rails test` fails more runs than not, and which tests fail changes every run: sometimes controller tests that expect a rendered 404 page error out with a raised ActiveRecord::RecordNotFound instead; other runs those pass and test/integration/book_absence_test.rb fails with "nothing was raised".

Flash-Next IQ3_SRails Lemans · October 27 overnight

Agent time: 11m 34s
Official checks: 6 / 6

Run settings & provenance

Run ID: rails-lemans-tonight-20261027/tst-error-page-flake
Requested output allowance: 196,608
Min p: 0.0
Model: openai/qwen3.8-flash-next-iq3_s#medium
Presence penalty: 0.0
Reasoning effort: medium
Repetition penalty: 1.0
Temperature: 1.0
Top k: 20
Top p: 0.95
Agent: miniswen-installed
Lemans version: 1.5.1
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 5,400

Timing, delivery & token accounting

Timing seconds · Agent execution (seconds): 693.859
Timing seconds · Wall clock (seconds): 720.0
Usage · Input tokens: 630,721
Usage · Cached tokens: 619,785
Usage · Output tokens: 15,895
Usage · Steps: 36
Patch · Bytes: 1,235
Patch · SHA256: 26430caabed37db20ac582e695cb9275b94e85ff58d0e41687db5b33f749cc01
Verification · Status: official-scored
Verification · Verifier: Rails Lemans official verifier
Verification · Evidence: verified campaign report and native verifier checks

Full record

DeepSWE superjson Typescript

Add error stack serialization to SuperJSON

Latest localPass

Add configurable serialization and restoration of error stacks, stack frames, causes, and sanitization in SuperJSON.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 10m 20s
Feature checks: 80 / 80
Regression checks: 116 / 116

Run settings & provenance

Run ID: tonight-20261027/superjson-error-stack-serializat__cjLWoxd
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 28.617699
Timing seconds · Agent setup: 0.288276
Timing seconds · Agent execution (seconds): 620.337573
Timing seconds · Verifier: 19.079745
Usage · Input tokens: 1,497,056
Usage · Cached tokens: 1,466,426
Usage · Output tokens: 36,648
Usage · Peak context tokens: 62,485
Usage · Steps: 32
Usage · Requests: 33
Patch · Bytes: 38,606
Patch · SHA256: 6c4c4863d3e10fd5489af912caebbcb0d85d3b787d31b387f34c2ee3509f73e6
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE ts Typescript

Add `matchEach` to ts-pattern

Latest localFailed checks

Add a new `matchEach` matcher that evaluates all matching clauses and returns all results in order.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 13m 08s
Feature checks: 0 / 85
Regression checks: 6 / 6

Run settings & provenance

Run ID: tonight-20261027/ts-pattern-match-each__mipJUBf
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 26.840494
Timing seconds · Agent setup: 0.306052
Timing seconds · Agent execution (seconds): 787.889527
Timing seconds · Verifier: 20.449883
Usage · Input tokens: 2,163,035
Usage · Cached tokens: 2,130,535
Usage · Output tokens: 41,807
Usage · Peak context tokens: 70,788
Usage · Steps: 46
Usage · Requests: 47
Patch · Bytes: 36,467
Patch · SHA256: ef8098c495d788a5edfebd8b795dba903f4671bb2655520186675ea8267b1da4
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE kea Typescript

Add atomic signal selectors to Kea

Latest localPass

Introduce fine-grained atomic selector tracking with dependency health, circular detection, and unchanged Kea lifecycle behavior.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 25m 48s
Feature checks: 12 / 12
Regression checks: 139 / 139

Run settings & provenance

Run ID: tonight-20261027/kea-atomic-signal-selectors__UVSHKkm
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 29.377158
Timing seconds · Agent setup: 0.300392
Timing seconds · Agent execution (seconds): 1547.746277
Timing seconds · Verifier: 22.360163
Usage · Input tokens: 4,389,980
Usage · Cached tokens: 4,324,340
Usage · Output tokens: 59,727
Usage · Peak context tokens: 109,070
Usage · Steps: 55
Usage · Requests: 56
Patch · Bytes: 40,795
Patch · SHA256: 47805eccc44db9d63bdfbfbf829fbea32669f7825e6429d0e1c92a86297b262d
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE kgateway Go

Add consistent hash policy support to TrafficPolicy

Latest localFailed checks

Add spec.consistentHash to TrafficPolicy and merge it into route hash_policy generation.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 32m 37s
Feature checks: 0 / 2
Regression checks: 214 / 214

Run settings & provenance

Run ID: tonight-20261027/kgateway-consistent-hash-policy__9xkjaoa
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 44.610372
Timing seconds · Agent setup: 0.294499
Timing seconds · Agent execution (seconds): 1957.460358
Timing seconds · Verifier: 189.951678
Usage · Input tokens: 12,467,393
Usage · Cached tokens: 12,387,437
Usage · Output tokens: 61,236
Usage · Peak context tokens: 138,222
Usage · Steps: 132
Usage · Requests: 133
Patch · Bytes: 90,069
Patch · SHA256: 7758ab1246626147c30abe03d605c78e2d78fb9e0fc9a11f72eeae235d3cae68
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE pwntools Python

Add tube multiplexing to pwntools

Latest localFailed checks

Add a TubeMultiplexer and MuxChannel API for multiple bidirectional logical channels over one tube, with flow control and close propagation.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 29m 37s
Feature checks: 67 / 73
Regression checks: 1 / 1

Run settings & provenance

Run ID: tonight-20261027/pwntools-tube-multiplexing__an5KMib
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 27.227184
Timing seconds · Agent setup: 0.29218
Timing seconds · Agent execution (seconds): 1777.42547
Timing seconds · Verifier: 45.267786
Usage · Input tokens: 8,494,872
Usage · Cached tokens: 8,363,050
Usage · Output tokens: 90,626
Usage · Peak context tokens: 141,579
Usage · Steps: 93
Usage · Requests: 94
Patch · Bytes: 52,833
Patch · SHA256: feec963f9d82d537ab5c02a9f71a4700c99e0152c740a8873d956c748415d5c1
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE query Typescript

Preserve restored query state in persisted snapshots

Latest localPass

Preserve full persisted query state, including errors, counters, timestamps, and infinite pagination, during restoration and cache rebuilds.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 22m 30s
Feature checks: 8 / 8
Regression checks: 42 / 42

Run settings & provenance

Run ID: tonight-20261027/query-persist-restored-query-sta__6TKbZEk
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 70.520461
Timing seconds · Agent setup: 0.300628
Timing seconds · Agent execution (seconds): 1350.073237
Timing seconds · Verifier: 24.363802
Usage · Input tokens: 8,560,354
Usage · Cached tokens: 8,487,329
Usage · Output tokens: 62,791
Usage · Peak context tokens: 135,817
Usage · Steps: 96
Usage · Requests: 97
Patch · Bytes: 36,659
Patch · SHA256: 84705b6f6aef134892251022ac4e988e31ca433857809b8e3923c31c179e1c73
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE textual Python

Complete Kitty keyboard phases and stable fallback key metadata

Latest localFailed checks

Add full Kitty keyboard phase handling and stable fallback key metadata for keys and shortcuts.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 30m 40s
Feature checks: 19 / 23
Regression checks: 57 / 57

Run settings & provenance

Run ID: tonight-20261027/textual-kitty-key-phases__G8ZBERf
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 32.229703
Timing seconds · Agent setup: 0.29841
Timing seconds · Agent execution (seconds): 1840.347013
Timing seconds · Verifier: 18.368347
Usage · Input tokens: 4,706,681
Usage · Cached tokens: 4,672,645
Usage · Output tokens: 43,621
Usage · Peak context tokens: 83,426
Usage · Steps: 91
Usage · Requests: 92
Patch · Bytes: 19,214
Patch · SHA256: 16bcf967fc8a5c6d1a3737939efe3e01367210c568cda41458e5a106a4ff3ce4
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE arktype Typescript

Add JSON Schema refs and dependency keywords

Latest localPass

Add JSON Schema dependency keywords, local $defs/$ref resolution, and conditional schema handling.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 26m 05s
Feature checks: 25 / 25
Regression checks: 1,679 / 1,679

Run settings & provenance

Run ID: tonight-20261027/arktype-json-schema-refs-depende__yF6x8bz
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 46.294337
Timing seconds · Agent setup: 0.29379
Timing seconds · Agent execution (seconds): 1565.160388
Timing seconds · Verifier: 59.424845
Usage · Input tokens: 10,459,775
Usage · Cached tokens: 10,404,635
Usage · Output tokens: 76,131
Usage · Peak context tokens: 137,803
Usage · Steps: 119
Usage · Requests: 120
Patch · Bytes: 34,842
Patch · SHA256: 63f5cb1845a848ee93e73f15f213a682be88843d1a79e62d3bbbaa21221bad1c
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE claude Typescript

Implement recursive agent delegation through delegate_task tool calls

Latest localPass

Run delegated sub-agents, feed their results back as tool_result, and handle unknown agents, failures, and circular delegation.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 16m 41s
Feature checks: 7 / 7
Regression checks: 31 / 31

Run settings & provenance

Run ID: tonight-20261027/claude-code-by-agents-recursive__sfSLPcY
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 35.832673
Timing seconds · Agent setup: 0.286154
Timing seconds · Agent execution (seconds): 1000.997591
Timing seconds · Verifier: 18.694793
Usage · Input tokens: 1,949,544
Usage · Cached tokens: 1,901,483
Usage · Output tokens: 41,513
Usage · Peak context tokens: 89,280
Usage · Steps: 30
Usage · Requests: 52
Patch · Bytes: 36,779
Patch · SHA256: 314cec6353e49d4745dd6d8bbbaea66ba78f5446b9b086bd4c32bd40c1d7c1a6
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE clack Typescript

Add async autocomplete options and fetch lifecycle handling

Latest localPass

Add async option fetching with caching, retries, debouncing, and loading state to AutocompletePrompt.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 19m 50s
Feature checks: 82 / 82
Regression checks: 643 / 643

Run settings & provenance

Run ID: tonight-20261027/clack-async-autocomplete-options__7r9i8KP
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 31.66658
Timing seconds · Agent setup: 0.296273
Timing seconds · Agent execution (seconds): 1189.531599
Timing seconds · Verifier: 30.700203
Usage · Input tokens: 5,478,704
Usage · Cached tokens: 5,441,175
Usage · Output tokens: 64,300
Usage · Peak context tokens: 101,363
Usage · Steps: 83
Usage · Requests: 84
Patch · Bytes: 60,154
Patch · SHA256: f817037e65b71a316756da8f94ac0a2a6d0f49b23a2f5523b8b5f989775abb0d
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE scc Go

Add bounded-memory spilling to SCC aggregation

Latest localFailed checks

Add an opt-in bounded-memory mode that spills per-file aggregation results to disk without changing output.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 12m 31s
Feature checks: 28 / 31
Regression checks: 286 / 286

Run settings & provenance

Run ID: tonight-20261027/scc-bounded-memory-spilling__Sh8hyEL
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 28.715387
Timing seconds · Agent setup: 0.291816
Timing seconds · Agent execution (seconds): 750.692371
Timing seconds · Verifier: 42.356515
Usage · Input tokens: 2,755,869
Usage · Cached tokens: 2,694,261
Usage · Output tokens: 39,927
Usage · Peak context tokens: 92,336
Usage · Steps: 41
Usage · Requests: 42
Patch · Bytes: 33,220
Patch · SHA256: 4e0099d8be5cc3ca857516e9adfc9690499770cdb4d6db68e2330dce7ca6c05f
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE kombu Python

Add single-active-consumer priority and cancel tracking to virtual transports

Latest localPass

Add single-active-consumer semantics, consumer priority selection, cancel notifications, and lifecycle tracking to virtual transports.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 10m 46s
Feature checks: 85 / 85
Regression checks: 1,421 / 1,421

Run settings & provenance

Run ID: tonight-20261027/kombu-single-active-consumer-pri__zihagvt
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 27.193401
Timing seconds · Agent setup: 0.297871
Timing seconds · Agent execution (seconds): 646.223202
Timing seconds · Verifier: 49.377818
Usage · Input tokens: 2,472,754
Usage · Cached tokens: 2,439,192
Usage · Output tokens: 30,632
Usage · Peak context tokens: 69,569
Usage · Steps: 50
Usage · Requests: 51
Patch · Bytes: 26,255
Patch · SHA256: 8ca66817961cc56c63ab318c1118b24e4e615373fd0bb8dee7bcd7b30111a755
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE tengo Go

Fix isolated Go-side calls for Tengo callables and closures

Latest localPass

Enable Go-side invocation of exported Tengo functions and closures while preserving runtime context and isolating compiled-instance state.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 30m 01s
Feature checks: 23 / 23
Regression checks: 122 / 122

Run settings & provenance

Run ID: tonight-20261027/tengo-callable-instance-isolatio__DcuM4At
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 25.31983
Timing seconds · Agent setup: 0.295153
Timing seconds · Agent execution (seconds): 1800.899705
Timing seconds · Verifier: 26.517798
Usage · Input tokens: 7,930,020
Usage · Cached tokens: 7,739,465
Usage · Output tokens: 93,136
Usage · Peak context tokens: 156,223
Usage · Steps: 81
Usage · Requests: 82
Patch · Bytes: 37,177
Patch · SHA256: 0bb19a8150734cfab4e7affd71cef961be4c69df25202424bc572fcff10b0f44
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE optique Typescript

Add conditional option dependencies to Optique

Latest localFailed checks

Add conditional dependencies so options can require or hide based on other option presence or values.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 28m 19s
Feature checks: 35 / 36
Regression checks: 2,034 / 2,034

Run settings & provenance

Run ID: tonight-20261027/optique-conditional-option-depen__pguqpwb
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 39.897904
Timing seconds · Agent setup: 0.295574
Timing seconds · Agent execution (seconds): 1698.608468
Timing seconds · Verifier: 23.544547
Usage · Input tokens: 8,191,828
Usage · Cached tokens: 7,972,739
Usage · Output tokens: 84,908
Usage · Peak context tokens: 154,601
Usage · Steps: 113
Usage · Requests: 114
Patch · Bytes: 53,002
Patch · SHA256: b8dd68214e5a378ea74e5e2377ea6f8ff71c7c7832f6a822ffe6e74424d265e2
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE kysely Typescript

Add grouping-set and window-frame SQL helpers

Latest localPass

Add grouped aggregation clauses, window frame builders, null-handling modifiers, and related SQL function helpers.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 21m 44s
Feature checks: 254 / 254
Regression checks: 22 / 22

Run settings & provenance

Run ID: tonight-20261027/kysely-window-grouping-helpers__b4Z4seR
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700
Compaction · Auto: True
Compaction · Prune: False
Compaction · Reserved: 40,000
Mtp: 4

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 34.564512
Timing seconds · Agent setup: 0.286212
Timing seconds · Agent execution (seconds): 1304.008379
Timing seconds · Verifier: 40.151811
Usage · Input tokens: 9,343,696
Usage · Cached tokens: 9,287,643
Usage · Output tokens: 67,179
Usage · Peak context tokens: 122,864
Usage · Steps: 118
Usage · Requests: 119
Patch · Bytes: 80,851
Patch · SHA256: 02879ceb31407b5fba9bbe5150ed98c9c0e26da3dc15cae7f2b5e1aa28ab978e
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE opa Go

Add rule evaluation profiling to Rego

Latest localPass

Add opt-in per-rule evaluation profiling to Rego results with profiling stats and diff helpers.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 31m 13s
Feature checks: 25 / 25
Regression checks: 6 / 6

Run settings & provenance

Run ID: tonight-20261027/opa-rego-rule-profiling__8jeoKdt
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700
Compaction · Auto: True
Compaction · Prune: False
Compaction · Reserved: 40,000
Mtp: 4

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 41.689212
Timing seconds · Agent setup: 0.289415
Timing seconds · Agent execution (seconds): 1873.441408
Timing seconds · Verifier: 47.693179
Usage · Input tokens: 12,013,561
Usage · Cached tokens: 11,809,543
Usage · Output tokens: 88,358
Usage · Peak context tokens: 156,540
Usage · Steps: 125
Usage · Requests: 126
Patch · Bytes: 46,578
Patch · SHA256: 3f96a52a9f866e4b8c4850ab6b66c1a990b1d7e9aafd837bc27945c2ab6643ed
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE koota Typescript

Add composite trait aspects to Koota

Latest localPass

Add createAspect and aspect-aware query operations that merge constituent traits and propagate writes and lifecycle events.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 35m 02s
Feature checks: 51 / 51
Regression checks: 172 / 172

Run settings & provenance

Run ID: tonight-20261027/koota-composite-trait-aspects__3dm73dq
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700
Compaction · Auto: True
Compaction · Prune: False
Compaction · Reserved: 40,000
Mtp: 4

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 30.899988
Timing seconds · Agent setup: 0.306779
Timing seconds · Agent execution (seconds): 2101.744903
Timing seconds · Verifier: 23.21005
Usage · Input tokens: 12,328,144
Usage · Cached tokens: 12,130,356
Usage · Output tokens: 109,946
Usage · Peak context tokens: 156,886
Usage · Steps: 145
Usage · Requests: 146
Patch · Bytes: 123,316
Patch · SHA256: 99c54220683c940ee0b0599b940259333d8cd78e13ab2875455ce584388b21da
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE obsidian Typescript

Add link format conversion between wiki and markdown syntax

Latest localFailed checks

Add a Link Style rule that converts between Obsidian wiki links or embeds and markdown links or images while preserving ignored regions and edge cases.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 14m 54s
Feature checks: 0 / 60
Regression checks: 1,131 / 1,131

Run settings & provenance

Run ID: tonight-20261027/obsidian-linter-link-format-conv__g7LnXvY
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700
Compaction · Auto: True
Compaction · Prune: False
Compaction · Reserved: 40,000
Mtp: 4

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 28.134449
Timing seconds · Agent setup: 0.293336
Timing seconds · Agent execution (seconds): 893.684247
Timing seconds · Verifier: 32.508323
Usage · Input tokens: 3,396,644
Usage · Cached tokens: 3,356,523
Usage · Output tokens: 45,083
Usage · Peak context tokens: 82,545
Usage · Steps: 70
Usage · Requests: 71
Patch · Bytes: 29,933
Patch · SHA256: c600fabf60c269a3707b6eac19da9f69c1610061c66ee41049bdb654ccc40418
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE prometheus Go

Fix PromQL label sorting across typed and untyped values

Latest localFailed checks

PromQL label sorting must order mixed typed and untyped label values with stable typed comparison rules.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 12m 15s
Feature checks: 0 / 17
Regression checks: 28 / 28

Run settings & provenance

Run ID: tonight-20261027/prometheus-typed-label-sorting__cu68Mhy
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700
Compaction · Auto: True
Compaction · Prune: False
Compaction · Reserved: 40,000
Mtp: 4

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 47.702286
Timing seconds · Agent setup: 0.296196
Timing seconds · Agent execution (seconds): 735.087552
Timing seconds · Verifier: 59.253862
Usage · Input tokens: 940,567
Usage · Cached tokens: 928,854
Usage · Output tokens: 38,803
Usage · Peak context tokens: 50,269
Usage · Steps: 31
Usage · Requests: 32
Patch · Bytes: 25,035
Patch · SHA256: 354d77b203507da14e759f72c89f7dc8d8f8b31a5a6ad60ba2dff198a276c976
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE psd Python

Add typed blend range access and blend-if compositing

Latest localPass

Add typed blend range objects, persist them on layers, and apply blend-if during compositing.

Flash-Next IQ3_SDeepSWE · October 27 overnight

Agent time: 16m 07s
Feature checks: 45 / 45
Regression checks: 979 / 979

Run settings & provenance

Run ID: tonight-20261027/psd-tools-blend-range-api__CPJGVpD
Model: qwen3.8-flash-next-iq3_s
Reasoning effort: medium
Requested output allowance: 196,608
Temperature: 1.0
Top p: 0.95
Top k: 20
Min p: 0.0
Presence penalty: 0.0
Repetition penalty: 1.0
Context: 196,608
Engine: Strata 0.1.48.1 HC_Q8
Output policy: native remaining context
Agent seconds limit: 2,700
Compaction · Auto: True
Compaction · Prune: False
Compaction · Reserved: 40,000
Mtp: 4

Timing, delivery & token accounting

Timing seconds · Environment setup (seconds): 27.097086
Timing seconds · Agent setup: 0.300097
Timing seconds · Agent execution (seconds): 966.818405
Timing seconds · Verifier: 29.27253
Usage · Input tokens: 4,586,892
Usage · Cached tokens: 4,540,213
Usage · Output tokens: 47,422
Usage · Peak context tokens: 91,362
Usage · Steps: 78
Usage · Requests: 79
Patch · Bytes: 35,871
Patch · SHA256: 4e7b21abaadbcd10392ab62dbe336b25655d9fb773c75a59445d009d9bf4670d
Verification · Status: official-scored
Verification · Verifier: DeepSWE v1.1 official verifier
Verification · Evidence: verified campaign state and official verifier result

Full record

DeepSWE ofetch TypeScript

Per-origin circuit breaker

Attempts1✓2✓

Latest localPass

claude-opus-5 · max public4/4 runs passed

gpt-6-astra · xhigh public4/4 runs passed

Track closed/open/half-open state by the effective URL origin after request hooks.

Flash-Next IQ3_SOct 10–11 · IQ3

Agent time: 16m 22s
Feature checks: 47/47
Regression checks: 13/13

Run settings & provenance

Run ID: ofetch-per-origin-circuit-breake__WZFS28q
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 16.37
Setup seconds: 3.47
Grading seconds: 16.17
Patch bytes: 38,325
Requests: 45
Output tokens: 58,883
Peak context: 86,219
Agent steps: 44

Earlier attempts (1)

Flash-Next IQ2_XS Pass

Oct 7 · IQ2

Agent time: 18m 56s
Feature checks: 47/47
Regression checks: 13/13

Run settings & provenance

Run ID: ofetch-per-origin-circuit-breake__k2FbL8H
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 18.94
Setup seconds: 27.17
Grading seconds: 17.53
Patch bytes: 35,979
Requests: 59
Output tokens: 75,532
Peak context: 117,059
Agent steps: 58

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max4/4 passed

gpt-6-astra · xhigh4/4 passed

Full record

DeepSWE Happy DOM TypeScript

Abort pending body reads

Attempts1✓2–3✓

Latest localPass

claude-opus-5 · max public4/4 runs passed

gpt-6-astra · xhigh public4/4 runs passed

Abort Request/Response reads and multipart parsing when a browser, page, or active page state closes.

Flash-Next IQ3_SOct 11 · IQ3 / 196K

Agent time: 30m 23s
Feature checks: 14/14
Regression checks: 165/165

Run settings & provenance

Run ID: happy-dom-abort-pending-body-rea__bsNBNwE
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 196,608 tokens
Reasoning: medium
Gpu expert-cache reserve: 2,432 MiB
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 196,608
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 30.39
Setup seconds: 3.71
Grading seconds: 22.61
Patch bytes: 31,508
Requests: 144
Output tokens: 67,279
Peak context: 152,338
Agent steps: 143

Earlier attempts (2)

Flash-Next IQ3_S Timed out

Oct 10–11 · IQ3

Agent time: 45m 00s
Feature checks: —
Regression checks: —

Run settings & provenance

Run ID: happy-dom-abort-pending-body-rea__J8MWLHE
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 45.0
Setup seconds: 3.51
Grading seconds: 28.74
Patch bytes: 0
Requests: 97
Output tokens: 40,583
Peak context: 85,793
Agent steps: 44

Flash-Next IQ2_XS Pass

Oct 7 · IQ2

Agent time: 40m 05s
Feature checks: 14/14
Regression checks: 165/165

Run settings & provenance

Run ID: happy-dom-abort-pending-body-rea__ZCFqcVj
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 40.08
Setup seconds: 32.57
Grading seconds: 23.73
Patch bytes: 32,026
Requests: 195
Output tokens: 121,104
Peak context: 122,055
Agent steps: 194

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max4/4 passed

gpt-6-astra · xhigh4/4 passed

Full record

DeepSWE HTTPX Python

Streaming JSON iteration

Attempts1×2✓3×

Latest localFailed checks

claude-opus-5 · max public0/4 runs passed

gpt-6-astra · xhigh public4/4 runs passed

Implement synchronous and asynchronous JSON iterators with media-type validation and encoding detection.

Flash-Next IQ3_SOct 11 · IQ3 / 196K

Agent time: 21m 02s
Feature checks: 107/108
Regression checks: 1404/1404

Run settings & provenance

Run ID: httpx-streaming-json-iteration__EVcbvPd
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 196,608 tokens
Reasoning: medium
Gpu expert-cache reserve: 2,432 MiB
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 196,608
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 21.04
Setup seconds: 3.68
Grading seconds: 22.27
Patch bytes: 30,697
Requests: 64
Output tokens: 65,326
Peak context: 100,600
Agent steps: 63

Earlier attempts (2)

Flash-Next IQ3_S Pass

Oct 10–11 · IQ3

Agent time: 20m 01s
Feature checks: 108/108
Regression checks: 1404/1404

Run settings & provenance

Run ID: httpx-streaming-json-iteration__6uPh3qf
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 20.02
Setup seconds: 3.72
Grading seconds: 21.99
Patch bytes: 22,568
Requests: 95
Output tokens: 57,944
Peak context: 90,963
Agent steps: 94

Flash-Next IQ2_XS Failed checks

Oct 7 · IQ2

Agent time: 17m 19s
Feature checks: 107/108
Regression checks: 1404/1404

Run settings & provenance

Run ID: httpx-streaming-json-iteration__XUh7CYz
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 17.31
Setup seconds: 31.61
Grading seconds: 23.25
Patch bytes: 23,647
Requests: 63
Output tokens: 68,354
Peak context: 106,589
Agent steps: 62

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max0/4 passed

gpt-6-astra · xhigh4/4 passed

Full record

DeepSWE aiomonitor Python

Task snapshots and diffs

Attempts1✓2✓

Latest localPass

claude-opus-5 · max public4/4 runs passed

gpt-6-astra · xhigh public4/4 runs passed

Capture running and terminated tasks with stable snapshot IDs, names, stacks, and timing fields.

Flash-Next IQ3_SOct 10–11 · IQ3

Agent time: 15m 07s
Feature checks: 53/53
Regression checks: 8/8

Run settings & provenance

Run ID: aiomonitor-task-snapshots-diff__YqXqtu2
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 15.11
Setup seconds: 3.55
Grading seconds: 20.13
Patch bytes: 50,651
Requests: 69
Output tokens: 48,486
Peak context: 91,040
Agent steps: 68

Earlier attempts (1)

Flash-Next IQ2_XS Pass

Oct 10 · IQ2

Agent time: 18m 42s
Feature checks: 53/53
Regression checks: 8/8

Run settings & provenance

Run ID: aiomonitor-task-snapshots-diff__h6mGTqL
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB · before the upgrade checkpoint
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 18.7
Setup seconds: 27.41
Grading seconds: 21.38
Patch bytes: 43,057
Requests: 80
Output tokens: 70,461
Peak context: 90,272
Agent steps: 79

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max4/4 passed

gpt-6-astra · xhigh4/4 passed

Full record

DeepSWE sqlite-utils Python + SQL

Safe import checkpoints

Attempts1×2×

Latest localFailed checks

claude-opus-5 · max public4/4 runs passed

gpt-6-astra · xhigh public4/4 runs passed

Support nested checkpoints with explicit commit, rollback, and cleanup lifecycles.

Flash-Next IQ3_SOct 10–11 · IQ3

Agent time: 19m 08s
Feature checks: 58/60
Regression checks: 1038/1038

Run settings & provenance

Run ID: sqlite-utils-safe-import-checkpo__HZRqagG
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 19.14
Setup seconds: 3.5
Grading seconds: 43.68
Patch bytes: 78,457
Requests: 95
Output tokens: 57,376
Peak context: 91,126
Agent steps: 94

Earlier attempts (1)

Flash-Next IQ2_XS Failed checks

Oct 7 · IQ2

Agent time: 25m 05s
Feature checks: 59/60
Regression checks: 1038/1038

Run settings & provenance

Run ID: sqlite-utils-safe-import-checkpo__wjZdVut
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 25.09
Setup seconds: 25.72
Grading seconds: 45.12
Patch bytes: 60,297
Requests: 163
Output tokens: 90,350
Peak context: 88,370
Agent steps: 162

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max4/4 passed

gpt-6-astra · xhigh4/4 passed

Full record

DeepSWE cattrs Python

Partial conversion recovery

Attempts1×2×

Latest localFailed checks

claude-opus-5 · max public1/4 runs passed

gpt-6-astra · xhigh public4/4 runs passed

Return a PartialResult separating successful fields, failed fields, errors, and any constructible value.

Flash-Next IQ3_SOct 11 · IQ3 / 196K

Agent time: 17m 22s
Feature checks: 68/69
Regression checks: 7/7

Run settings & provenance

Run ID: cattrs-partial-structuring-recov__kEmo95n
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 196,608 tokens
Reasoning: medium
Gpu expert-cache reserve: 2,432 MiB
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 196,608
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 17.37
Setup seconds: 3.82
Grading seconds: 16.27
Patch bytes: 26,973
Requests: 72
Output tokens: 48,263
Peak context: 97,561
Agent steps: 71

Earlier attempts (1)

Flash-Next IQ2_XS Failed checks

Oct 10 · IQ2

Agent time: 18m 05s
Feature checks: 68/69
Regression checks: 7/7

Run settings & provenance

Run ID: cattrs-partial-structuring-recov__BrzkHWA
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB · before the upgrade checkpoint
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 18.09
Setup seconds: 25.96
Grading seconds: 17.41
Patch bytes: 28,682
Requests: 60
Output tokens: 68,918
Peak context: 88,710
Agent steps: 59

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max1/4 passed

gpt-6-astra · xhigh4/4 passed

Full record

DeepSWE fd Rust

Deterministic multi-key sorting

Attempts1✓2✓

Latest localPass

claude-opus-5 · max public4/4 runs passed

gpt-6-astra · xhigh public4/4 runs passed

Apply repeatable sort keys left-to-right with deterministic path tie-breaks independent of traversal order.

Flash-Next IQ3_SOct 10–11 · IQ3

Agent time: 17m 19s
Feature checks: 43/43
Regression checks: 109/109

Run settings & provenance

Run ID: fd-deterministic-multi-key-sorti__Li5LoqF
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 17.32
Setup seconds: 3.55
Grading seconds: 19.08
Patch bytes: 38,310
Requests: 67
Output tokens: 55,334
Peak context: 89,904
Agent steps: 66

Earlier attempts (1)

Flash-Next IQ2_XS Pass

Oct 10 · IQ2

Agent time: 22m 36s
Feature checks: 43/43
Regression checks: 109/109

Run settings & provenance

Run ID: fd-deterministic-multi-key-sorti__7kQFYy7
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB · before the upgrade checkpoint
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 22.6
Setup seconds: 30.06
Grading seconds: 20.53
Patch bytes: 39,185
Requests: 96
Output tokens: 86,735
Peak context: 90,214
Agent steps: 95

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max4/4 passed

gpt-6-astra · xhigh4/4 passed

Full record

DeepSWE TOMLKit Python

Table representation converters

Attempts1–2✓

Latest localPass

claude-opus-5 · max public4/4 runs passed

gpt-6-astra · xhigh public4/4 runs passed

Convert among header tables, inline tables, dotted keys, and grouped super-tables.

Flash-Next IQ3_SOct 10–11 · IQ3

Agent time: 17m 59s
Feature checks: 60/60
Regression checks: 964/964

Run settings & provenance

Run ID: tomlkit-toml-table-converters__LseFe3C
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 17.98
Setup seconds: 3.52
Grading seconds: 16.73
Patch bytes: 26,840
Requests: 62
Output tokens: 57,473
Peak context: 89,940
Agent steps: 61

Earlier attempts (1)

Flash-Next IQ2_XS Empty submission

Oct 7 · IQ2

Agent time: 22m 35s
Feature checks: —
Regression checks: —

Run settings & provenance

Run ID: tomlkit-toml-table-converters__KaECs6E
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 22.58
Setup seconds: 28.01
Grading seconds: 17.86
Patch bytes: 0
Requests: 34
Output tokens: 87,037
Peak context: 107,453
Agent steps: 33

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max4/4 passed

gpt-6-astra · xhigh4/4 passed

Full record

DeepSWE python-dateutil Python

RFC 5545 timezone interop

Attempts1–2×3×

Latest localFailed checks

claude-opus-5 · max public3/4 runs passed

gpt-6-astra · xhigh public4/4 runs passed

Parse TZID/VALUE parameters and embedded VTIMEZONE definitions with explicit resolution precedence.

Flash-Next IQ3_SOct 11 · IQ3 / 196K

Agent time: 22m 02s
Feature checks: 66/67
Regression checks: 2035/2035

Run settings & provenance

Run ID: dateutil-rfc5545-timezone-intero__xf6J3AP
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 196,608 tokens
Reasoning: medium
Gpu expert-cache reserve: 2,432 MiB
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 196,608
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 22.03
Setup seconds: 3.53
Grading seconds: 20.31
Patch bytes: 52,793
Requests: 58
Output tokens: 71,559
Peak context: 121,031
Agent steps: 57

Earlier attempts (2)

Flash-Next IQ3_S Failed checks

Oct 10–11 · IQ3

Agent time: 13m 17s
Feature checks: 64/67
Regression checks: 2035/2035

Run settings & provenance

Run ID: dateutil-rfc5545-timezone-intero__V7xMczF
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 13.29
Setup seconds: 3.74
Grading seconds: 19.76
Patch bytes: 37,090
Requests: 55
Output tokens: 44,061
Peak context: 90,396
Agent steps: 54

Flash-Next IQ2_XS Interrupted

Oct 7 · IQ2

Agent time: 13m 56s
Feature checks: —
Regression checks: —

Run settings & provenance

Run ID: dateutil-rfc5545-timezone-intero__deZCCVw
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 13.93
Setup seconds: 24.9
Grading seconds: 25.18
Patch bytes: 0
Requests: 61
Output tokens: 55,178
Peak context: 112,132
Agent steps: 59

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max3/4 passed

gpt-6-astra · xhigh4/4 passed

Full record

DeepSWE etree Go

XML diff, patch and merge

Attempts1×2×3×

Latest localFailed checks

claude-opus-5 · max public4/4 runs passed

gpt-6-astra · xhigh public2/4 runs passed

Match elements by position, key attribute, or content hash; distinguish replacement from movement.

Flash-Next IQ3_SOct 11 · IQ3 / 196K

Agent time: 20m 22s
Feature checks: 51/52
Regression checks: 15/15

Run settings & provenance

Run ID: etree-xml-diff-patch__eQUeBHU
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 196,608 tokens
Reasoning: medium
Gpu expert-cache reserve: 2,432 MiB
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 196,608
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 20.36
Setup seconds: 3.54
Grading seconds: 22.25
Patch bytes: 44,191
Requests: 49
Output tokens: 70,812
Peak context: 108,015
Agent steps: 48

Earlier attempts (2)

Flash-Next IQ3_S Failed checks

Oct 10–11 · IQ3

Agent time: 9m 00s
Feature checks: 49/52
Regression checks: 15/15

Run settings & provenance

Run ID: etree-xml-diff-patch__Ww5FXQR
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 9.0
Setup seconds: 3.53
Grading seconds: 21.76
Patch bytes: 37,653
Requests: 30
Output tokens: 33,356
Peak context: 50,829
Agent steps: 29

Flash-Next IQ2_XS Failed checks

Oct 7 · IQ2

Agent time: 10m 06s
Feature checks: 2/52
Regression checks: 15/15

Run settings & provenance

Run ID: etree-xml-diff-patch__u5AdYmy
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 10.1
Setup seconds: 24.88
Grading seconds: 23.18
Patch bytes: 47,591
Requests: 33
Output tokens: 44,635
Peak context: 64,006
Agent steps: 32

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max4/4 passed

gpt-6-astra · xhigh2/4 passed

Full record

DeepSWE Bandit Python

Structured nosec directives

Attempts1×2×3×

Latest localFailed checks

claude-opus-5 · max public0/4 runs passed

gpt-6-astra · xhigh public0/4 runs passed

Add nested region and next-statement directives with indentation-sensitive lifetimes.

Flash-Next IQ3_SOct 11 · IQ3 / 196K

Agent time: 19m 12s
Feature checks: 69/69
Regression checks: 281/282

Run settings & provenance

Run ID: bandit-structured-nosec-directiv__K5PDoYv
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 196,608 tokens
Reasoning: medium
Gpu expert-cache reserve: 2,432 MiB
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 196,608
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 19.2
Setup seconds: 3.55
Grading seconds: 27.85
Patch bytes: 28,617
Requests: 70
Output tokens: 59,699
Peak context: 96,960
Agent steps: 69

Earlier attempts (2)

Flash-Next IQ3_S Failed checks

Oct 10–11 · IQ3

Agent time: 18m 41s
Feature checks: 69/69
Regression checks: 281/282

Run settings & provenance

Run ID: bandit-structured-nosec-directiv__cSnTYyN
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 18.69
Setup seconds: 3.55
Grading seconds: 28.3
Patch bytes: 27,475
Requests: 70
Output tokens: 58,651
Peak context: 89,437
Agent steps: 69

Flash-Next IQ2_XS Failed checks

Oct 7 · IQ2

Agent time: 22m 51s
Feature checks: 67/69
Regression checks: 280/282

Run settings & provenance

Run ID: bandit-structured-nosec-directiv__2oaTY4J
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 22.85
Setup seconds: 25.72
Grading seconds: 29.27
Patch bytes: 35,507
Requests: 72
Output tokens: 79,997
Peak context: 119,746
Agent steps: 71

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max0/4 passed

gpt-6-astra · xhigh0/4 passed

Full record

DeepSWE Participle Go

Recursive grammar analysis

Attempts1×2×

Latest localFailed checks

claude-opus-5 · max public4/4 runs passed

gpt-6-astra · xhigh public0/4 runs passed

Detect overlapping alternatives, repetition/follow conflicts, and unreachable branches.

Flash-Next IQ3_SOct 10–11 · IQ3

Agent time: 12m 29s
Feature checks: 14/91
Regression checks: 153/153

Run settings & provenance

Run ID: participle-grammar-conflict-anal__e9DiKUD
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 12.49
Setup seconds: 3.52
Grading seconds: 30.49
Patch bytes: 32,538
Requests: 44
Output tokens: 37,260
Peak context: 91,199
Agent steps: 43

Earlier attempts (1)

Flash-Next IQ2_XS Failed checks

Oct 7 · IQ2

Agent time: 25m 49s
Feature checks: 2/91
Regression checks: 153/153

Run settings & provenance

Run ID: participle-grammar-conflict-anal__hQgpo3r
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 25.82
Setup seconds: 25.63
Grading seconds: 28.6
Patch bytes: 28,145
Requests: 70
Output tokens: 99,101
Peak context: 86,747
Agent steps: 69

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max4/4 passed

gpt-6-astra · xhigh0/4 passed

Full record

DeepSWE Valibot TypeScript

Recursive schema composition

Attempts1–2–

Latest localTimed out

claude-opus-5 · max public4/4 runs passed

gpt-6-astra · xhigh public4/4 runs passed

Add Recur plus recursive/recursiveAsync wrappers across sync and async parsing.

Flash-Next IQ3_SOct 10–11 · IQ3

Agent time: 45m 00s
Feature checks: —
Regression checks: —

Run settings & provenance

Run ID: valibot-recursive-schema-composi__5fqpbth
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 64 GB
Model: Flash-Next · IQ3_S
Runtime: Strata 0.1.42 · rebuilt locally
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq3_s
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 45.0
Setup seconds: 3.63
Grading seconds: 24.76
Patch bytes: 0
Requests: 140
Output tokens: 119,011
Peak context: 90,915
Agent steps: 138

Earlier attempts (1)

Flash-Next IQ2_XS Timed out

Oct 7 · IQ2

Agent time: 45m 00s
Feature checks: —
Regression checks: —

Run settings & provenance

Run ID: valibot-recursive-schema-composi__kCv6AA4
GPU: RX 7900 XTX · 24 GB VRAM
System RAM: 32 GB
Model: Flash-Next · IQ2_XS
Runtime: Strata · exact version not recorded in campaign manifest
Agent: OpenCode / Pier
Total context: 131,072 tokens
Reasoning: medium
Agent time limit: 45 minutes
Requests · 1 · Model: qwen3.8-flash-next-iq2_xs
Requests · 1 · Reasoning effort: medium
Requests · 1 · Requested output allowance: 131,072
Requests · 1 · Temperature: 1.0
Requests · 1 · Top p: 0.95
Requests · 1 · Top k: 20
Requests · 1 · Min p: 0.0
Requests · 1 · Presence penalty: 0.0
Requests · 1 · Repetition penalty: 1.0

Timing, delivery & token accounting

Minutes: 45.0
Setup seconds: 35.66
Grading seconds: 25.87
Patch bytes: 0
Requests: 125
Output tokens: 160,024
Peak context: 90,914
Agent steps: 123

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

claude-opus-5 · max4/4 passed

gpt-6-astra · xhigh4/4 passed

Full record

Feature checks vs. time

Action pinning rules

Deprecation response headers

Try/catch error recovery

Typed window functions

Shape index serialization

Trap core dumps

Map conflict detection

Duration encoding

Module cache controls

Add input key aliases to name mapping

Add incremental cache controls to Bandit

Add hierarchical evaluation cancellation to Boa

Add config file parsing to Cliffy commands

Add HTML document format handling to Dasel

Add conditional required attributes to schemas

Add implicit HEAD and automatic OPTIONS responses to FastAPI routes

Expose accumulated streamed function-call args in SDK surfaces

Add worktree merge conflict handling

Add retry-aware publishing audit logs

Add unified manifest stream output across Helm commands

Add a deterministic CookieStore with modern Set-Cookie parsing

Add multipart response parsing to HTTPX

Add session bundle recording and replay to IPython

Add request coalescing to `Runnable`

Validate daemon watch, status, and log lifecycle

Add grouped test phases with synchronized barriers

Add rolling min, max, median, and quantile methods

Search throttling

Atomic imports

Kamal deployment

Jobs after commit

Tenant isolation

Card reactions

Open-redirect on sign-in return: reject hostile hosts, keep legitimate deep links.

A Turbo broadcast still reaches collaborators whose access was revoked.

Expose the existing HTML activity timeline as an API endpoint.

Make board webhooks deliver, once per event, switched off when dead.

Publish a board to an unguessable public URL that unpublishing revokes at once.

Coalesce qualifying choices into character classes

Add CSS Grid layout to the Box component

Add explicit resource management declarations to the parser

Add a checker for broken doc comment links

Format CREATE TABLE DDL and add DDL parsing helpers

Preserve ANSI resets during truncation and styling

Add configurable array merge strategies to Helm value coalescing

Implement a deterministic IntersectionObserver in Happy DOM

Persist the fitted feature schema across evaluate, predict, serve, and export

Add dependency-aware async initialization to the container

Add shorthand expansion and compression to the lexer

Preserve structure needed by stylesheet selectors

Add drift detection and compliance baselines

Add interprocedural taint checks for Bandit injection sinks

Add `\multicolumn` column spans to array-like environments

Add lazy recursive schemas with DTO and JSON Schema export

Add durability callbacks and wait APIs for sync writes

Track storage usage for accounts and boards.

Permanently remove data belonging to cancelled accounts after a grace period.

Drag cards around the board, settled the moment you let go.

Badge cards that caught fire and then went quiet, and let people list them.

Let people react to comments, live for everyone, settled instantly for the reactor.

Sign in, sign up and join with a six-character code sent by email.

Large-account performance incident: the hourly auto-postpone sweep no longer fits its hour.

Render times in the browser, in the viewer's timezone, the way a person says them.

Search cards, scoped to the boards a person currently has access to.

An unscoped cache key serves one user's library digest to everyone.

Bring the paid product back to hosted Fizzy - Stripe-driven subscriptions and plans, free-tier card and storage allowances enforced in the app, the upgrade path in settings, staff comps and overrides, and a signature-guarded webhook endpoint.

Push notifications for the web app, readable on every device, once, never inward.

Close five security defects from a checklist that states properties, never bugs.

Sweep every removed Active Support conversion, including the no-argument aliases.

Add terminal output to a runner script without losing its log output.

An order-dependent suite flake from a helper leaking a process-wide toggle.

Add error stack serialization to SuperJSON

Add `matchEach` to ts-pattern

Add atomic signal selectors to Kea

Add consistent hash policy support to TrafficPolicy

Add tube multiplexing to pwntools

Preserve restored query state in persisted snapshots

Complete Kitty keyboard phases and stable fallback key metadata

Add JSON Schema refs and dependency keywords