Benchmarks

DeepSWE / abs · Go

Harden module loading, cache introspection, and script flags

Harden ABS module resolution and caching, expose cache introspection APIs, and make module flags work correctly in script mode.

Latest localFailed checks
gpt-6-astra · xhigh public4/4 runs passed
claude-opus-5 · max public4/4 runs passed

Run history

Flash-Next IQ3_SDeepSWE · follow-on · 2026-10-12

Agent time
8m 32s
Feature checks
13 / 20
Regression checks
3 / 3
Run settings & provenance
Run ID
tonight-20261012-followon/abs-module-cache-flags__eLaT3ZE
Model
qwen3.8-flash-next-iq3_s
Reasoning effort
medium
Requested output allowance
196,608
Temperature
1.0
Top p
0.95
Top k
20
Min p
0.0
Presence penalty
0.0
Repetition penalty
1.0
Agent
OpenCode/Pier
Runtime
Strata 0.1.42
Agent timeout seconds
2,700
Timing, delivery & token accounting
Timing seconds · Agent execution (seconds)
511.82121
Timing seconds · Environment setup (seconds)
3.665557
Timing seconds · Verifier
30.864019
Usage · Input tokens
1,892,538
Usage · Cached tokens
1,868,392
Usage · Output tokens
22,798
Usage · Peak context tokens
46,598
Usage · Steps
63
Usage · Requests
64
Patch · Bytes
19,487
Patch · SHA256
b0e8090ad4178406cfad69a6c73fd01c4382c87c26f7301f96ae399d547dc645
Verification · Status
official-scored
Verification · Verifier
DeepSWE v1.1 official verifier
Verification · Evidence
verified campaign state and official verifier result

Published comparisons

Published comparison sources

Published successful attempts using mini-swe-agent on the same task, not matched local runs.

gpt-6-astra · xhigh4/4 passed

Published attempt details
  1. run attempt
    1
    outcome
    pass
    passed
    True
  2. run attempt
    2
    outcome
    pass
    passed
    True
  3. run attempt
    3
    outcome
    pass
    passed
    True
  4. run attempt
    4
    outcome
    pass
    passed
    True

claude-opus-5 · max4/4 passed

Published attempt details
  1. run attempt
    None
    outcome
    pass
    passed
    True
  2. run attempt
    None
    outcome
    pass
    passed
    True
  3. run attempt
    None
    outcome
    pass
    passed
    True
  4. run attempt
    None
    outcome
    pass
    passed
    True

Official assignment

Open official task source ↗

Recorded assignment context

Harden ABS module resolution and caching, expose cache introspection APIs, and make module flags work correctly in script mode.