Source: https://anchor-sdk.dev/benchmarks.html
Package: anchor-ai-sdk@0.1.7
+ COMPLETED EVAL / PARASAIL GLM 5.3
# Nine of ten.
Both records open.
Anchor and mini-swe-agent each resolved nine tasks in the published Lite-10 artifacts. The different failures and setup limits matter as much as the aggregate.
COMPLETED RECORDS 19–20 OCT 2026
[Every task, every result](https://anchor-sdk.dev/benchmarks.html#results)
SWE-BENCH LITE / PARASAIL GLM 5.3OFFICIAL GRADER
Anchor**9/10**
\=
mini-swe-agent**9/10**
RESOLVED / 10 TASKS
Anchor**9/10**
mini-swe-agent**9/10**
Anchor: one unresolved patch. Mini: one empty patch after extraction failed. Both published grader reports record nine resolved tasks.
Parasail · temperature 0 · seed 42 · different budgets
READ THE FINE PRINT
Ten selected Lite tasks, not the full benchmark. Both records name `zai-org/GLM-5.3` on Parasail and Docker-backed task environments. Anchor's published limits are 120 steps / 25M tokens; mini uses its own 250-step / $3 loop. These are not identical budget policies, and fixed sampling parameters do not establish statistical significance or general superiority.
+ 01 / THE COMPLETED LEDGER
## Read the patches.
Not just the score.
[Anchor manifest](https://github.com/hrithikkoduri/anchor-ai-sdk/blob/main/evals/benchmarks/swe-bench/results/glm53-parasail/manifest.json)
zai-org/GLM-5.3 10 TASKS / PARASAIL
Completed Parasail GLM 5.3 ten-task results. Missing mini usage is shown as not reported, never zero.
Task
Anchor
mini-swe-agent
Anchor $
Mini $
django\_\_django-10914
Resolved
Resolved
$0.2579
Not reported
matplotlib\_\_matplotlib-18869
Resolved
—No patch
$0.3979
Not reported
mwaskom\_\_seaborn-2848
Resolved
Resolved
$1.4578
Not reported
pallets\_\_flask-4045
Resolved
Resolved
$0.4817
Not reported
psf\_\_requests-1963
Resolved
Resolved
$0.4610
Not reported
pydata\_\_xarray-3364
Resolved
Resolved
$0.3211
Not reported
pylint-dev\_\_pylint-5859
Resolved
Resolved
$0.1524
Not reported
pytest-dev\_\_pytest-11143
Resolved
Resolved
$0.1003
Not reported
sphinx-doc\_\_sphinx-10325
Resolved
Resolved
$0.5598
Not reported
sympy\_\_sympy-11400
—Unresolved
Resolved
$0.1263
Not reported
TOTAL / 10 TASKS
9/10
9/10
$4.3162
Not reported
Different stopping paths.
Mini has nine reported step counts (454 in total); the extraction-failed task has no step count. Its complete step aggregate, token usage, and cost are not reported. Task times are rounded here; source records retain milliseconds.
+ 02 / THE HONEST READ
## Equal resolution.
Different failure paths.
01
### Anchor: sympy remained unresolved.
The published patch did not satisfy the official tests. The record explains the missing relational printer; a local test pass was not enough.
02
### Mini: matplotlib produced no extracted patch.
The runner records a git index.lock error during patch extraction. This is a pipeline failure/empty patch, not evidence that the model submitted a patch which failed the hidden tests.
03
### No cost-efficiency ratio for this round.
Anchor records $4.3162 and 11.36M tokens. Mini does not report comparable usage. The earlier $0.60 / ≈$1.99 comparison belongs to an older OpenRouter baseline and is not reused here.
+ 03 / SOURCE ARTIFACTS
## Two records.
Inspectable outcomes.
Each arm has results, verdicts, manifests, and grading reports. Anchor also includes engine traces; mini does not emit Anchor's event schema.
[Anchor record](https://github.com/hrithikkoduri/anchor-ai-sdk/blob/main/evals/benchmarks/swe-bench/results/glm53-parasail/README.md) [Mini record](https://github.com/hrithikkoduri/anchor-ai-sdk/blob/main/evals/benchmarks/swe-bench/results/glm53-mini/README.md) [Anchor grader](https://github.com/hrithikkoduri/anchor-ai-sdk/blob/main/evals/benchmarks/swe-bench/results/glm53-parasail/grading/glm-5.3-parasail.json) [Mini grader](https://github.com/hrithikkoduri/anchor-ai-sdk/blob/main/evals/benchmarks/swe-bench/results/glm53-mini/grading/mini.json)
###
Both run records identify Docker-backed tasks and the same direct Parasail model endpoint with temperature 0 and seed 42. They ran at different times and used different step/budget policies; do not describe this as a perfectly controlled single-variable experiment.
###
###
###
###
+ 04 / RUN YOUR OWN EVAL
## Use the setup guide.
Keep your own records.
The commands start new runs on current code. Install eval dependencies and prepare task images first. Changing budgets or SDK/model versions changes the experiment; new outcomes are not guaranteed to match these artifacts.
[Environment & grading guide](https://github.com/hrithikkoduri/anchor-ai-sdk/blob/main/evals/benchmarks/swe-bench/README.md)
run-benchmark.shSHELL
```
# From the repository root, after eval setup. Cloud/model calls can cost money.
# PARASAIL_API_KEY must be set. Docker must be running and on PATH.
npm run eval --prefix evals -- \
--tasks benchmarks/swe-bench/data/swebench-lite-10.jsonl \
--provider docker --arm anchor --llm parasail \
--max-steps 120 --max-tokens 25000000
# Mini has its own loop/limits; consult the current runner's options.
npm run eval --prefix evals -- \
--tasks benchmarks/swe-bench/data/swebench-lite-10.jsonl \
--provider docker --arm mini --llm parasail
```
Evaluate the harness on your own workload.[Build your agent](https://anchor-sdk.dev/docs.html#quickstart)