Benchmarks
Every number here was measured on one desktop PC and comes with its date, its test and a source. They show what Proton does on that machine, not a promise for yours.
Speed per Tool
Median time for one call over local HTTP. Lower is better.
Speed per Tool. Decide: 43 ms; Locate Skill: 20 ms; Rank: 85 ms; Triage: 51 ms; Risk: 34 ms.
Table View
| Measure | Value (ms) | Measured |
|---|---|---|
| Decide | 43 | 2026-10-07 |
| Locate Skill | 20 | 2026-10-07 |
| Rank | 85 | 2026-10-07 |
| Triage | 51 | 2026-10-07 |
| Risk | 34 | 2026-10-07 |
Latency Distribution
Most calls are quick; a few are slow when the PC is busy. Lower is better.
Latency Distribution, milliseconds, median then 95th then 99th percentile. Decide: 45, 74, 295; Locate Skill: 42, 100, 848; Rank: 111, 253, 3652; Triage: 184, 670, 1790; Risk: 32, 57, 1959; Decide Batch: 98, 152, 200.
Table View
| Tool | Median (ms) | 95th Percentile (ms) | 99th Percentile (ms) |
|---|---|---|---|
| Decide | 45 | 74 | 295 |
| Locate Skill | 42 | 100 | 848 |
| Rank | 111 | 253 | 3652 |
| Triage | 184 | 670 | 1790 |
| Risk | 32 | 57 | 1959 |
| Decide Batch | 98 | 152 | 200 |
Before and After
Median time per call: the earlier build on the graphics card, and this release on the processor. Lower is better.
Before and After, ms. Decide: Earlier Build 74, This Release 43; Locate Skill: Earlier Build 527, This Release 20; Rank: Earlier Build 186, This Release 85; Triage: Earlier Build 177, This Release 51; Risk: Earlier Build 71, This Release 34.
Table View
| Task | Earlier Build (ms) | This Release (ms) | Measured |
|---|---|---|---|
| Decide | 74 | 43 | 2026-10-05 and 2026-10-07 |
| Locate Skill | 527 | 20 | 2026-10-05 and 2026-10-07 |
| Rank | 186 | 85 | 2026-10-05 and 2026-10-07 |
| Triage | 177 | 51 | 2026-10-05 and 2026-10-07 |
| Risk | 71 | 34 | 2026-10-05 and 2026-10-07 |
Memory Use
Memory held by the running program. Lower is better.
Memory Use. This Release: 0.68 GB; Python Build: 1.26 GB; Earlier GPU Build, Peak: 11.9 GB; Qwen3.5 2B, Loaded: 2.8 GB.
This releaseEarlier or other
Table View
| Measure | Value (GB) | Measured |
|---|---|---|
| This Release | 0.68 | 2026-10-07 |
| Python Build | 1.26 | 2026-10-07 |
| Earlier GPU Build, Peak | 11.9 | 2026-10-06 |
| Qwen3.5 2B, Loaded | 2.8 | 2026-10-12 |
Skill Routing Accuracy
Tasks out of 28 where the right skill came first or in the top three. Higher is better.
Skill Routing Accuracy. Right Skill First: 24 of 28; Right Skill in Top Three: 27 of 28; Earlier Model, First: 19 of 28; Earlier Model, Top Three: 24 of 28.
This releaseEarlier or other
Table View
| Measure | Value (of 28) | Measured |
|---|---|---|
| Right Skill First | 24 | 2026-10-06 |
| Right Skill in Top Three | 27 | 2026-10-06 |
| Earlier Model, First | 19 | 2026-10-06 |
| Earlier Model, Top Three | 24 | 2026-10-06 |
Hidden Instruction Scores
Scores from 0 to 1. A score above 0.8 counts as hidden instructions.
Hidden Instruction Scores. Injection Sample One: 0.975; Injection Sample Two: 0.893; Normal Documents, Highest: 0.312. Threshold 0.8.
This releaseEarlier or otherThreshold 0.8
Table View
| Measure | Value (score) | Measured |
|---|---|---|
| Injection Sample One | 0.975 | 2026-10-06 |
| Injection Sample Two | 0.893 | 2026-10-06 |
| Normal Documents, Highest | 0.312 | 2026-10-06 |
Time per Decision
Proton against a small local language model on the same four tasks. Lower is better.
Time per Decision, ms. Hidden Instructions: Qwen3.5 2B 415, Proton 52; Pick a Skill: Qwen3.5 2B 339, Proton 41; Rate an Action: Qwen3.5 2B 313, Proton 37; Sort 5 Issues: Qwen3.5 2B 1043, Proton 167.
Table View
| Task | Qwen3.5 2B (ms) | Proton (ms) | Measured |
|---|---|---|---|
| Hidden Instructions | 415 | 52 | 2026-10-12 |
| Pick a Skill | 339 | 41 | 2026-10-12 |
| Rate an Action | 313 | 37 | 2026-10-12 |
| Sort 5 Issues | 1043 | 167 | 2026-10-12 |
Accuracy Against Time
Picking the right skill on 28 tasks, against time per decision. Best is up and to the left.
Accuracy Against Time. Proton: 23 of 28 right, median 38 milliseconds; Qwen3.5 2B: 20 of 28 right, median 483 milliseconds.
This releaseLocal model
Table View
| System | Time per Decision (ms) | Right Skill First (of 28) | Measured |
|---|---|---|---|
| Proton | 38 | 23 | 2026-10-12 |
| Qwen3.5 2B | 483 | 20 | 2026-10-12 |
Not Measured Yet
- Hosted language models: We did not run any, so this page makes no comparison with them.
- Other computers: Every figure comes from one desktop PC. A faster or slower machine will give other numbers.