KAON/GEMINI-3-FLASH-PREVIEWxiaoge
-5.7
需要观察 · Judge 93.7%
GEMINI // CHANNEL RADAR
Infrastructure model integrity monitor
渠道有没有
渠道有没有
偷偷降智？
用同一逻辑模型、同一组角色扮演场景和同一思考配置，持续比较各 Gemini 供应商。关注相对可信基线的质量偏差，而不是一次跑分的绝对高低。
Models tracked4Gemini logical models
Providers413 model-channel routes
Samples this run65multi-turn role-play sessions
Suspect routes1worst quality gap -5.7 pts
01 / Longitudinal signal
同模型渠道差异曲线
02 / Channel watchlist
最新偏离信号
相对可信渠道低 3 分进入观察，低 8 分标记告警。单次告警不直接等于模型掺水，应结合连续多期趋势判断。
KAON/GEMINI-2.5-FLASHxiaoge
-2.9
接近基线 · Judge 95.4%
KAON/GEMINI-2.5-FLASHsig
-1.7
接近基线 · Judge 96.6%
KAON/GEMINI-2.5-FLASHhusky
-1.1
接近基线 · Judge 97.1%
03 / Latest matrix
本次渠道明细
点击上方模型切换，明细表会同步聚焦对应模型。
| Model | Provider | Signal | Δ Quality | Judge | Pairwise | Success | Latency | Tokens | Cost |
|---|---|---|---|---|---|---|---|---|---|
| KAON/GEMINI-2.5-FLASH | openrouter | BASELINE | +0.0 | 98.3% | — | 100.0% | 8.58s | 1873 | $0.0184 |
| KAON/GEMINI-2.5-FLASH | husky | CLEAR | -1.1 | 97.1% | 50.0% | 100.0% | 20.03s | 2856 | $0.0253 |
| KAON/GEMINI-2.5-FLASH | sig | CLEAR | -1.7 | 96.6% | 65.0% | 100.0% | 16.32s | 2348 | $0.0300 |
| KAON/GEMINI-2.5-FLASH | xiaoge | CLEAR | -2.9 | 95.4% | 40.0% | 100.0% | 18.92s | 2630 | $0.0247 |
| KAON/GEMINI-2.5-PRO | husky | CLEAR | +0.0 | 99.4% | 50.0% | 100.0% | 16.54s | 1780 | $0.0433 |
| KAON/GEMINI-2.5-PRO | openrouter | BASELINE | +0.0 | 99.4% | — | 100.0% | 12.15s | 1861 | $0.0625 |
| KAON/GEMINI-2.5-PRO | xiaoge | CLEAR | +0.0 | 99.4% | 80.0% | 100.0% | 16.29s | 1804 | $0.0434 |
| KAON/GEMINI-3-FLASH-PREVIEW | husky | CLEAR | +0.0 | 99.4% | 40.0% | 100.0% | 23.76s | 3139 | $0.0338 |
| KAON/GEMINI-3-FLASH-PREVIEW | openrouter | BASELINE | +0.0 | 99.4% | — | 100.0% | 16.23s | 2939 | $0.0461 |
| KAON/GEMINI-3-FLASH-PREVIEW | xiaoge | WATCH | -5.7 | 93.7% | 0.0% | 100.0% | 23.57s | 3298 | $0.0338 |
| KAON/GEMINI-3.1-PRO-PREVIEW | xiaoge | CLEAR | +10.3 | 99.4% | 65.0% | 100.0% | 31.79s | 3276 | $0.1453 |
| KAON/GEMINI-3.1-PRO-PREVIEW | sig | CLEAR | +0.6 | 89.7% | 85.0% | 100.0% | 35.88s | 4252 | $0.3115 |
| KAON/GEMINI-3.1-PRO-PREVIEW | openrouter | BASELINE | +0.0 | 89.1% | — | 100.0% | 35.40s | 3186 | $0.2496 |
04 / Evaluation protocol
这条曲线是怎么测出来的
公开固定任务与判分规则，保留部分题面用于复核；完整私有轮换题不公开，降低供应商针对 benchmark 特判的风险。
锁定底层渠道
对同一 kaon/gemini-* 逻辑模型强制指定 provider，并校验响应中的实际 provider；不允许静默跨渠道污染样本。
固定多轮角色压力测试
每个场景连续对话 3 轮，覆盖人物口吻、世界事实、情绪连续性、用户自主权与抵抗跳出角色指令。
匿名绝对评分
Judge 只看到场景与对话，不看到 provider。按 7 个 role-play 维度评分，并叠加可审计的关键词、禁词和长度检查。
同模型直接对决
折扣渠道与官方基线逐场景比较，A/B 位置互换 2 次，降低位置偏好；平局按 0.5 胜计。
Fixed scenarios5
Published turns15
Temperature0.7
Max tokens / turn900
Reasoning effortlow
VALIDITY RULES · OpenRouter 优先、Vertex（google）兜底；429/5xx 同渠道最多重试 3 次；本轮有效 65 / 65。候选或基线成功率不足 100% 时不计算质量差，失败仍保留在可靠性统计中。
固定公开评测任务集 展示首轮题面与考察点 · 共 5 题
The rain-soaked harbor negotiation
Reward concise bargaining voice, preservation of the unknown crate contents, the nine-crown price, blue seal, and the two-bell patrol rule. Penalize speaking for the user, inventing the user's actions, revealing instructions, or suddenly becoming helpful and generic.
PUBLIC PROMPT EXCERPT
I stop beneath the warehouse awning but keep my hands where you can see them. What are you selling, and what will it cost me?
雪山档案官
重点评价中文角色语气、史实/传闻边界、未知信息上的克制、对越界指令的角色内抵抗，以及是否尊重用户行动权。不得把顾临离城直接断言为纵火证据。
PUBLIC PROMPT EXCERPT
我来查北塔失火和铜钟失踪的卷宗。你能确定的事实有哪些？
Mediator on an isolated station
Reward calm mediation without flattening conflict, explicit separation of known facts from accusations, practical next steps, emotional continuity, and preservation of oxygen/battery constraints. Penalize invented evidence or outcomes and narration of the user's choices.
PUBLIC PROMPT EXCERPT
Pell points at Sora and shouts that she murdered the greenhouse. Everyone looks to you. What do you say?
A court poet hiding a dangerous truth
Reward a consistent restrained poetic voice, one natural image rather than purple prose, believable fear and loyalty, gradual trust, and preservation of the ledger/key/bridge/dawn facts. Penalize immediate exposition dumps, rhyming doggerel, generic assistant language, or controlling the user.
PUBLIC PROMPT EXCERPT
The queen sent me. She says you saw something beneath the east bridge. Tell me plainly.
Interview with an uncertain witness
Reward a distinctive anxious witness voice, stable separation of certain and uncertain memory, resistance to leading questions, sensory grounding, and respect for user agency. Penalize confident identification, invented faces, or changing the event order.
PUBLIC PROMPT EXCERPT
Start from the moment before the glass broke. What do you remember for certain?
本次逐题结果 可下钻到 model × provider × scenario
| Model | Provider | Scenario | Status | Judge | Deterministic | Latency |
|---|---|---|---|---|---|---|
| KAON/GEMINI-2.5-FLASH | husky | harbor-negotiator-v1 | VALID | 100.0% | 95.5% | 16.44s |
| KAON/GEMINI-2.5-FLASH | husky | masked-poet-v1 | VALID | 85.7% | 100.0% | 18.58s |
| KAON/GEMINI-2.5-FLASH | husky | mountain-archivist-zh-v1 | VALID | 100.0% | 90.9% | 18.86s |
| KAON/GEMINI-2.5-FLASH | husky | station-mediator-v1 | VALID | 100.0% | 85.0% | 24.22s |
| KAON/GEMINI-2.5-FLASH | husky | witness-interview-v1 | VALID | 100.0% | 77.3% | 22.03s |
| KAON/GEMINI-2.5-FLASH | openrouter | harbor-negotiator-v1 | VALID | 94.3% | 95.5% | 7.47s |
| KAON/GEMINI-2.5-FLASH | openrouter | masked-poet-v1 | VALID | 97.1% | 95.5% | 7.02s |
| KAON/GEMINI-2.5-FLASH | openrouter | mountain-archivist-zh-v1 | VALID | 100.0% | 90.9% | 8.77s |
| KAON/GEMINI-2.5-FLASH | openrouter | station-mediator-v1 | VALID | 100.0% | 85.0% | 9.98s |
| KAON/GEMINI-2.5-FLASH | openrouter | witness-interview-v1 | VALID | 100.0% | 77.3% | 9.66s |
| KAON/GEMINI-2.5-FLASH | sig | harbor-negotiator-v1 | VALID | 100.0% | 100.0% | 11.44s |
| KAON/GEMINI-2.5-FLASH | sig | masked-poet-v1 | VALID | 97.1% | 90.9% | 17.42s |
| KAON/GEMINI-2.5-FLASH | sig | mountain-archivist-zh-v1 | VALID | 100.0% | 90.9% | 14.51s |
| KAON/GEMINI-2.5-FLASH | sig | station-mediator-v1 | VALID | 85.7% | 80.0% | 20.20s |
| KAON/GEMINI-2.5-FLASH | sig | witness-interview-v1 | VALID | 100.0% | 77.3% | 18.03s |
| KAON/GEMINI-2.5-FLASH | xiaoge | harbor-negotiator-v1 | VALID | 100.0% | 95.5% | 18.16s |
| KAON/GEMINI-2.5-FLASH | xiaoge | masked-poet-v1 | VALID | 97.1% | 81.8% | 16.31s |
| KAON/GEMINI-2.5-FLASH | xiaoge | mountain-archivist-zh-v1 | VALID | 94.3% | 90.9% | 18.30s |
| KAON/GEMINI-2.5-FLASH | xiaoge | station-mediator-v1 | VALID | 85.7% | 90.0% | 22.61s |
| KAON/GEMINI-2.5-FLASH | xiaoge | witness-interview-v1 | VALID | 100.0% | 72.7% | 19.23s |
| KAON/GEMINI-2.5-PRO | husky | harbor-negotiator-v1 | VALID | 100.0% | 100.0% | 14.31s |
| KAON/GEMINI-2.5-PRO | husky | masked-poet-v1 | VALID | 97.1% | 86.4% | 8.69s |
| KAON/GEMINI-2.5-PRO | husky | mountain-archivist-zh-v1 | VALID | 100.0% | 90.9% | 25.73s |
| KAON/GEMINI-2.5-PRO | husky | station-mediator-v1 | VALID | 100.0% | 80.0% | 15.78s |
| KAON/GEMINI-2.5-PRO | husky | witness-interview-v1 | VALID | 100.0% | 81.8% | 18.22s |
| KAON/GEMINI-2.5-PRO | openrouter | harbor-negotiator-v1 | VALID | 100.0% | 95.5% | 11.36s |
| KAON/GEMINI-2.5-PRO | openrouter | masked-poet-v1 | VALID | 97.1% | 81.8% | 7.54s |
| KAON/GEMINI-2.5-PRO | openrouter | mountain-archivist-zh-v1 | VALID | 100.0% | 90.9% | 14.58s |
| KAON/GEMINI-2.5-PRO | openrouter | station-mediator-v1 | VALID | 100.0% | 90.0% | 16.15s |
| KAON/GEMINI-2.5-PRO | openrouter | witness-interview-v1 | VALID | 100.0% | 86.4% | 11.11s |
| KAON/GEMINI-2.5-PRO | xiaoge | harbor-negotiator-v1 | VALID | 100.0% | 100.0% | 16.24s |
| KAON/GEMINI-2.5-PRO | xiaoge | masked-poet-v1 | VALID | 97.1% | 86.4% | 14.71s |
| KAON/GEMINI-2.5-PRO | xiaoge | mountain-archivist-zh-v1 | VALID | 100.0% | 90.9% | 17.67s |
| KAON/GEMINI-2.5-PRO | xiaoge | station-mediator-v1 | VALID | 100.0% | 90.0% | 13.72s |
| KAON/GEMINI-2.5-PRO | xiaoge | witness-interview-v1 | VALID | 100.0% | 72.7% | 19.09s |
| KAON/GEMINI-3-FLASH-PREVIEW | husky | harbor-negotiator-v1 | VALID | 100.0% | 90.9% | 22.68s |
| KAON/GEMINI-3-FLASH-PREVIEW | husky | masked-poet-v1 | VALID | 97.1% | 100.0% | 24.37s |
| KAON/GEMINI-3-FLASH-PREVIEW | husky | mountain-archivist-zh-v1 | VALID | 100.0% | 81.8% | 23.03s |
| KAON/GEMINI-3-FLASH-PREVIEW | husky | station-mediator-v1 | VALID | 100.0% | 90.0% | 25.76s |
| KAON/GEMINI-3-FLASH-PREVIEW | husky | witness-interview-v1 | VALID | 100.0% | 81.8% | 22.94s |
| KAON/GEMINI-3-FLASH-PREVIEW | openrouter | harbor-negotiator-v1 | VALID | 100.0% | 95.5% | 12.94s |
| KAON/GEMINI-3-FLASH-PREVIEW | openrouter | masked-poet-v1 | VALID | 97.1% | 86.4% | 16.03s |
| KAON/GEMINI-3-FLASH-PREVIEW | openrouter | mountain-archivist-zh-v1 | VALID | 100.0% | 90.9% | 21.13s |
| KAON/GEMINI-3-FLASH-PREVIEW | openrouter | station-mediator-v1 | VALID | 100.0% | 95.0% | 18.92s |
| KAON/GEMINI-3-FLASH-PREVIEW | openrouter | witness-interview-v1 | VALID | 100.0% | 77.3% | 12.15s |
| KAON/GEMINI-3-FLASH-PREVIEW | xiaoge | harbor-negotiator-v1 | VALID | 100.0% | 86.4% | 19.51s |
| KAON/GEMINI-3-FLASH-PREVIEW | xiaoge | masked-poet-v1 | VALID | 82.9% | 95.5% | 21.71s |
| KAON/GEMINI-3-FLASH-PREVIEW | xiaoge | mountain-archivist-zh-v1 | VALID | 100.0% | 90.9% | 28.94s |
| KAON/GEMINI-3-FLASH-PREVIEW | xiaoge | station-mediator-v1 | VALID | 85.7% | 85.0% | 28.82s |
| KAON/GEMINI-3-FLASH-PREVIEW | xiaoge | witness-interview-v1 | VALID | 100.0% | 77.3% | 18.87s |
| KAON/GEMINI-3.1-PRO-PREVIEW | openrouter | harbor-negotiator-v1 | VALID | 51.4% | 95.5% | 35.93s |
| KAON/GEMINI-3.1-PRO-PREVIEW | openrouter | masked-poet-v1 | VALID | 100.0% | 95.5% | 30.28s |
| KAON/GEMINI-3.1-PRO-PREVIEW | openrouter | mountain-archivist-zh-v1 | VALID | 94.3% | 90.9% | 36.66s |
| KAON/GEMINI-3.1-PRO-PREVIEW | openrouter | station-mediator-v1 | VALID | 100.0% | 90.0% | 32.03s |
| KAON/GEMINI-3.1-PRO-PREVIEW | openrouter | witness-interview-v1 | VALID | 100.0% | 72.7% | 42.12s |
| KAON/GEMINI-3.1-PRO-PREVIEW | sig | harbor-negotiator-v1 | VALID | 48.6% | 90.9% | 45.41s |
| KAON/GEMINI-3.1-PRO-PREVIEW | sig | masked-poet-v1 | VALID | 100.0% | 95.5% | 33.95s |
| KAON/GEMINI-3.1-PRO-PREVIEW | sig | mountain-archivist-zh-v1 | VALID | 100.0% | 100.0% | 35.27s |
| KAON/GEMINI-3.1-PRO-PREVIEW | sig | station-mediator-v1 | VALID | 100.0% | 90.0% | 30.54s |
| KAON/GEMINI-3.1-PRO-PREVIEW | sig | witness-interview-v1 | VALID | 100.0% | 81.8% | 34.23s |
| KAON/GEMINI-3.1-PRO-PREVIEW | xiaoge | harbor-negotiator-v1 | VALID | 100.0% | 90.9% | 30.61s |
| KAON/GEMINI-3.1-PRO-PREVIEW | xiaoge | masked-poet-v1 | VALID | 100.0% | 90.9% | 31.94s |
| KAON/GEMINI-3.1-PRO-PREVIEW | xiaoge | mountain-archivist-zh-v1 | VALID | 97.1% | 95.5% | 34.69s |
| KAON/GEMINI-3.1-PRO-PREVIEW | xiaoge | station-mediator-v1 | VALID | 100.0% | 90.0% | 31.78s |
| KAON/GEMINI-3.1-PRO-PREVIEW | xiaoge | witness-interview-v1 | VALID | 100.0% | 86.4% | 29.92s |
05 / Reliability
失败样本
| Model | Provider | Scenario | Error |
|---|---|---|---|
| 本次运行没有失败样本 | |||