Gemini Latency Benchmark

🔬 Claims Verification

Empirically testing the assertions from the GDE discussion

Reporter PENDING

"The model consumed 62,910 thought tokens to generate 1,069 output tokens"

Max Thought Tokens Observed —

Avg Thought Tokens (Monolithic) —

Member A PENDING

"Flash won't be able to handle such complex prompt. Likely is going round and round in thoughts"

Flash Failure/Timeout Rate —

Thinking Budget Effectiveness —

Member B PENDING

"Pro models tend not to get caught in these reasoning exhaustion traps... can actually be the same or cheaper"

Pro Avg Cost vs Flash —

Pro Failure Rate —

📊 Performance Analysis

Visual comparison of all strategies

Latency Distribution

Average, min, and max latency per strategy

Thought Token Consumption

Average and peak thought tokens — the core metric

Cost Comparison

Average cost per request in USD

Thought Tokens vs Latency

Correlation between thinking depth and response time

📋 Detailed Comparison

Full metrics table across all strategies

🔍 Individual Runs

Every single API call with full details

Show failures only

🚀 Our Strategy: Structured Cascade

A novel approach to eliminate reasoning explosions

Stage 1

Meaning Extraction

temp=0.2 thinking_level=LOW

Extract definitions, synonyms, antonyms

→

⚡ Parallel per meaning

Stage 2

CEFR Examples

temp=0.7 thinking_level=LOW

Generate A1–C2 sentences

→

Stage 3

SpokenFi Transform

temp=0 thinking_level=MINIMAL

Minimal thinking — deterministic transform

Key Results

Thought Token Reduction —

Latency vs Monolithic —

Cost vs Monolithic —

Failure Rate —

Loading benchmark data...

Serve the repo root. If no snapshot exists yet, the dashboard will show an empty state until you run the benchmark.