Your Pickup Games Were 67/33. We Fixed That.

You know the feeling. Eight players, someone says “captains” or “same teams as yesterday,” and twelve minutes into dm2 one side is up 140 and checking Discord between deaths. Everyone shrugs — that’s pickup.

It does not have to be. We ran the numbers on every tracked 4on4 in the DeepFrag database, and the verdict on how we have been picking teams is grim: the average pickup game was effectively a 67/33 mismatch — while a genuinely fair split of the same eight players existed 94.5% of the time. We just kept not finding it. Human team-picking found the fair split 16.4% of the time.

How we know pickup teams were broken

Hindsight is cheap, so we did not use it. The backtest walks forward through 19,877 matches from January 2022 on: for every game, the engine rates all eight players using only the matches played before that day, predicts the outcome, then finds out what actually happened. No peeking. The headline numbers — 67/33 actual versus 52/48 available — come from the 9,946 of those games where all eight players had established ratings.

For every lobby we asked two questions. How even was the split people actually played? And how even was the best split hiding among the 35 ways to divide 8 players into two fours? The gap between those answers is the entire story: the fair game existed almost every night, in almost every lobby, and we walked past it.

The engine under the hood

Player ratings run on OpenSkill, the same Bayesian skill-rating family used across competitive games — with three QuakeWorld-specific upgrades, each of which had to earn its place in the walk-forward test before shipping:

• Frag margins, not just W/L. A 250–100 dm2 and a 220–218 overtime thriller are not the same win. Margin carries most of the outcome signal in team games.
• Per-map skill. Your schloss is not your dm2 — map-specific deviations are statistically real in 4on4 and improve prediction. (They also produce fun facts: see what 4,096 games say about schloss.)
• Individual contribution. Your rating moves on your performance relative to what your rating predicted for your roster, not just on whether your team won. Carry a losing team and the engine notices.

Two ways to grade a predictor. Winner accuracy is the obvious one: how often did it pick the right team? Log-loss is the stricter one: it grades confidence, not just correctness. Call a game 90% and win, you score well; call it 90% and lose, it hammers you; hedge everything at 50/50 and you sit at 0.693 forever. Lower is better, and every hundredth below 0.693 is knowledge the engine actually earned — you cannot cheat it by being loud or by being vague.

Accuracy is nice; calibration is what makes a balancer trustworthy. When this engine calls a game 50–60% for the favorite, the favorite historically wins 56% of those games. Call it 70–80%, they win 74.9%. Call it 90+, they win 95%. The probabilities mean what they say — which is exactly what you need when the whole point is engineering a coin flip. Credit where due: phylter’s 4on4 ratings project broke ground on QW team rating years ago and shaped our thinking.

Live fire

Validation is backtests until it is not. Recently an 8-player lobby ran two games on our watch. The engine called dm2 at 59.7% for red and schloss at just 51.8% — nearly even. A confident human in chat called schloss 65%.

Result: red won both — dm2 by 99, schloss by 33 (213–180), a game inside three frags with three minutes left. Two for two on winners, and the ordering was right too: the map the engine tilted harder was the bigger beating, and its near-coin-flip call was the nail-biter. That is what calibration looks like in the wild.

Use it tonight

The balancer is live on DeepFrag. It reads the active server list, pulls the eight players in your lobby, and hands you the three most even splits with win probabilities — projected across dm2, dm3, e1m2, and schloss, because a split that is fair on average can hide two lopsided maps. Someone aliasing? Anyone in the lobby can tag who it really is. Planning rather than playing? Build a hypothetical 8-stack from any rated players and see the odds before anyone connects.

Here is who the engine currently rates highest in 4on4:

One honest limitation: the engine treats skill as additive, and a practiced fixed roster plays better than the sum of its ratings — chemistry is real and it is not modeled (yet). And if your ratings look off because you just got back after twenty years: welcome home, here is the 10-minute path — the engine sorts you out within about ten games.

Fair games are more fun. We have the receipts now. Come argue about your rating in the Discord.