16 boxes, 1 jackpot, 8 searchersSixteen boxes, one jackpot, eight searchers
Each searcher gets a private clue. The clue shows 1 box. The clue is correct with probability 0.20. If it is not correct, it shows a random different box. Each searcher opens 1 box. The group wins if 1 or more searchers open the jackpot box. The clues do not change. Only the protocol changes โ the rule that makes actions from clues.
Each searcher gets a private clue naming a box โ right with probability 0.20, otherwise a uniformly random wrong box. Everyone opens one box; the group wins if anyone opens the jackpot. The clues are fixed; the only choice is the protocol that turns clues into actions.
Select a protocol. Then click Run once.Pick a protocol and press Run once.
Each bar shows your measured hit rate. The black tick shows the exact value from the paper. When you do many runs, each bar moves to its tick.Bars fill with your empirical hit rate; the black tick on each bar is the exact value the paper proves. Run enough trials and every bar converges to its tick.
Look at the differences. โ Consensus sends all 8 searchers to the box with the most clues. โ The market puts the searchers on the best 2 or 3 boxes. โ With private clues, the searchers open approximately 6 different boxes. โ The planner opens the 8 best boxes, with 1 searcher for each box. โ Blind does not use the clues. It opens 8 random boxes.
Watch what changes. โ Consensus piles all eight searchers on the plurality box. โ The market spreads over the top two or three. โ Private clue-followers scatter across six distinct boxes on average. โ The planner opens the eight best boxes, one searcher each โ and โ blind ignores the clues entirely, opening eight boxes at random.
Quality up, discovery down
We measure each protocol in 2 ways. Q is the average accuracy of the actions โ the quality of the usual guess. G is the probability that the group finds the jackpot. Shared information always increases Q. But it can decrease G:
Score each protocol two ways. Q is the average accuracy of the actions taken โ how good the typical guess is. G is the probability the group finds the jackpot. Sharing information strictly improves Q. It can strictly destroy G:
The 2 sequences are exact opposites: consensus has the best average action and the worst group result. The dashed line shows the blind control at 0.500. Put the pointer on a bar to see the exact values.The two orderings are exact inverses: consensus has the best average action and the worst group outcome. Dashed line: the blind control at 0.500. Hover any bar for exact values.
View as table
The difference between Q and G is redundancy. The identity is NยทQ โ G = E[(Kโ1)โบ] โ the expected number of successful actions that hit the same box again. Consensus gets its high action quality because all 8 actions are the same action. On average, 2.68 of its hits open a box that is already open.
The gap between the columns is not mysterious โ it is redundancy, exactly: NยทQ โ G = E[(Kโ1)โบ], the expected number of redundant successful actions. Consensus maximizes per-action quality by making all eight actions the same action: 2.68 of its expected hits land on a box that was already open.
Discovery uses a budget, not 1 answerDiscovery is a budget, not an answer
Keep the collected ranking. But give it a budget of L different actions, not 1 answer. Consensus is the point L = 1 on a curve. Move the budget:
Keep the pooled ranking, but give it a budget of L distinct actions instead of one answer. Consensus is just the L = 1 endpoint of a frontier. Drag the budget:
Budgets L = 1 to 8. The dashed line shows private search, 0.8322. 7 planned actions are better than 8 private actions. The recovery budget is L* = 7.Pooled frontier, L = 1โฆ8. Dashed line: decentralized private search, 0.8322. Seven coordinated actions already beat eight decentralized ones โ the recovery budget is L* = 7.
View as table
More conformity always decreases discoveryThe conformity dial only goes down
A possible compromise: a of the 8 searchers use the committee box. The other searchers keep their own clues. This never helps. The committee box is always in the set of private proposals. Thus each adopter adds no new box and removes their own box. Move the dial:
The natural compromise โ let only a of the eight defer to the committee's pick while the rest keep their own clue โ never helps. The plurality winner is already in the private proposal set, so each adopter adds nothing and deletes their own proposal. Drag the dial:
The exact G(a) always decreases. The first adopter costs 4.1 points. To get dispersion, use information, prices, or a plan. Small quantities of conformity do not give dispersion.Exact G(a), strictly decreasing โ the first adopter already costs 4.1 points. Dispersion must be informed, priced, or assigned; it cannot be obtained by dosing conformity.
View as table
Can self-interest correct it?Can selfishness fix it?
Partially. Remove the planner. The collected reports stay public. 8 searchers divide a prize of 1 between the searchers that open the jackpot box. A crowd makes each part of the prize smaller. Thus a searcher does not want to stand where all the other searchers stand. The equilibrium water-fills the posterior โ each used box gives the same expected payoff ฮป:
Partially. Remove the planner, keep the pooled reports public, and let eight self-interested searchers split a unit prize among whoever opens the jackpot. Crowding dilutes the prize, so nobody wants to stand where everyone stands. The equilibrium water-fills the posterior โ every occupied box offers the same expected payoff ฮป, like water finding its level:
Top: the posterior after 1 set of clues, with the equilibrium water level ฮป. The equilibrium uses the boxes that are higher than the line. A taller bar gets a larger share. Bottom: the equilibrium shares, with a random sample of 8 searchers.Top: the pooled posterior after one draw of clues, with the equilibrium water level ฮป. Boxes above the line get searched; the deeper a bar sits under water, the larger its share. Bottom: equilibrium shares, with eight searchers dropped at random from them.
The average result of the market is 0.5991 โ more than consensus, less than each diversified protocol. The market usually opens only 2.7 different boxes. Self-interest gives some dispersion, but not enough.
Averaged over worlds the market reaches 0.5991 โ above consensus, below everything diversified. It typically covers only 2.7 distinct boxes: profit-seeking buys some dispersion, not enough.
The loss has a limitThe damage is bounded
The exact mixed price of anarchy is 2 โ 1/N. No equilibrium loses more than this factor against the best portfolio. The proof is a smoothness argument. Thus the same limit applies to no-regret learning agents. These agents get the guarantee without a calculation of the equilibrium. The exact mixed price of anarchy is 2 โ 1/N: no equilibrium loses more than that factor against the optimal portfolio. And because the proof is a smoothness argument, the same guarantee covers no-regret learners โ adaptive agents inherit it without ever computing an equilibrium.One pay rule removes the gapOne reward rule closes the gap
Pay the prize only when exactly 1 searcher opens the jackpot box โ pay for the rescue, not for the win. Then each pure Nash equilibrium is the planner's portfolio. Payment for marginal contribution makes self-interested search the best possible. Pay the prize only when exactly one searcher covers the target โ reward the rescue, not the win โ and every pure Nash equilibrium becomes the planner's portfolio. Marginal-contribution pay makes selfish search first-best.When all searchers read the same sourceWhen everyone reads the same memo
Real clues are frequently correlated. With probability c, a searcher copies a common source and does not get an independent signal. Then the 8 reports contain fewer independent channels. Independence made private search strong. Move c to see the effect:
Real clues are correlated. With probability c each searcher copies a common source instead of drawing an independent signal, and eight nominal reports collapse into fewer effective channels. Independence is what made private search strong โ drag c and watch it fall:
The curves show the exact values. The market decreases slowly. Private search decreases quickly. The curves cross 1 time, at c* = 0.7885. On a grid of 72 combinations (M, N, p), the difference changes its sign 1 time maximum. Put the pointer on the chart to see the exact values.Exact canonical curves. The dispersing market degrades slowly; private report-following collapses. They cross exactly once, at c* = 0.7885 โ and on a 72-combination (M, N, p) grid, the sign never changes more than once. Hover for exact values.
View as table
The paradox stays in large worldsIt survives at scale
This is not only a property of 16 boxes. Increase the numbers of boxes and searchers together, and keep the evidence sparse. Then all the values go to their limits. Consensus goes to zero, because one large incorrect signal wins the vote against the truth:
Not a sixteen-box curiosity. Grow boxes and searchers together while evidence stays sparse, and the whole ladder converges โ consensus to zero, because an extreme false positive eventually outvotes the truth:
View as table
The opposite case shows the domain of the paradox. Keep the clue accuracy constant while the world becomes larger. The evidence becomes abundant, and consensus becomes correct โ it goes to 1. The paradox is a property of searches with sparse evidence: new science, pre-seed investment, new hypotheses. In these areas, organizations use consensus the most.
The flip side sharpens the domain. Hold clue accuracy fixed while the world grows โ evidence becomes abundant โ and consensus becomes consistent, rising toward one. The paradox is a property of sparse-evidence search: frontier science, pre-seed investing, novel hypotheses. Exactly the places where institutions lean hardest on consensus.
Share the evidence.
Diversify the actions.
To collect information and to collect actions are 2 different operations. The first gives almost free improvement. The second changes a portfolio into 1 repeated choice, without a warning. Committees, prize rules, and feeds that end in 1 answer select the consensus row of this table:
Pooling information and pooling actions are different operations. The first is nearly free improvement; the second silently converts a portfolio into a point estimate. Committees, prize rules, and feeds that end in one answer are choosing the left column of this table:
The benchmark in 1 table. The information is the same; the action protocols are different. All values are exact.The benchmark in one table โ same information, different action protocols. All values exact.
Reproduce everything
One script calculates each number on this page again. It uses exact enumeration (490,314 count vectors in 67 classes). It solves the Poisson water-filling limit to 12 digits. It makes sure that each claim is correct. It ends with All v8.6 checks passed.
One script re-derives every number above by exact enumeration (490,314 count vectors reduced to 67 classes), solves the Poisson water-filling limit to 12 digits, and asserts each claim. It ends with All v8.6 checks passed.
git clone https://github.com/yoheinakajima/shared-discovery-paradox pip install -r requirements.txt python verify_shared_discovery_v8_6.py