TreasureBench
How performance is measured
A measure is omitted when its required observables are absent. The benchmark preserves task vectors and family profiles; no composite score is active.
Scroll horizontally to see every column.
| Metric | Definition | Units | Required observables |
|---|---|---|---|
| discovery | Expected union discovery or registered coverage value. | probability-or-coverage | covered_outcomes |
| average-action-quality | Expected correct actions divided by actions. | probability | actions, target_outcomes |
| distinct-actions | Expected number of distinct actions. | actions | actions |
| weighted-coverage | Declared weighted union coverage. | coverage-units | coverage_sets, weights |
| redundant-hits | Expected successful hits beyond the first. | hits | target_hits |
| action-concentration | Declared concentration of action shares. | index | actions |
| source-concentration | Declared concentration of source exposure. | index | source_ids |
| effective-channels | Model-defined reciprocal source concentration. | channels | source_ids, channel_definition |
| protocol-loss | Registered frontier minus protocol discovery. | objective-units | frontier, discovery |
| recovery-budget | Smallest action budget recovering the comparator. | actions | budget_frontier, comparator |
| information-cost | Expected declared source-acquisition cost. | payoff-units | source_choices, costs |
| expected-actions | Expected number of actions used. | actions | actions |
| expected-rounds | Expected number of decision rounds. | rounds | rounds |
| private-payoff | Expected declared private payoff. | payoff-units | rewards, costs |
| social-net-value | Discovery value net of declared information and transfer costs. | payoff-units | discovery, information_cost, transfer_budget |
| transfer-budget | Expected external transfer budget. | payoff-units | transfers |
| truthfulness | Whether truthful reporting satisfies the registered incentive condition. | boolean-or-margin | reports, deviations |
| obedience | Whether recommended action satisfies the registered incentive condition. | boolean-or-margin | recommendations, deviations |
| strict-margin | Minimum truthful/obedient payoff advantage over registered deviations. | payoff-units | deviation_payoffs |
| attention-count | Number of agents using the public signal. | agents | attention_choices |
| public-signal-user-count | Number of agents whose action uses the public signal. | agents | actions, public_signal |
| first-use-gain | Discovery gain from the first public-signal user over private-only play. | probability | discovery_frontier |
| duplicate-use-loss | Discovery loss from duplicate public-signal use relative to one user. | probability | discovery_frontier |
| attention-wedge | Planner optimum minus the registered voluntary-attention equilibrium. | probability | planner_optimum, equilibrium_discovery |
| audience-size | Number of agents given access to the public signal. | agents | information_permissions |
| publicity-cost | Discovery loss relative to the optimal audience assignment. | probability | audience_frontier |
| conditional-attention-category | Registered conditional policy used on disagreement. | category | contingent_policy |
| planner-discovery | Exact deterministic minimum-team planner discovery. | probability | planner_frontier |
| expected-viable-candidates | Expected count of candidates meeting the declared threshold. | candidates | occupancies, threshold |
| zero-worst-equilibrium-games | Registered games whose worst weak pure equilibrium has zero discovery. | games | equilibrium_registry |
| pair-instability-games | Registered games with no pairwise-strict-stable weak pure equilibrium. | games | pair_deviations |
| tau-instability-games | Registered games with no exact-size-tau-strict-stable weak equilibrium. | games | tau_deviations |
| tied-mode-failure-games | Registered games where tied-mode uniform mixing is not an equilibrium. | games | mixed_action_payoffs |
| planner-strict-gain-rows | Exact rows where the dynamic planner strictly exceeds autonomous play. | rows | dynamic_values |
| visibility-joint-loss-cells | Fixed-budget cells where visibility lowers discovery and dispersion. | cells | visibility_control |
| stopping-action-savings-cells | Cells where stopping strictly lowers expected actions. | cells | fixed_and_stopping_values |
| planner-portfolio-rows | Mechanism rows attaining the minimum-team planner portfolio. | rows | mechanism_registry |
| pair-stable-rows | Rows passing the declared strict-member pair-deviation check. | rows | pair_deviations |
| equilibrium-multiplicity | Exact pure-equilibrium count or fixture-ordered count vector. | count-or-vector | equilibrium_registry |