PUBLIC RESEARCH VIEW · FIVE DATASETS ACROSS TWO CANDIDATE BANKS

Planned and direct,
five clear views.

Compare GPT-OSS and DeepSeek V4 Pro rewards across planned generations, then inspect a new DeepSeek-scored collection where Qwen emits the final response directly—with no plan.

Qwen3.8-27B candidatesGPT-OSS + DeepSeek judgesplanned ↔ direct response
Loading the audit index…
What is included, and why?

Five selectable variants: four planned variants cross two judges with inline or filesystem generation. The fifth uses inline context with no plan and is scored by DeepSeek V4 Pro.

Candidate-bank boundary: the direct-response variant is aligned to the same target identities, but its 64 responses per target were generated independently rather than paired candidate-by-candidate with the planned variants.

Alignment: training rows are batch-aligned to a multiple of 16. Excluded zero-candidate and alignment target identities are pinned in provenance rather than hidden.

Best@k: exact expected maximum for k candidates sampled without replacement from one target's scored panel. Lift subtracts Best@1, which is the panel mean.

Lexical diversity: token-set Jaccard is descriptive only. Lower overlap means different wording; it is not a semantic-diversity judgment.

DATASET VIEW

Choose one of five dataset variants

Counts below describe the complete published datasets; interactive charts and candidates use the same target-aligned audit cohort.

AUDIT COHORT

Reward distribution

MEAN ACROSS AUDIT TARGETS

Best@k lift