A shared pretrain. Three canonical cooldowns. The SFT, reinforcement learning, long-context and weight-merging branches that followed. Every lineage points back to its experiment record.
Dates use checkpoint labels or dated source records; click a node to see its date basis and full evidence. Columns follow chronology rather than a proportional time scale. Undated nodes appear at the end.
An arrow means initialization, or a labeled progression between retained checkpoints of the same training-state chain. Merge checkpoints draw one arrow from each pinned parent. The three cooldowns branch from the continuous pretrain; they are not successive post-training stages. Export repairs and retained checkpoints are labeled separately from new optimization.
Dashed links preserve an inference or an unresolved artifact identity. Canonical GitHub issues take priority; Discord discussion anchors are used for releases without a dedicated issue. Model cards and pinned comments supply finer checkpoint evidence.
The early 32K SFT estimates use the reported analytic 2.117×10¹⁰ FLOPs/token, including attention and the output head. Other supervised training uses 6 × 2B active parameters × trained token positions. The 6ND stage proxies exclude inherited pretraining, attention, output projection, optimizer overhead, retries and evaluation. The detail panel states each estimate’s scope. Packed positions can include padding.
Long-context attention can be a material extra cost. RL rollout generation, policy updates and reference passes require separate accounting. An unknown FLOP cost remains unknown; step counts or GPU-hours are shown where documented. Do not add incompatible proxies as an audited total.
Includes established public checkpoints and relevant internal or failed training branches found in the refreshed GitHub/Discord corpus and Hugging Face cards. A failed trial is a run node, not a released model. Same-run retained checkpoints can share a node with individual links.
The canonical 2.7T / 5.7T / 10T labels follow the post-training records. Older pretraining posts use ~2.3T / ~6.55T for the first two; native checkpoint IDs are the stronger identity. September 11 SFT uses the 4× LCE branch; September 16/17/20/21 use 8×. September 14’s exact LCE variant remains unresolved.
Terms: SFT = supervised fine-tuning; RLVR = reinforcement learning with verifiable rewards; LCE = long-context extension; QK = attention-logit gain; EOT = end of turn; FTPO = final-token preference optimization. OTA2 = OpenThoughts-Agent traces with IssueTasks removed; RST = Recursive Task Synthesis. TB2 / TB2.1 / TBLite = Terminal-Bench variants. Organization color identifies experiment ownership, independent of the upload namespace. The 2.7T branch belongs to open-athena, including checkpoints hosted under penfever or laion.