Every work cited in this vault, the implementation notes and the documentation’s tutorials, in one list. Generated from
docs/src/refs.bibbydocs/site/bib2vault.py; edit the.bibfile, not this note. The documentation renders the same file on its References page.
Sources: the citations of the individual notes, with metadata checked against arXiv and Crossref.
Theory (CT-ML wiki): Home
The code in backticks is the BibTeX key, for citing in the documentation as [key](@cite).
Notes link here through the “Bibliography:” line under their sources, generated by
docs/site/link_sources.py; each entry lists the notes that cite it.
Papers and books (24)
- Marianne Arriola, Aaron Gokaslan, Justin T. Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo & Volodymyr Kuleshov (2025) Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models. arXiv:2503.09573 (cited in Diffusion Language Models, How a Diffusion Language Model Generates)
arriola2025block - Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow & Rianne van den Berg (2021) Structured Denoising Diffusion Models in Discrete State-Spaces. Advances in Neural Information Processing Systems (NeurIPS). arXiv:2107.03006 (cited in Assembling a Language Model from Implicit Learners, Diffusion Language Models, tokens)
austin2021d3pm - Boyuan Chen, Diego Marti Monso, Yilun Du, Max Simchowitz, Russ Tedrake & Vincent Sitzmann (2024) Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion. arXiv:2407.01392 (cited in Incremental Next-Token Prediction)
chen2024forcing - Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H. Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, Curtis Hawthorne, Rémi Leblond, Will Grathwohl & Jonas Adler (2022) Continuous Diffusion for Categorical Data. arXiv:2211.15089 (cited in Diffusion Language Models, Embeddings in a Diffusion Language Model)
dieleman2022cdcd - Yilun Du, Conor Durkan, Robin Strudel, Joshua B. Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet & Will Grathwohl (2023) Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC. International Conference on Machine Learning (ICML). arXiv:2302.11552
du2023reduce - Zhenghan Fang, Mateo Díaz, Sam Buchanan & Jeremias Sulam (2025) Beyond Scores: Proximal Diffusion Models. arXiv:2507.08956 (cited in Incremental Next-Token Prediction)
fang2025proxdm - William Fedus, Barret Zoph & Noam Shazeer (2021) Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. arXiv:2101.03961 (cited in Factor Graphs against Mixtures of Experts and Agent Chains)
fedus2021switch - Ishaan Gulrajani & Tatsunori B. Hashimoto (2023) Likelihood-Based Diffusion Language Models. arXiv:2305.18619 (cited in Diffusion Language Models, Embeddings in a Diffusion Language Model)
gulrajani2023plaid - Geoffrey E. Hinton (2002) Training Products of Experts by Minimizing Contrastive Divergence. Neural Computation, vol. 14, no. 8, pp. 1771–1800. doi:10.1162/089976602760128018 (cited in Assembling a Language Model from Implicit Learners, Factor Graphs against Mixtures of Experts and Agent Chains, sequence)
hinton2002poe - Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan & Geoffrey E. Hinton (1991) Adaptive Mixtures of Local Experts. Neural Computation, vol. 3, no. 1, pp. 79–87. doi:10.1162/neco.1991.3.1.79 (cited in Factor Graphs against Mixtures of Experts and Agent Chains)
jacobs1991experts - Michael Kaess, Hordur Johannsson, Richard Roberts, Viorela Ila, John J. Leonard & Frank Dellaert (2012) iSAM2: Incremental Smoothing and Mapping Using the Bayes Tree. The International Journal of Robotics Research, vol. 31, no. 2, pp. 216–235. doi:10.1177/0278364911430419 (cited in Incremental Next-Token Prediction)
kaess2012isam2 - Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang & Tatsunori B. Hashimoto (2022) Diffusion-LM Improves Controllable Text Generation. Advances in Neural Information Processing Systems (NeurIPS). arXiv:2205.14217 (cited in Assembling a Language Model from Implicit Learners, Diffusion Language Models, Embeddings in a Diffusion Language Model, How a Diffusion Language Model Generates, tokens)
li2022diffusionlm - Nan Liu, Shuang Li, Yilun Du, Antonio Torralba & Joshua B. Tenenbaum (2022) Compositional Visual Generation with Composable Diffusion Models. European Conference on Computer Vision (ECCV). arXiv:2206.01714 (cited in Assembling a Language Model from Implicit Learners)
liu2022composable - Aaron Lou, Chenlin Meng & Stefano Ermon (2024) Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution. International Conference on Machine Learning (ICML). arXiv:2310.16834 (cited in Assembling a Language Model from Implicit Learners, Diffusion Language Models)
lou2024sedd - Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen & Chongxuan Li (2025) Large Language Diffusion Models. arXiv:2502.09992 (cited in Assembling a Language Model from Implicit Learners, Diffusion Language Models, How a Diffusion Language Model Generates)
nie2025llada - Artidoro Pagnoni, Ram Pasunuru, Pedro Rodriguez, John Nguyen, Benjamin Muller, Margaret Li, Chunting Zhou, Lili Yu, Jason Weston, Luke Zettlemoyer, Gargi Ghosh, Mike Lewis, Ari Holtzman & Srinivasan Iyer (2024) Byte Latent Transformer: Patches Scale Better Than Tokens. arXiv:2412.09871 (cited in Embeddings in a Diffusion Language Model)
pagnoni2024blt - David Ruhe, Jonathan Heek, Tim Salimans & Emiel Hoogeboom (2024) Rolling Diffusion Models. arXiv:2402.09470 (cited in Incremental Next-Token Prediction)
ruhe2024rolling - Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush & Volodymyr Kuleshov (2024) Simple and Effective Masked Diffusion Language Models. Advances in Neural Information Processing Systems (NeurIPS). arXiv:2406.07524 (cited in Assembling a Language Model from Implicit Learners, Diffusion Language Models, How a Diffusion Language Model Generates)
sahoo2024mdlm - Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton & Jeff Dean (2017) Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. arXiv:1701.06538 (cited in Factor Graphs against Mixtures of Experts and Agent Chains)
shazeer2017moe - Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet & Michalis K. Titsias (2024) Simplified and Generalized Masked Diffusion for Discrete Data. arXiv:2406.04329 (cited in Diffusion Language Models)
shi2024md4 - Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, Zhongyu Wei, Jian Guo, Nan Duan & Weizhu Chen (2023) AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation. arXiv:2305.09515 (cited in Incremental Next-Token Prediction)
wu2023ardiffusion - Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts & Colin Raffel (2021) ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models. arXiv:2105.13626 (cited in Embeddings in a Diffusion Language Model)
xue2021byt5 - Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan & Yuan Cao (2022) ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629 (cited in Factor Graphs against Mixtures of Experts and Agent Chains)
yao2022react - Lili Yu, Dániel Simig, Colin Flaherty, Armen Aghajanyan, Luke Zettlemoyer & Mike Lewis (2023) MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers. arXiv:2305.07185 (cited in Embeddings in a Diffusion Language Model)
yu2023megabyte
Software (0)
Related: Map of Content, Start Here