Linux · Notes
Foundational papers in AI and ML
Listing of important papers in AI and ML.
Manoj Kumar R
September 13, 2026 · 5 min read
Foundational LLM & ML Systems Papers
Core Architecture & Pretraining
- Attention Is All You Need (Transformers) — introduced the Transformer, replacing recurrence with self-attention.
https://arxiv.org/abs/1706.03762 - BERT — bidirectional pretraining via masked language modeling for language understanding.
https://arxiv.org/abs/1810.04805 - GPT (Improving Language Understanding by Generative Pre-Training) — showed generative pretraining + fine-tuning works well for NLP tasks. (Hosted by OpenAI, not on arXiv.)
https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf - GPT-2 (Scaling + Zero-Shot) — scaling GPT shows strong zero-shot task transfer. (Hosted by OpenAI.)
https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf - Scaling Laws for Neural Language Models — power-law relationships between loss, model size, data, and compute.
https://arxiv.org/abs/2001.08361 - GPT-3 (Few-Shot Learning) — 175B model shows strong in-context few-shot learning.
https://arxiv.org/abs/2005.14165 - The Pile — 800GB curated open-source dataset for large-scale LM training.
https://arxiv.org/abs/2101.00027 - Chinchilla (Compute-Optimal Scaling) — for a given compute budget, smaller models on more data outperform larger undertrained ones.
https://arxiv.org/abs/2203.15556 - PaLM — 540B dense model; strong reasoning and multilingual results via Pathways.
https://arxiv.org/abs/2204.02311 - OPT — Meta's suite of open, replicable decoder-only LLMs up to 175B.
https://arxiv.org/abs/2205.01068 - BLOOM — open, multilingual 176B model built by the BigScience collaboration.
https://arxiv.org/abs/2211.05100 - LLaMA — Meta's efficient open foundation models trained on public data only.
https://arxiv.org/abs/2302.13971
Pretraining Lineage (additional)
- ELMo — deep contextualized word representations from bidirectional LSTMs; precursor to BERT-style contextual embeddings.
https://arxiv.org/abs/1802.05365 - RoBERTa — BERT retrained longer, on more data, without next-sentence prediction; showed BERT was undertrained.
https://arxiv.org/abs/1907.11692 - T5 — reframes every NLP task as text-to-text, with a systematic study of transfer learning setups.
https://arxiv.org/abs/1910.10683 - Gopher — DeepMind's 280B dense model and accompanying scaling analysis, contemporaneous with Chinchilla.
https://arxiv.org/abs/2112.11446
Alignment & Instruction Tuning
- InstructGPT (RLHF) — fine-tuning GPT-3 with human feedback to follow instructions.
https://arxiv.org/abs/2203.02155 - Constitutional AI — training models to self-critique/revise against a set of principles, reducing reliance on human harm labels.
https://arxiv.org/abs/2212.08073 - Self-Instruct — bootstrapping instruction-tuning data from a model's own generations.
https://arxiv.org/abs/2212.10560 - Direct Preference Optimization (DPO) — reframes RLHF as a simple classification-style loss, no reward model/RL needed.
https://arxiv.org/abs/2305.18290 - ORPO — combines SFT and preference alignment into a single odds-ratio-based objective.
https://arxiv.org/abs/2403.07691 - GRPO — introduced in the DeepSeekMath paper; critic-free RL using group-relative baselines.
https://arxiv.org/abs/2402.03300 - Deep RL from Human Preferences — the original RLHF paper; learns a reward model from pairwise human comparisons.
https://arxiv.org/abs/1706.03741 - Learning to Summarize from Human Feedback — RLHF applied to summarization, direct precursor to InstructGPT.
https://arxiv.org/abs/2009.01325 - WebGPT — RLHF-trained model that browses the web and cites sources when answering questions.
https://arxiv.org/abs/2112.09332 - KTO (Kahneman-Tversky Optimization) — aligns models using only a binary desirable/undesirable signal, no paired preferences, motivated by prospect theory.
https://arxiv.org/abs/2402.01306 - SimPO — reference-model-free simplification of DPO using length-normalized sequence likelihood as the implicit reward.
https://arxiv.org/abs/2405.14734 - LIMA ("Less Is More for Alignment") — argues most of a model's capability comes from pretraining, and a small, high-quality SFT set is enough for alignment.
https://arxiv.org/abs/2305.11206 - DeepSeek-R1 — large-scale RL (built on GRPO) elicits strong reasoning without heavy SFT; major 2025 reasoning-model release.
https://arxiv.org/abs/2501.12948
Efficient Fine-Tuning
- LoRA — low-rank adapters for efficient fine-tuning of large models.
https://arxiv.org/abs/2106.09685 - QLoRA — LoRA fine-tuning on top of 4-bit quantized frozen weights.
https://arxiv.org/abs/2305.14314
Efficient Attention & Inference
- FlashAttention — IO-aware exact attention algorithm, big speed/memory wins.
https://arxiv.org/abs/2205.14135 - FlashAttention-2 — further parallelism/work-partitioning improvements.
https://arxiv.org/abs/2307.08691 - RoPE (Rotary Position Embeddings) — encodes position by rotating query/key vectors.
https://arxiv.org/abs/2104.09864 - ALiBi — linear position bias added to attention scores, enabling length extrapolation.
https://arxiv.org/abs/2108.12409 - Multi-Query Attention (MQA) — shares key/value heads across query heads to cut KV-cache size.
https://arxiv.org/abs/1911.02150 - Grouped-Query Attention (GQA) — interpolates between MHA and MQA for a quality/speed tradeoff.
https://arxiv.org/abs/2305.13245 - Speculative Decoding — drafts tokens with a small model, verifies with the large one in parallel.
https://arxiv.org/abs/2211.17192 - PagedAttention (vLLM) — OS-style paged memory management for the KV cache, enabling high-throughput serving.
https://arxiv.org/abs/2309.06180 - KV Cache Compression & Optimization — this is a broad research area rather than one paper; a widely-cited entry point is H2O (Heavy-Hitter Oracle), which evicts low-importance KV cache entries.
https://arxiv.org/abs/2306.14048
Alternative Efficient Architectures
- Mamba — selective state-space model achieving Transformer-quality results with linear-time sequence modeling.
https://arxiv.org/abs/2312.00752
Prompting & Agentic Reasoning
Retrieval-Augmented Generation (RAG) — combines a retriever with a generator conditioned on retrieved passages.
https://arxiv.org/abs/2005.11401Chain-of-Thought Prompting — eliciting step-by-step reasoning via prompting improves multi-step task performance.
https://arxiv.org/abs/2201.11903Tree of Thoughts — generalizes CoT to search over a tree of intermediate reasoning steps.
https://arxiv.org/abs/2305.10601ReAct — interleaves reasoning traces with actions (e.g., tool/API calls).
https://arxiv.org/abs/2210.03629Test-Time Scaling — a widely-cited recent formalization is "Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Parameters."
https://arxiv.org/abs/2408.03314Toolformer — model teaches itself which APIs to call, when, and how, via self-supervised annotation.
https://arxiv.org/abs/2302.04761Reflexion — agents that verbally self-reflect on failures and retry, as a lightweight alternative to weight updates.
https://arxiv.org/abs/2303.11366Self-Refine — iterative self-critique-and-revise loop with the same model acting as generator and critic, no extra training.
https://arxiv.org/abs/2303.17651
Mixture of Experts
- Mixture of Experts (Switch Transformer) — simplifies MoE routing to top-1 expert for efficient trillion-parameter scaling.
https://arxiv.org/abs/2101.03961 - DeepSeekMoE — fine-grained expert segmentation + shared experts for better specialization at lower cost.
https://arxiv.org/abs/2401.06066
Optimizers
- Muon Optimizer — matrix-orthogonalization-based optimizer for hidden layers, originally released as a technical write-up rather than a formal paper; also see the Moonshot AI paper validating it at scale.
https://kellerjordan.github.io/posts/muon/ (original write-up)
https://arxiv.org/abs/2502.16982 ("Muon is Scalable for LLM Training")
Training Systems (how you actually train at this scale)
- Megatron-LM — tensor and pipeline parallelism techniques for training multi-billion-parameter Transformers.
https://arxiv.org/abs/1909.08053 - ZeRO (DeepSpeed) — memory-optimized data parallelism that eliminates redundancy across GPUs, enabling much larger models per device.
https://arxiv.org/abs/1910.02054
Emergent Behavior & Evaluation
- Emergent Abilities of Large Language Models — documents capabilities that appear discontinuously at scale (a claim later contested by follow-up work on measurement artifacts).
https://arxiv.org/abs/2206.07682 - Discovering Language Model Behaviors with Model-Written Evaluations — Anthropic's method for generating large evaluation datasets using LLMs themselves.
https://arxiv.org/abs/2212.09251
Note: "KV Cache Compression & Optimization" and "Test-Time Scaling" are research areas rather than single canonical papers — the links above are commonly-cited representative entries, not the sole reference. The "Emergent Abilities" claim has also been disputed by subsequent work arguing the effect is partly a function of metric choice.
