Llm

Introduction A list of curated landmark papers in the field of LLMs. Foundational Efficient Estimation of Word Representations in Vector Space (Word2Vec) (2013) GloVe: Global Vectors for Word Representation (2014) Neural Machine Translation by Jointly Learning to Align and Translate (2014) Introduced the concept of attention Transformer Attention Is All You Need (2017) Introduced the Transformer architecture Self-Attention with Relative Position Representations (2018) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018) Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation (2021) RoFormer: Enhanced Transformer with Rotary Position Embedding (2021) Large Language Models Universal Language Model Fine-tuning for Text Classification (ULMFiT)(2018) Improving Language Understanding by Generative Pre-Training (GPT-1)(2018) Language Models are Unsupervised Multitask Learners (GPT-2)(2019) Language Models are Few-Shot Learners (GPT-3) (2020) What Can Transformers Learn In-Context? A Case Study of Simple Function Classes (2021) GPT-4 Technical Report (2023) Alignment Deep reinforcement learning from human preferences (2017) Training language models to follow instructions with human feedback (2022) Constitutional AI: Harmlessness from AI Feedback (2022) Scaling Laws, Emergence Scaling Laws for Neural Language Models (2020) Training Compute-Optimal Large Language Models (2022) Emergent Abilities of Large Language Models (2022) Prompt / Context Engineering Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (2021) Efficient Transformers Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context (2019) Reformer: The Efficient Transformer (2019) Longformer: The Long-Document Transformer (2020) Generating Long Sequences with Sparse Transformers (2020) Big Bird: Transformers for Longer Sequences (2020) FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022) Survey Papers A Survey of Transformers (2022) Efficient Transformers: A Survey (2020) A Survey of Large Language Models (2023) On the Opportunities and Risks of Foundation Models (2022) Pre-train, Prompt, and Predict: A Survey of Prompting Methods in NLP (2021) Speed Always Wins: A Survey on Efficient Architectures for Large Language Models (2025)