https://teleported.in/tags/papers/ Recent content in Papers on Teleported.in Hugo -- 0.151.0 en-us 2025 Anand Saha. Sat, 20 Dec 2025 23:27:27 -0400 https://teleported.in/blog/2025/12/landmark-llm-papers/ Sat, 20 Dec 2025 23:27:27 -0400 https://teleported.in/blog/2025/12/landmark-llm-papers/ <h3 id="introduction">Introduction</h3> <p>A list of curated landmark papers in the field of LLMs.</p> <h3 id="foundational">Foundational</h3> <ul> <li><a href="https://arxiv.org/abs/1301.3781">Efficient Estimation of Word Representations in Vector Space (Word2Vec) (2013)</a></li> <li><a href="https://nlp.stanford.edu/pubs/glove.pdf">GloVe: Global Vectors for Word Representation (2014)</a></li> <li><a href="https://arxiv.org/abs/1409.0473">Neural Machine Translation by Jointly Learning to Align and Translate (2014)</a> <ul> <li>Introduced the concept of attention</li> </ul> </li> </ul> <h3 id="transformer">Transformer</h3> <ul> <li><a href="https://arxiv.org/abs/1706.03762">Attention Is All You Need (2017)</a> <ul> <li>Introduced the Transformer architecture</li> </ul> </li> <li><a href="https://arxiv.org/abs/1803.02155">Self-Attention with Relative Position Representations (2018)</a></li> <li><a href="https://arxiv.org/abs/1810.04805">BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018)</a></li> <li><a href="https://arxiv.org/abs/2104.09813">Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation (2021)</a></li> <li><a href="https://arxiv.org/abs/2104.09864">RoFormer: Enhanced Transformer with Rotary Position Embedding (2021)</a></li> </ul> <h3 id="large-language-models">Large Language Models</h3> <ul> <li><a href="https://arxiv.org/abs/1801.06146">Universal Language Model Fine-tuning for Text Classification (ULMFiT)(2018)</a></li> <li><a href="https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf">Improving Language Understanding by Generative Pre-Training (GPT-1)(2018)</a></li> <li><a href="https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf">Language Models are Unsupervised Multitask Learners (GPT-2)(2019)</a></li> <li><a href="https://arxiv.org/abs/2005.14165">Language Models are Few-Shot Learners (GPT-3) (2020)</a></li> <li><a href="https://arxiv.org/abs/2101.03961">What Can Transformers Learn In-Context? A Case Study of Simple Function Classes (2021)</a></li> <li><a href="https://arxiv.org/abs/2303.08774">GPT-4 Technical Report (2023)</a></li> </ul> <h3 id="alignment">Alignment</h3> <ul> <li><a href="https://arxiv.org/abs/1706.03741">Deep reinforcement learning from human preferences (2017)</a></li> <li><a href="https://arxiv.org/abs/2203.02155">Training language models to follow instructions with human feedback (2022)</a></li> <li><a href="https://arxiv.org/abs/2203.02155">Constitutional AI: Harmlessness from AI Feedback (2022)</a></li> </ul> <h3 id="scaling-laws-emergence">Scaling Laws, Emergence</h3> <ul> <li><a href="https://arxiv.org/abs/2001.08361">Scaling Laws for Neural Language Models (2020)</a></li> <li><a href="https://arxiv.org/abs/2203.05556">Training Compute-Optimal Large Language Models (2022)</a></li> <li><a href="https://arxiv.org/abs/2206.07682">Emergent Abilities of Large Language Models (2022)</a></li> </ul> <h3 id="prompt--context-engineering">Prompt / Context Engineering</h3> <ul> <li><a href="https://arxiv.org/abs/2109.08203">Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (2021)</a></li> </ul> <h3 id="efficient-transformers">Efficient Transformers</h3> <ul> <li><a href="https://arxiv.org/abs/1901.02860">Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context (2019)</a></li> <li><a href="https://arxiv.org/abs/1904.00969">Reformer: The Efficient Transformer (2019)</a></li> <li><a href="https://arxiv.org/abs/2004.05150">Longformer: The Long-Document Transformer (2020)</a></li> <li><a href="https://arxiv.org/abs/2004.05718">Generating Long Sequences with Sparse Transformers (2020)</a></li> <li><a href="https://arxiv.org/abs/2007.04509">Big Bird: Transformers for Longer Sequences (2020)</a></li> <li><a href="https://arxiv.org/abs/2205.14135">FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022)</a></li> </ul> <h3 id="survey-papers">Survey Papers</h3> <ul> <li><a href="https://arxiv.org/abs/2106.04554">A Survey of Transformers (2022)</a></li> <li><a href="https://arxiv.org/abs/2009.06732">Efficient Transformers: A Survey (2020)</a></li> <li><a href="https://arxiv.org/abs/2303.18223">A Survey of Large Language Models (2023)</a></li> <li><a href="https://arxiv.org/abs/2108.07258">On the Opportunities and Risks of Foundation Models (2022)</a></li> <li><a href="https://arxiv.org/abs/2107.13586">Pre-train, Prompt, and Predict: A Survey of Prompting Methods in NLP (2021)</a></li> <li><a href="https://arxiv.org/abs/2508.09834">Speed Always Wins: A Survey on Efficient Architectures for Large Language Models (2025)</a></li> </ul>