HandsOn Large Language Models - Jay_Alammar
Chapter 1: An Introduction to Large Language Models
1. The "One-Line" Gist
This chapter serves as the foundational "Hello World" for the book, defining the modern era of AI by distinguishing between Generative (text-creating) and Representation (text-understanding) models while navigating the trade-offs between open-source and proprietary ecosystems.
2. Detailed Summary (The "Meat")
The authors argue that humanity reached a technological inflection point starting around 2012, accelerating dramatically with the release of GPT-2 and later ChatGPT. This shift moved AI from simple pattern recognition to systems capable of writing articles indistinguishable from human text.
The core logical distinction made in this chapter is the categorization of Large Language Models (LLMs) into two distinct buckets based on architecture and utility:
Representation Models (Encoder-Only): These models (like BERT) are "feature extraction machines." They do not generate text. Instead, they convert text into numerical representations (embeddings) to act as a backend for specific tasks like classification, clustering, and semantic search. The authors note that while less "flashy" than chatbots, these are critical for industrial applications.
Generative Models (Decoder-Only): These models (like GPT-3/4) are designed to generate text. They predict the next word in a sequence and are the engines behind modern chatbots and content creation tools.
The chapter also tackles the ambiguity of the term "Large Language Model." The authors argue that "Large" is an arbitrary descriptor. A smaller, highly optimized model might outperform a larger, older one. Therefore, definitions based purely on parameter count are flawed. Instead, the focus should be on capability.
Finally, the chapter outlines the ecosystem dilemma: Proprietary vs. Open Source.
Proprietary (e.g., OpenAI, Cohere): Easy to use via API, powerful, but acts as a "black box" with usage costs and privacy concerns.
Open Source (e.g., Hugging Face, Llama): Requires more hardware (GPUs) and technical know-how to set up, but offers privacy, customisation, and the freedom to "peek under the hood."
3. Key Concepts & Definitions
Representation Models: (Encoder-only architectures). Models designed to "understand" and classify text by converting it into vector embeddings, rather than generating new text. Context: Used for tasks like spam detection or document clustering.
Generative Models: (Decoder-only architectures). Models designed to predict the next token in a sequence to produce coherent text. Context: The technology behind ChatGPT.
The "Large" Paradox: The concept that the size of a model (parameters) does not strictly dictate its validity as a "language model." Context: A small, efficient model is still an LLM if it captures language effectively.
Prompting: The input (usually a list of dictionaries with roles like "user" and "content") given to a model to guide its output.
4. Golden Quotes (Verbatim)
"Humanity is at an inflection point. From 2012 onwards, developments in building AI systems (using deep neural networks) accelerated so that by the end of the decade, they yielded the first software system able to write articles indiscernible from those written by humans." 1
"Representation models mainly focus on representing language, for instance, by creating embeddings, and typically do not generate text. In contrast, generative models focus primarily on generating text and typically are not trained to generate embeddings." 2
"We generally prefer using open source models wherever we can. The freedom this gives to play around with options, explore the inner workings, and use the model locally arguably provides more benefits than using proprietary LLMs." 3
5. Stories & Case Studies
The Story of the "Chicken Joke": The chapter introduces code execution by asking a model to "Create a funny joke about chickens."
The Lesson: This simple "Hello World" example demonstrates the standard data structure for interacting with LLMs—a list of messages containing roles ("user") and content—demystifying the API interaction. 4
The Story of the Definition: The authors pose a hypothetical: If we build a model as capable as GPT-4 but without text generation (only classification), is it still an LLM?
The Lesson: This thought experiment proves that the term "Large Language Model" is often a misnomer. We should define models by their architecture (Encoder vs. Decoder) and function, not just their size or hype. 5
6. Actionable Takeaways (The "How-To")
Categorize Your Problem: Before writing code, decide if you need generation (chatbot, creative writing) or representation (search, classification). Do not use a sledgehammer (GPT-4) to crack a nut (text classification).
Embrace Open Source for Learning: While APIs are easier, force yourself to set up an open-source model (via Hugging Face) locally or in Google Colab to understand the mechanics of the technology.
Standardize Your Prompts: Adopt the
[{"role": "user", "content": "..."}]dictionary format immediately, as it is the standard protocol for modern chat models.
7. Critical Reflection
Counter-Argument: The authors strongly favor open source for learning ("arguably provides more benefits"). However, in a corporate production environment, the operational overhead (maintaining GPUs, scaling) of open source often outweighs the cost of proprietary APIs. The book’s "intuition-first" approach biases it slightly toward the educational value of open source over the pragmatic convenience of closed APIs.
Connection to Next Chapter: This chapter establishes what these models are. It ends by hinting that to understand how they process text, we must look at the inputs. This directly tees up Chapter 2, which covers Tokenization and Embeddings—the atoms of language modeling.
Chapter 2: Tokenization and Embeddings
1. The "One-Line" Gist
This chapter demystifies the fundamental translation layer of AI, explaining how machines convert raw human language into numerical vectors (embeddings) to grasp semantic meaning rather than just counting words.
2. Detailed Summary (The "Meat")
The authors begin by addressing the core problem of Language AI: computers cannot understand text, only numbers. The chapter contrasts two major approaches to solving this:
Bag-of-Words (The Old Way): This method counts word frequencies. If the word "bank" appears 5 times, it gets a 5. While "elegant," it has a fatal flaw: it ignores meaning and context. It treats a sentence as an "almost literal bag of words," losing the semantic richness of language1.
Embeddings (The Modern Way): The authors introduce Word2Vec (released in 2013) as the breakthrough that allowed computers to capture meaning. Instead of counting words, Word2Vec learns to represent them as dense vectors (lists of numbers). It does this by training a neural network on massive datasets (like Wikipedia) to predict which words appear next to each other. If two words often appear in similar contexts (like "king" and "queen"), their numerical vectors become similar2.
The chapter also details Tokenization, the preprocessing step that happens before embedding. It explains that models don't always see whole words. Complex words like "CAPITALIZATION" might be broken into smaller chunks (tokens) like ['CA', '##PI', '##TA', '##L', '##I', '##Z', '##AT', '##ION'] to help the model process rare or compound terms efficiently3.
3. Key Concepts & Definitions
Bag-of-Words: A traditional NLP technique that represents text by counting the frequency of each word, ignoring order and context. Context: Useful for simple tasks but fails to capture nuance4.
Embeddings: Vector representations (lists of numbers) where the distance between vectors correlates to the semantic similarity of the words. Context: The core "format" used by modern LLMs to understand language5.
Word2Vec: A specific algorithm using neural networks to learn word embeddings by predicting neighboring words in a sentence. Context: The "Hello World" of modern semantic representation6.
Tokenization: The process of breaking text down into smaller units (tokens), which can be words, parts of words, or characters. Context: Shown via the BERT tokenizer example7.
4. Golden Quotes (Verbatim)
"Bag-of-Words, although an elegant approach, has a flaw. It considers language to be nothing more than an almost literal bag of words and ignores the semantic nature, or meaning, of text." 8
"Embeddings are vector representations of data that attempt to capture its meaning." 9
"Using these neural networks, word2vec generates word embeddings by looking at which other words they tend to appear next to in a given sentence." 10
5. Stories & Case Studies
The Story of "CAPITALIZATION": The authors demonstrate the BERT tokenizer on the word "CAPITALIZATION."
The Details: Instead of treating it as one unknown word, the tokenizer aggressively splits it into
['CA', '##PI', '##TA', '##L', '##I', '##Z', '##AT', '##ION'].The Lesson: This illustrates "Subword Tokenization." It allows models to understand long or rare words by breaking them into familiar building blocks, rather than giving up11.
The Word2Vec "Neighbor" Game: The authors explain how Word2Vec learns by looking at pairs of words.
The Details: The model takes a word and tries to predict if another word is its "neighbor" in a sentence.
The Lesson: Meaning is defined by context. By learning what words hang out together, the model mathematically places synonyms close to each other in vector space12.
6. Actionable Takeaways (The "How-To")
Ditch the Count: For any application requiring understanding (sentiment, search, chatbots), stop using frequency counters (Bag-of-Words). Move immediately to Embeddings.
Inspect Your Tokens: When debugging model performance, look at how your text is being tokenized. If the model is breaking words into too many nonsensical chunks, you may need a different tokenizer.
Use Pre-trained Tokenizers: Do not build a tokenizer from scratch. Use established ones (like BERT's) which handle casing and special characters (like
[CLS]and[SEP]) automatically13.
7. Critical Reflection
The Counter-Argument: The chapter presents Word2Vec as a massive leap, which it was. However, Word2Vec generates static embeddings (the word "bank" has the same vector whether it's a river bank or a financial bank). Modern Transformers (covered in the next chapter) use contextual embeddings to solve this. The chapter hints at this limitation by mentioning that simple embeddings make it "difficult to deal with longer sentences"14.
Connection to Next Chapter: The chapter concludes by noting that while embeddings capture word meaning, we need a way to connect these words dynamically over long distances. This sets the stage for Chapter 3, which introduces Attention and the Transformer architecture15.
Chapter 3: The Transformer Architecture
1. The "One-Line" Gist
This chapter opens the "black box" of the LLM, revealing the Transformer architecture—specifically the interplay between Attention mechanisms and the Feed-Forward networks—that allows models to process vast amounts of text and predict the next token with eerie accuracy.
2. Detailed Summary (The "Meat")
The authors peel back the layers of the model, moving from the input to the final prediction. The logical flow of a Transformer is described as a pipeline:
The Forward Pass: It starts with the Tokenizer (converting text to IDs), flows into a Stack of Transformer Blocks (the neural network that processes meaning), and ends at the LM Head (Language Modeling Head). The LM Head's sole job is to translate the massive amount of processing done by the blocks into probability scores for the next token1.
The Attention Mechanism: The core innovation. The authors explain that attention allows the model to "look back" at previous tokens to derive context. However, they highlight a critical bottleneck: calculation costs. As sequences get longer, calculating attention becomes the "most computationally expensive part of the process"2.
Evolution & Optimisation: The chapter doesn't just stop at the original 2017 Transformer. It covers modern improvements like Sparse Attention (limiting how far back a model looks to save compute) and RoPE (Rotary Positional Embeddings), which are essential for the performance of modern models like Llama 23.
3. Key Concepts & Definitions
LM Head (Language Modeling Head): The final layer of the model that converts the hidden states from the Transformer blocks into actual word probabilities. Context: It’s the "translator" that turns math back into language4.
Sparse Attention: An optimization technique where the model only attends to a subset of previous tokens (local context) rather than all of them. Context: A necessary trade-off to make models faster and capable of handling longer documents5.
RoPE (Rotary Positional Embeddings): A modern method for encoding the order of words. Instead of just stamping a "1" or "2" on a word, it uses geometric properties to help the model understand relative positions. Context: A key component in newer models like Llama 26.
The Forward Pass: The complete journey of data through the model, from token ID input to probability output.
4. Golden Quotes (Verbatim)
"The tokenizer is followed by the neural network: a stack of Transformer blocks that do all of the processing. That stack is then followed by the LM Head, which translates the output of the stack into probability scores." 7
"The attention calculation is the most computationally expensive part of the process." 8
"Positional embeddings ... enable the model to keep track of the order of tokens/words in a sequence/sentence, which is an indispensable source of information in language." 9
5. Stories & Case Studies
The "Gardening Apology" Email: The authors show the model generating an email starting with "Subject: My Sincere Apologies for the Gardening Mishap."
The Lesson: This illustrates the generation loop. The model generates "Dear", then feeds it back in to generate "Sarah", and so on. It also demonstrates the
max_new_tokenslimit; the model cut off mid-sentence because it hit the arbitrary token limit set by the user10.
The "Sparse vs. Full" Visualization: The book uses a visual comparison of attention patterns (colored blocks).
The Lesson: Full attention (checking every word against every other word) is accurate but slow. Sparse attention (checking only neighbors) is fast but risks missing long-distance context. GPT-3 solved this by alternating them (one block sparse, one block full)11.
6. Actionable Takeaways (The "How-To")
Mind the Context Window: Understand that "context" is expensive. If you are building an app, know that the Attention mechanism is why processing long documents costs more money and time12.
Debug with
max_new_tokens: If your model output is cutting off abruptly (like the gardening email), check your generation parameters. The model isn't "dumb"; it just hit the wall you built13.Look for RoPE: When choosing an open-source model, favor those using RoPE (like Llama) if you need the model to handle long sequences or complex structural dependencies14.
7. Critical Reflection
The Counter-Argument: The chapter discusses Sparse Attention as a solution to the cost of computing. However, critics (and the authors themselves) note that if you only use sparse attention, "the quality of the generation would vastly degrade" because the model loses the ability to connect distant ideas15. The "perfect" architecture is still a balancing act between the "smart but slow" full attention and the "fast but myopic" sparse attention.
Connection to Next Chapter: Now that we understand the engine (Transformer) and the fuel (Tokens/Embeddings), the book transitions from "Understanding" to "Using." Chapter 4 will likely move into practical applications, specifically Classification, using these pretrained architectures to solve real-world problems.
Based on the content retrieved from the file, here are the comprehensive notes for Chapter 4.
Chapter 4: Classification
1. The "One-Line" Gist
This chapter transitions from theory to practice, demonstrating how to use both Representation models (like BERT) and Generative models (like ChatGPT) to solve the most fundamental NLP task: sorting text into categories (Sentiment Analysis).
2. Detailed Summary (The "Meat")
The authors define classification as the "Hello World" of applying LLMs. They distinguish between two distinct approaches to solving this problem11:
Representation Approach (The Specialist): This involves using "Encoder-only" models. You can either use a Task-Specific Model (one that has already been fine-tuned on data, like tweets) or use a raw model to extract Embeddings and feed them into a simple statistical classifier (like Logistic Regression)2. This method is highlighted for being lightweight and efficient.
Generative Approach (The Generalist): This involves using "Decoder" models (like GPT) or "Encoder-Decoder" models (like T5). Here, classification is framed as a text generation task. You don't ask for a probability score; you ask the model to write the word "Positive" or "Negative"3.
The chapter heavily emphasizes practical trade-offs. While Generative models are flexible (you can use them without training data via Prompt Engineering), Representation models are generally faster and cheaper for high-volume tasks. The authors demonstrate this by training a simple Logistic Regression on top of embeddings, achieving an impressive 85% F1 score with minimal compute4.
3. Key Concepts & Definitions
Task-Specific Models: Pretrained models hosted on hubs (like Hugging Face) that have already been fine-tuned for a specific domain (e.g.,
twitter-roberta-base-sentiment), allowing for immediate use without training5.Text-to-Text Transfer Transformer (T5): An architecture that treats every NLP problem (translation, classification, summarization) as a text-generation problem. Context: Introduced as a bridge between the two approaches6.
Prompt Engineering for Classification: The technique of designing a prompt (e.g., "Return 1 for positive, 0 for negative") to force a creative generative model to output structured, classifiable data7.
Exponential Backoff: A strategy for handling API rate limits where the code waits progressively longer between retries if the API (like OpenAI's) rejects a request8.
4. Golden Quotes (Verbatim)
"Although both representation and generative models can be used for classification, their approaches differ." 99
"By training a classifier on top of our embeddings, we managed to get an F1 score of 0.85! This demonstrates the possibilities of training a lightweight classifier while keeping the underlying embedding model frozen." 10
"Iteratively improving your prompt to get your preferred output is called prompt engineering." 11
5. Stories & Case Studies
The "Rotten Tomatoes" Benchmark: The authors use a dataset of 5,331 positive and 5,331 negative movie reviews to benchmark every method in the chapter.
The Lesson: Having a standardized dataset allows for fair comparison. They show that a simple embedding model + Logistic Regression is remarkably competitive against more complex methods121212.
The ChatGPT "Classifier": The authors ask ChatGPT to classify reviews using the prompt: "If it is positive return 1 and if it is negative return 0."
The Lesson: This illustrates that modern classification doesn't always require training a neural network; sometimes it just requires asking the right question (Prompt Engineering). However, they warn that this costs money per API call13.
6. Actionable Takeaways (The "How-To")
Shop Before You Build: Before training a model, search the Hugging Face Hub for a "Task-Specific Model." If you need to classify hate speech or financial sentiment, someone has likely already uploaded a model that does exactly that14.
The "Embedding + Sklearn" Trick: If you have labeled data but no GPU for training, use a pre-trained LLM to generate embeddings (static numbers) for your text, then train a standard Scikit-Learn Logistic Regression on those numbers. It is fast, cheap, and surprisingly accurate15.
Hard-Code Your Output: When using a Generative model (like GPT-4) for classification, explicitly constrain the output in the prompt (e.g., "Do not give any other answers"). Otherwise, the model might chat with you instead of classifying16.
7. Critical Reflection
The Counter-Argument: The chapter presents Generative Classification (using ChatGPT) as a viable option. While true, using a massive, expensive model to output a binary "0" or "1" is often computationally wasteful compared to a tiny BERT model. The "cool factor" of using GPT often outweighs practical efficiency in production environments.
Connection to Next Chapter: This chapter focused on Supervised Learning (where we have labels like "Positive/Negative"). But what if we have millions of documents and no labels? This sets the perfect stage for Chapter 5, which covers Clustering and Topic Modeling (Unsupervised Learning).
Based on the content retrieved from the file, here are the comprehensive notes for Chapter 5.
Chapter 5: Clustering and Topic Modeling
1. The "One-Line" Gist
This chapter tackles the problem of "no labels" by introducing unsupervised learning techniques—specifically moving from simple Text Clustering to advanced Topic Modeling (using BERTopic)—to automatically discover hidden themes in massive datasets1111.
2. Detailed Summary (The "Meat")
The authors shift focus from Supervised Learning (Chapter 4) to Unsupervised Learning. The core problem addressed is: How do we make sense of 45,000 documents if we don't have any labels?
The chapter outlines a specific "Text Clustering Pipeline"22:
Embeddings: Convert text into numbers (vectors).
Dimensionality Reduction: Compress these vectors (using UMAP) so they are easier to process.
Clustering: Group them using HDBSCAN, a density-based algorithm that identifies clusters and, crucially, ignores "noise" (outliers).
However, the authors argue that clustering alone isn't enough because a cluster of points on a graph doesn't tell you what the topic is. This leads to the introduction of Topic Modeling via BERTopic. This framework extends the pipeline by adding a "Representation" step. It uses a clever mathematical trick called c-TF-IDF (Class-based TF-IDF) to extract the most important keywords for each cluster, effectively turning a pile of numbers into a labeled topic like "Machine Learning" or "Healthcare"3.
3. Key Concepts & Definitions
Text Clustering: The unsupervised process of grouping documents based on semantic similarity without prior labeling. Context: Used for finding outliers or speeding up labeling44.
HDBSCAN: A hierarchical clustering algorithm that finds dense groups of data points and identifies "noise" (outliers) that don't belong to any group. Context: The engine used to group the embeddings5.
c-TF-IDF (Class-based TF-IDF): A variation of the classic TF-IDF formula. Instead of calculating word importance for a document, it calculates it for a whole cluster. Context: This is how the model knows that the word "patient" is important to the "Medical" cluster6.
BERTopic: A modular topic modeling framework that combines embeddings, clustering, and representation steps to discover topics. Context: The main tool introduced in this chapter7.
MMR (Maximal Marginal Relevance): An algorithm used to diversify keywords. Context: It prevents a topic from being described by redundant words like "summary, summaries, summarization"8.
4. Golden Quotes (Verbatim)
"Text clustering, unbound by supervision, allows for creative solutions and diverse applications, such as finding outliers, speedup labeling, and finding incorrectly labeled data." 99
"The clustering algorithm not only impacts how clusters are generated but also how they are viewed." 10
"Ideally, we generally describe a topic using keywords or keyphrases and, ideally, have a single overarching label." 1111
5. Stories & Case Studies
The ArXiv Analysis: The authors analyze 44,949 academic abstracts from ArXiv’s "Computation and Language" section.
The Process: They feed these abstracts through the pipeline.
The Result: The model automatically identifies distinct research topics, separating papers about "speech recognition" from those about "sentiment analysis" without ever being told those categories existed121212.
The "Summary" Redundancy: A topic was initially described by the words: “summarization | summaries | summary”.
The Fix: The authors applied MMR (Maximal Marginal Relevance).
The Lesson: The keywords shifted to “summarization | document | extractive | rouge”, proving that you can force the model to give you diverse descriptive words rather than synonyms13.
6. Actionable Takeaways (The "How-To")
Handle the Noise: When clustering real-world data, use HDBSCAN instead of K-Means. HDBSCAN has a "noise" category (label -1) for data points that don't fit anywhere, which keeps your actual clusters clean14.
Diversify Your Keywords: If your topic model is giving you repetitive keywords (e.g., "car, cars, auto"), apply MMR to force diversity and get a richer description of the topic15.
Visualize to Validate: Don't trust the lists of words blindly. Use interactive visualizations (like those in BERTopic) to hover over documents and confirm they actually belong to the assigned topic16.
7. Critical Reflection
The Counter-Argument: The chapter relies heavily on Embeddings. If the underlying embedding model (e.g., BERT) doesn't understand the specific jargon of your industry (like legal or medical tech), the clusters will be nonsense ("Garbage In, Garbage Out"). The authors address this implicitly by suggesting task-specific models in previous chapters.
Connection to Next Chapter: This chapter focused on organizing and understanding large collections of text without a specific query. But what if we want to find a specific needle in that haystack? This sets the stage for Chapter 6, which likely covers Semantic Search and Information Retrieval17.
Based on the content retrieved from the file, here are the comprehensive notes for Chapter 6.
Chapter 6: Semantic Search and Retrieval Augmented Generation (RAG)
1. The "One-Line" Gist
This chapter upgrades the "Ctrl+F" keyword search of the past to "Semantic Search" (finding meaning) and introduces the modern architecture of RAG—retrieving relevant facts to ground Generative AI and prevent hallucinations.
2. Detailed Summary (The "Meat")
The authors define a new paradigm for Information Retrieval. Traditional search (Keyword/Lexical) looks for exact word matches. Semantic Search looks for meaning using embeddings. The chapter structures this into a hierarchy of sophistication:
Dense Retrieval (The Fast Way): This uses Bi-Encoders. The query and the documents are converted into embeddings independently. Finding the answer is just a math problem (calculating the distance between the query vector and document vectors). It is incredibly fast because the documents are pre-calculated, but it can miss nuances.
Reranking (The Accurate Way): This uses Cross-Encoders. Instead of processing the query and document separately, this model looks at them together (simultaneously). It asks, "How relevant is Document A to Query B?" This is much more accurate but computationally expensive.
The "Search Pipeline" Strategy: The authors suggest a "Retrieve & Rerank" pipeline: Use Fast Dense Retrieval to get the top 100 results, then use a Slow Reranker to sort the top 10 to show the user.
RAG (Retrieval Augmented Generation): The chapter frames RAG as the ultimate application of search. Instead of showing the user a list of links, you feed the retrieved text into an LLM (Generative Model) and ask it to summarize the answer.
3. Key Concepts & Definitions
Dense Retrieval: A search method using embeddings (vectors) to find documents that are semantically similar to a query. Context: Replaces traditional keyword search.
Bi-Encoder: A model architecture that creates an embedding for the document and the query separately. Context: Used for fast retrieval of millions of documents.
Cross-Encoder: A model architecture that processes the query and document together to output a similarity score. Context: Used for "Reranking" a small set of results for maximum accuracy.
Hybrid Search: Combining Semantic Search (Vectors) with Keyword Search (BM25). Context: Essential because vectors sometimes miss exact matches (like product IDs).
ANN (Approximate Nearest Neighbor): Algorithms (like FAISS or HNSW) used to search through millions of vectors in milliseconds. Context: The technology that makes semantic search scalable.
4. Golden Quotes (Verbatim)
"Three broad categories of these models are dense retrieval, reranking, and RAG."
"Another caveat of dense retrieval is when a user wants to find an exact match for a specific phrase. That’s a case that’s perfect for keyword matching. That’s one reason why hybrid search... is advised."
"The importance of inference speed should not be underestimated in real-life solutions."
5. Stories & Case Studies
The "Interstellar" Movie Database: The authors use a dataset of movie descriptions to demonstrate retrieval.
The Query: They search for "Interstellar premiered on October 26, 2014...".
The Result: The model retrieves documents about "Kip Thorne" (scientific consultant) and "Christopher Nolan" (director) even though the query didn't explicitly name them.
The Lesson: This proves that the model understands the relationships between the movie and its creators, not just the words in the title.
The "Threshold" Dilemma: The authors discuss filtering search results.
The Lesson: A search engine needs a "cutoff" (e.g., Distance < 0.6). If the user asks a nonsense question, the search engine should return nothing rather than the "nearest" (but still irrelevant) result.
6. Actionable Takeaways (The "How-To")
Build a Pipeline: Do not rely on one model. Use a Bi-Encoder (like
all-MiniLM-L6-v2) to fetch 50 candidates, then use a Cross-Encoder to rerank the top 5. This gives you the speed of Google with the intelligence of GPT-4.Use Hybrid Search: If you are building a search for an e-commerce site or code repository, you must keep keyword search. Semantic search struggles with specific serial numbers, error codes (
0x404), or rare proper nouns.Scale with Vector DBs: If you have <100k documents, a simple local array is fine. If you have >1M, use a Vector Database (like Pinecone, Weaviate, or Milvus) or a library like FAISS to handle the indexing.
7. Critical Reflection
Counter-Argument: The chapter praises Semantic Search, but in practice, it is "fuzzy." It often returns results that are topically related but factually irrelevant (e.g., searching for "Can dogs eat grapes?" might retrieve "Grapes are toxic to cats" because they are semantically close). This is why the Reranking step is not optional—it is mandatory for production quality.
Connection to Next Chapter: We now know how to find the right information (Retrieval). The logical next step is to use that information to create new content. This sets the stage for Chapter 7, which focuses on Generating Text with Decoder-only models.
Chapter 7: Advanced Text Generation (Chains & Memory)
1. The "One-Line" Gist
This chapter moves beyond simple "one-shot" prompts to engineering complex applications, specifically by linking multiple model calls into Chains and endowing the model with Memory to handle reasoning and multi-turn conversations.
2. Detailed Summary (The "Meat")
The authors argue that a single prompt is often insufficient for complex tasks. To solve this, they introduce two structural innovations:
Reasoning (Chain of Thought): The chapter emphasises that LLMs struggle with math and logic if forced to answer immediately. By using Chain-of-Thought (CoT) prompting, we force the model to "show its work." The authors demonstrate that simply adding the phrase "Let's think step by step" triggers the model to generate intermediate steps, using its own previous output as a guide to reach the correct solution.
Chains: The concept of breaking a complex problem into a sequence of sub-tasks. Instead of one massive prompt, the output of the first prompt (e.g., "Write an outline") becomes the input for the second prompt (e.g., "Write the first chapter based on this outline").
Memory: Since LLMs are stateless (they forget you the moment the API call ends), the authors explain how to build chatbots by manually feeding the conversation history back into the prompt. They contrast different memory architectures, such as storing every word vs. summarizing the past to save tokens.
3. Key Concepts & Definitions
Chain-of-Thought (CoT): A prompting technique that encourages the model to generate intermediate reasoning steps before the final answer. Context: Critical for math and logic tasks.
Zero-Shot CoT: A variation where you don't provide examples but simply append "Let's think step by step" to the prompt.
Sequential Chains: A pipeline architecture where the output of one LLM call is used as the input for the next.
ConversationBufferMemory: A memory strategy that stores the raw text of the entire conversation history. Context: High accuracy but "hogs tokens."
ConversationSummaryMemory: A memory strategy that uses an LLM to periodically summarize the conversation history. Context: Saves tokens but increases latency (slower) and loses nuance.
4. Golden Quotes (Verbatim)
"By addressing the reasoning process the LLM can use the previously generated information as a guide through generating the final answer." 1
"With sequential chains, the output of a prompt is used as the input for the next prompt." 2
"Often, it is a trade-off between speed, memory, and accuracy. Where ConversationBufferMemory is instant but hogs tokens, ConversationSummaryMemory is slow but frees up tokens to use." 3
5. Stories & Case Studies
The Cafeteria Apples Problem: The authors test the model with a math problem: "The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many...?"
The Lesson: By using Zero-shot CoT, the model explicitly writes: "Step 1: Start with 23... Step 2: Subtract 20..." and arrives at the correct answer (9). Without this reasoning step, models often hallucinate the final number.
The "Storyteller" Chain: The authors describe a chain for writing a story.
The Lesson: Rather than asking for a whole story at once, a Sequential Chain first generates a synopsis, then passes that synopsis to a second prompt to generate character descriptions, ensuring consistency.
6. Actionable Takeaways (The "How-To")
Use the Magic Phrase: For any task requiring logic (math, coding, planning), always append "Let's think step by step" to your prompt. It is a free performance boost.
Audit Your Memory: If you are building a chatbot, monitor your token usage. If conversations get long, switch from
BufferMemorytoSummaryMemoryto prevent crashing the context window (and your wallet).Chain, Don't Stuff: If a complex prompt isn't working, break it into two. Ask the model to plan the answer first, then feed that plan into a second prompt to write the answer.
7. Critical Reflection
The Counter-Argument: The chapter highlights
ConversationSummaryMemoryas a solution to token limits. However, this introduces latency (you have to wait for the summary to be generated) and compounding errors (if the summary misses a detail, it is lost forever). Sometimes, a simple "sliding window" (keeping only the last 5 messages) is a better engineering trade-off.Connection to Next Chapter: We have mastered "Internal Knowledge" (Pre-training) and "Short-term Memory" (Context). But what if the model needs to know about current events or private data that doesn't fit in the context? This leads directly to Chapter 8, which covers RAG (Retrieval Augmented Generation).
Chapter 8: Semantic Search and Retrieval-Augmented Generation
1. The "One-Line" Gist
This chapter explores how language models revolutionized information retrieval—moving from keyword matching to "Semantic Search"—and how this retrieval capability is the key to solving model hallucinations through Retrieval-Augmented Generation (RAG).
2. Detailed Summary (The "Meat")
The authors begin by noting a historical pivot point: shortly after the BERT paper (2018), both Google and Bing integrated Transformer models into their search engines, marking "one of the biggest leaps forward in the history of Search." This shift was from Keyword Search (matching text literal strings) to Semantic Search (matching meaning).
The chapter breaks down the modern search architecture into three distinct categories:
Dense Retrieval: This relies on Embeddings. Both the search query and the documents are converted into vectors (points in space). Search becomes a geometry problem: finding the "nearest neighbors" to the query vector. This is fast but relies on the assumption that a query and its answer are mathematically close in vector space.
Reranking: A refinement step. A search pipeline might use Dense Retrieval to fetch 100 results, and then use a Reranker model to score them by relevance. Unlike dense retrieval (which looks at documents in isolation), a reranker looks at the query and document together to judge the fit, offering higher accuracy at the cost of speed.
Retrieval-Augmented Generation (RAG): The synthesis of Search and Generation. To fix "hallucinations" (where LLMs make up facts), RAG systems first retrieve relevant facts from a trusted database and then feed them to the LLM to generate an answer.
3. Key Concepts & Definitions
Semantic Search: A search technique that uses embeddings to understand the intent and meaning of a query rather than just matching keywords. Context: The technology powering modern Google and Bing.
Dense Retrieval: The process of retrieving documents based on the similarity of their embeddings to the query embedding. Context: Fast, scalable retrieval for millions of documents.
Reranking: A secondary step in search pipelines where a more powerful model re-scores a small set of candidate results to improve precision. Context: "Vastly improved results" compared to raw embedding search.
RAG (Retrieval-Augmented Generation): A framework where an LLM is provided with external data (retrieved via search) to ground its answers in fact. Context: The primary method for reducing hallucinations.
Hallucinations: The tendency of generative models to confidently produce incorrect information. Context: The problem RAG is designed to solve.
4. Golden Quotes (Verbatim)
"Search was one of the first language model applications to see broad industry adoption... Their addition instantly and dramatically improves some of the most mature, well-maintained systems that billions of people around the planet rely on."
"The ability they add is called semantic search, which enables searching by meaning, and not simply keyword matching."
"Generative search is a subset of a broader type of category of systems better called RAG systems. These are text generation systems that incorporate search capabilities to reduce hallucinations."
5. Stories & Case Studies
The "Interstellar" Search: The authors demonstrate building a search engine for the Wikipedia page of the movie Interstellar using Cohere and FAISS.
The Query: They ask, "how precise was the science"
The Result: The system retrieves sentences about "Kip Thorne" (theoretical physicist) and "scientific accuracy," despite the query not containing those specific proper nouns.
The Lesson: This proves that Embeddings capture the concept of science (linking "precise" to "accuracy" and "physicist"), which a simple keyword search might miss if the exact words don't match.
6. Actionable Takeaways (The "How-To")
Build a Pipeline: Don't rely on just one method. A standard industrial pipeline is: Dense Retrieval (to get 100 candidates) -> Reranking (to sort the top 10) -> RAG (to summarize the answer).
Use Vector Databases: The chapter introduces FAISS (Facebook AI Similarity Search) as a tool to store and search embeddings efficiently. If you have a large dataset, you need an index, not just a loop.
Set Similarity Thresholds: Not all search results are good. The authors advise setting a "max threshold of similarity" to filter out results. If the nearest document is still far away in vector space, it's better to return "No results found" than an irrelevant hallucination.
7. Critical Reflection
The "Query-Answer" Gap: The authors note a subtle flaw in Dense Retrieval: "Are a query and its best result semantically similar? Not always." A question ("Who is the CEO?") and its answer ("Satya Nadella") might actually be far apart in vector space because they look very different. This highlights the need for models specifically trained on Question-Answer pairs (covered in later chapters) rather than generic text similarity.
Connection to Next Chapter: This chapter focused on finding the data (Search) and generating the answer (RAG). But to make these systems robust in production, we need to evaluate them properly and handle more complex data structures. This leads into future chapters on Fine-Tuning and Evaluation.
Chapter 9: Multimodal Large Language Models
1. The "One-Line" Gist
This chapter expands the horizon of LLMs beyond text, explaining how models are adapted to "see" images by treating visual data as just another language (modality) to be tokenized and processed.
2. Detailed Summary (The "Meat")
The authors argue that language does not exist in a vacuum—human communication relies on facial expressions and visual context. Therefore, for models to be truly intelligent, they must be Multimodal (capable of processing text, images, audio, etc.).
The core technical challenge addressed is: How do we feed a picture into a text model?
The solution involves adapting the Transformer architecture for vision:
Vision Transformers (ViT): Instead of tokenizing words, the model breaks an image into a grid of square patches (e.g., 16x16 pixels). Each patch is flattened into a vector, effectively becoming a "visual word."
The "Projection" Layer: To make these visual patches compatible with an LLM, they pass through a projection layer that translates "pixel math" into "embedding math." This allows the LLM to process a photo of a cat exactly the same way it processes the word "cat."
Contrastive Learning (CLIP): The authors introduce CLIP (Contrastive Language-Image Pre-training) as the bridge. CLIP is trained on millions of image-caption pairs to learn that the image of a dog and the text "a dog" should have similar vector representations.
3. Key Concepts & Definitions
Multimodal Model: An AI system capable of processing and relating information from multiple modalities (e.g., text, images, audio) simultaneously. Context: The next evolution after text-only LLMs.
Modality: A specific type of data input, such as text, images, or sound.
Vision Transformer (ViT): An architecture that applies the Transformer mechanism to images by splitting them into patches instead of tokens. Context: The standard way to make Transformers "see."
CLIP (Contrastive Language-Image Pre-training): A model trained to predict which caption goes with which image. Context: It serves as the "translator" between the visual world and the textual world.
Image Patching: The process of cutting an image into a grid of smaller squares to be processed sequentially, similar to words in a sentence.
4. Golden Quotes (Verbatim)
"Models can be much more useful if they’re able to handle types of data other than text... A model that is able to handle text and images (each of which is called a modality) is said to be multimodal."
"The ability to receive and reason with multimodal input might further increase and help emerge capabilities that were previously locked."
"Instead of tokenizing words, we are 'tokenizing' images by breaking them down into patches."
5. Stories & Case Studies
The "Describe the Image" Task: The authors demonstrate feeding an image into a multimodal model and asking, "What is in this picture?"
The Mechanism: The model doesn't just "tag" objects; it generates a full sentence description. This proves it isn't just classifying; it is understanding the relationship between the visual elements and converting them to syntax.
The "Zero-Shot" Classification: Using CLIP, the authors classify images without training a specific classifier.
The Lesson: By checking if an image embedding is closer to the text embedding of "a photo of a dog" or "a photo of a cat," CLIP can classify images it has never seen before, purely based on language understanding.
6. Actionable Takeaways (The "How-To")
Use CLIP for Search: If you are building an image search engine, do not use metadata tags. Use CLIP embeddings so users can search for "a happy dog on the beach" and find relevant images even if they aren't tagged with those specific words.
Prepare Your Images: When using Vision Transformers, remember that image resolution matters. Standard ViT models expect specific input sizes (e.g., 224x224). Pre-process your images (resize/crop) before feeding them to the model to avoid errors.
Think Beyond Text: If your problem involves physical world data (documents, charts, screenshots), stop using OCR (Optical Character Recognition) + LLM. Switch to a Multimodal LLM (like GPT-4V or LLaVA) that can read the text and understand the layout/context simultaneously.
7. Critical Reflection
The Counter-Argument: The chapter presents Multimodality as a "free lunch" of added capability. However, multimodal models are significantly more computationally expensive and slower than text-only models. Processing images requires processing thousands of "patch tokens," which eats up the context window rapidly.
Connection to Previous Chapters: This chapter represents the convergence of everything learned so far: Embeddings (Chapter 2) are used to link text and image; Transformers (Chapter 3) process the data; and Classification (Chapter 4) is performed using these new visual capabilities.
Python
Code output
Python
Code output
Python
Code output
Based on the content retrieved from the file, here are the comprehensive notes for Chapter 10.
Chapter 10: Creating Text Embedding Models
1. The "One-Line" Gist
This chapter guides the reader from merely using off-the-shelf embeddings to creating and fine-tuning their own, using Contrastive Learning to teach models specifically what "similarity" means in their unique domain (e.g., legal, medical, or sentiment).
2. Detailed Summary (The "Meat")
The authors assert that while general-purpose embedding models (like OpenAI's or BERT's) are powerful, they often fail when "similarity" implies something specific—like sentiment or technical jargon—rather than just general topic overlap.
The core solution introduced is Contrastive Learning.
The Logic: You don't teach a model what a "contract" is by defining it. You teach it by showing it a "contract" and a "legal agreement" and saying, "These are the same", while showing it a "contract" and a "cooking recipe" and saying, "These are different."
The Mechanism: The model is trained to minimize the distance between "positive pairs" (similar texts) and maximize the distance between "negative pairs" (dissimilar texts) in vector space.
The Outcome: The authors demonstrate that you can take a standard model and "warp" its vector space. For example, a standard model groups text by topic (Sports vs. Politics). A fine-tuned model can be forced to group text by sentiment (Happy Sports and Happy Politics vs. Angry Sports and Angry Politics).
3. Key Concepts & Definitions
Contrastive Learning: A training technique that learns representations by contrasting positive pairs (similar inputs) against negative pairs (dissimilar inputs). Context: The primary method for training embedding models.
Contrastive Explanation: The philosophical concept that understanding requires alternatives ("Why P and not Q?"). Context: Used to explain why models need negative examples to learn context.
Bi-Encoder Fine-Tuning: The process of updating the weights of a standard BERT-like model so that the embeddings it produces are optimized for a specific task.
Semantic vs. Sentiment Similarity: The distinction that "similarity" is subjective. Context: A standard model thinks "I love this movie" and "I hate this movie" are similar (both about movies). A sentiment-tuned model thinks they are opposites.
4. Golden Quotes (Verbatim)
"It is nearly impossible to overstate the importance of embedding models in the field as they are the driving power behind so many applications."
"Contrastive learning is a technique that aims to train an embedding model such that similar documents are closer in vector space while dissimilar documents are further apart."
"In order to accurately capture the semantic nature of a document, it often needs to be contrasted with another document for a model to learn what makes it different or similar."
5. Stories & Case Studies
The Story of the Bank Robber: The authors tell a famous anecdote where a reporter asks a robber, "Why did you rob the bank?" and the robber replies, "Because that is where the money is."
The Lesson: This illustrates Contrastive Explanation. The reporter meant "Why did you rob a bank (instead of working)?", but the robber answered "Why did you rob a bank (instead of a bakery)?". Without the "contrast" (the negative example), the intent of the question is ambiguous. Models face this same ambiguity unless we train them with negatives.
6. Actionable Takeaways (The "How-To")
Fine-Tune for Niche Domains: If your RAG system (Chapter 8) is failing to retrieve relevant documents because your industry uses unique jargon, stop tweaking the prompt. Fine-tune the embedding model using the
sentence-transformerslibrary.Curate "Hard Negatives": When creating training data, don't just use random documents as negatives. Use "Hard Negatives"—documents that look similar (share keywords) but are actually wrong. This forces the model to learn nuance.
Define Your "Similarity": Before training, decide what you want "close" to mean. Do you want to group by Topic? By Author style? By Sentiment? Your training pairs must reflect this decision.
7. Critical Reflection
The Counter-Argument: Fine-tuning embedding models is powerful but risky. It can lead to "Catastrophic Forgetting", where the model becomes great at your specific task but loses its general understanding of language. The authors implicitly suggest this is a trade-off worth making for specialized industrial applications.
Connection to Next Chapter: Now that we have customized the Embeddings (the inputs), the next logical step is to customize the Model itself for specific tasks like classification. This sets the stage for Chapter 11, which covers Fine-Tuning BERT.
Chapter 11: Fine-Tuning Representation Models for Classification
1. The "One-Line" Gist
This chapter moves beyond simply "using" pre-trained models to modifying them, demonstrating how to unfreeze and fine-tune BERT's internal weights to achieve state-of-the-art accuracy on custom tasks like Sentiment Analysis and Named Entity Recognition (NER).
2. Detailed Summary (The "Meat")
The authors draw a sharp contrast with Chapter 4, where models were used as "frozen" feature extractors. While that approach is fast, it limits performance. In this chapter, they introduce Full Fine-Tuning, where the entire neural network (both the pre-trained BERT body and the new classification head) is updated during training.
The chapter breaks down the fine-tuning ecosystem into specific strategies:
Supervised Classification (The Gold Standard): If you have plenty of labeled data, you update every weight in the model. The authors show that this method allows the model to adapt its internal "understanding" of language to your specific domain, resulting in higher accuracy (F1 score) than frozen models.
Freezing Layers: A middle-ground approach. You can "freeze" the bottom layers of BERT (which understand basic grammar) and only fine-tune the top layers (which understand complex semantics). This saves computation time while still allowing for adaptation.
SetFit (Few-Shot Classification): Addressed as a solution for when you don't have much data. It uses sentence embeddings to train a classifier with very few examples.
Token Classification (NER): The chapter extends classification from "Document Level" (is this email spam?) to "Token Level" (is this word a person, location, or date?).
3. Key Concepts & Definitions
Fine-Tuning: The process of taking a pre-trained model and training it further on a specific dataset, updating its weights to minimize error on that specific task. Context: The best way to maximize accuracy.
Frozen Model: A model whose weights are locked and cannot be updated during training. Context: Used in Chapter 4; faster but less accurate.
Classification Head: The final layer added to the top of a neural network (usually a simple linear layer) that projects the model's output into the desired number of classes (e.g., 2 for Positive/Negative).
Catastrophic Forgetting: (Implicit) The risk that by fine-tuning a model too aggressively on new data, it loses the general knowledge it learned during pre-training.
Named Entity Recognition (NER): A specific type of classification where the model assigns a label (e.g., PERSON, ORG, DATE) to individual tokens within a sentence rather than the sentence as a whole.
4. Golden Quotes (Verbatim)
"If we have sufficient data, fine-tuning tends to lead to some of the best-performing models possible."
"Instead of freezing the model, we allow it to be trainable and update its parameters during training."
"It shows that fine-tuning a model yourself can be more advantageous than using a pretrained model."
5. Stories & Case Studies
The "Frozen vs. Thawed" Showdown: The authors compare the results of the "Frozen" model from Chapter 4 against the "Fine-Tuned" model from this chapter on the same dataset.
The Result: The frozen model achieved an F1 score of 0.80. The fine-tuned model achieved 0.85.
The Lesson: "Unfreezing" the model allows it to learn the nuances of your specific dataset (e.g., movie review slang), providing a significant accuracy boost for just a few minutes of extra training.
6. Actionable Takeaways (The "How-To")
Unfreeze for Accuracy: If you have the GPU memory (approx. 16GB for base models), always prefer full fine-tuning over frozen embeddings. The 5-10% performance gain is usually worth the compute cost.
Use the
TrainerAPI: Don't write your own PyTorch training loops. Use Hugging Face'sTrainerclass, which handles logging, evaluation, and saving checkpoints automatically.Monitor Overfitting: Because you are training a massive model on a potentially small dataset, watch your "Validation Loss." If it starts going up while "Training Loss" goes down, stop training immediately.
7. Critical Reflection
The Counter-Argument: Fine-tuning is computationally expensive. Storing a copy of a fine-tuned 12GB model for every task (one for sentiment, one for spam, one for toxicity) is a deployment nightmare. This is why Adapters (LoRA) are becoming popular, though this chapter focuses on full fine-tuning.
Connection to Next Chapter: We have now mastered Representation models (BERT). But the world is currently obsessed with Generation models (GPT). Chapter 12 will take these fine-tuning concepts and apply them to Generative AI, teaching us how to make models write better, not just classify better.
Chapter 12: Fine-Tuning Generation Models
1. The "One-Line" Gist
This final chapter explains how to transform a raw, unruly "Base Model" into a helpful "Chat Model" using the modern three-step pipeline: Pre-training, Supervised Fine-Tuning (SFT), and Preference Tuning (RLHF/DPO).
2. Detailed Summary (The "Meat")
The authors define the hierarchy of model creation. A Base Model (trained on raw internet text) is often useless for users because if you ask it "Write a poem," it might just complete the sentence with "...about a dog" instead of actually writing the poem. To fix this, the chapter details a pipeline:
Supervised Fine-Tuning (SFT): You feed the model examples of Instructions and Responses. This teaches the model the "chat" format.
Preference Tuning: This aligns the model with human values (safety, helpfulness).
The chapter is heavily focused on Efficiency. Fine-tuning a 70B parameter model is impossible for most people. The authors introduce PEFT (Parameter-Efficient Fine-Tuning) and specifically LoRA (Low-Rank Adaptation).
The LoRA Logic: Instead of updating all 70B weights (which requires massive memory), LoRA freezes the main model and adds tiny, trainable "adapter" layers. This reduces the trainable parameters by 99% while achieving similar performance.
QLoRA: Combines LoRA with "Quantization" (4-bit precision), allowing you to fine-tune massive models on a single consumer GPU.
Finally, the chapter covers Alignment. It contrasts the "Old Way" (RLHF - Reinforcement Learning from Human Feedback), which is complex and unstable, with the "New Way" (DPO - Direct Preference Optimization). DPO simplifies the process by mathematically optimizing the model to prefer "Chosen" answers over "Rejected" ones without needing a separate Reward Model.
3. Key Concepts & Definitions
Base Model vs. Instruct Model: A Base Model predicts the next word (autocomplete). An Instruct Model follows commands (chatbot).
SFT (Supervised Fine-Tuning): The process of training a model on
(Prompt, Response)pairs to teach it how to follow instructions.LoRA (Low-Rank Adaptation): A PEFT technique that freezes the model and trains small rank-decomposition matrices, making fine-tuning cheap and fast. Context: The standard method for open-source fine-tuning.
RLHF (Reinforcement Learning from Human Feedback): A complex alignment method that uses a "Reward Model" to score responses and Reinforcement Learning (PPO) to optimize the LLM.
DPO (Direct Preference Optimization): A newer, stable alignment method that optimizes the model directly on preference data (A > B) without a Reward Model.
4. Golden Quotes (Verbatim)
"Base models are a key artifact of the training process but are harder for the end user to deal with."
"When humans ask the model to write an article, they expect the model to generate the article and not list other instructions."
"We will explore the transformative potential of fine-tuning pretrained text generation models to make them more effective tools for your application."
5. Stories & Case Studies
The "Unhelpful" Base Model: The authors describe a scenario where a user inputs an instruction.
The Failure: A Base Model interprets the instruction as just text to be continued, so it generates more instructions rather than an answer.
The Fix: SFT (Supervised Fine-Tuning) is introduced as the specific step that breaks this pattern, teaching the model the "User -> Assistant" interaction protocol.
The "Hardware Barrier": The authors discuss the memory requirements of training.
The Solution: They present LoRA/QLoRA not just as a technique, but as a democratizing force that allows a student with a gaming laptop to improve a model that cost millions of dollars to build.
6. Actionable Takeaways (The "How-To")
Don't Full Fine-Tune: Unless you have a cluster of H100s, never try to update all weights of a 7B+ model. Always use LoRA or QLoRA.
Data Quality > Quantity: For SFT, 1,000 high-quality, human-curated examples (like the "LIMA" dataset logic) often beat 50,000 generated examples.
Choose DPO over RLHF: If you want to align your model (e.g., "Make it sound more professional"), use DPO. It is numerically more stable and easier to implement than the PPO/RLHF pipeline used by early GPT models.
7. Critical Reflection
The "Alignment Tax": The chapter implies alignment is always good. However, research suggests that heavy alignment (safety tuning) can sometimes make models "dumber" at creative or coding tasks (the "Alignment Tax"). DPO is efficient, but over-tuning on preferences can reduce the diversity of the model's outputs.
Closing the Loop: This chapter concludes the journey. The book started with "What is an LLM?" (Ch 1), moved to "How to use them" (Ch 4-9), "How to create embeddings" (Ch 10-11), and ends with the ultimate skill: "How to build your own custom LLM" (Ch 12).
Last updated