Back to Tracker

NLP & Transformers

Text · Embeddings · Transformers · BERT · GPT · Fine-tuning · RAG · Agents

Specialization3 Weeks7 LessonsPrepflix AI Roadmap
Word Embeddings

Embedding Methods

Bag of WordsSparse, ignores order and semantics
TF-IDFTerm freq × inverse doc freq — weights rare terms
Word2VecPredict word from context (CBOW) or context from word (Skip-gram)
GloVeGlobal co-occurrence statistics
FastTextCharacter n-grams, handles OOV words
ContextualBERT/GPT: same word, different embeddings per context

Word2Vec Intuition

king − man + woman ≈ queen
Similarity = cos(v₁, v₂) = v₁·v₂/(‖v₁‖‖v₂‖)

Neural network learns dense 100-300 dim vectors where semantically similar words cluster together. Trained by predicting neighbors in a large text corpus.

Modern practice: use pre-trained sentence embeddings (sentence-transformers) rather than word embeddings for most downstream tasks.
Transformers Architecture

Input Representation

Input = Token Embedding + Positional Encoding
PE(pos,2i) = sin(pos/10000^(2i/d_model))
PE(pos,2i+1) = cos(pos/10000^(2i/d_model))

Positional encoding injects sequence order (Transformers have no inherent notion of order unlike RNNs).

Tokenization

BPEGPT family — merge frequent byte pairs
WordPieceBERT — maximize LM likelihood of splits
SentencePieceLanguage-agnostic, works from raw text

Encoder vs Decoder

Encoder-only (BERT)

Bidirectional attention — sees full sequence. Best for: classification, NER, question answering, semantic search.

Decoder-only (GPT)

Causal (masked) attention — only sees past tokens. Best for: text generation, chat, code generation, reasoning.

Encoder-Decoder (T5, BART)

Encoder processes input, decoder generates output. Best for: translation, summarization, structured prediction.

BERT, GPT & Modern LLMs

BERT Pre-training Tasks

MLM (Masked LM)Predict 15% randomly masked tokens
NSPPredict if sentence B follows sentence A
[CLS] tokenPooled representation for classification
[SEP] tokenSeparates two sentences

GPT Pre-training

Objective: P(x_t | x_1,...,x_{t-1})

Autoregressive: predict next token. Scales amazingly — GPT-4 shows emergent abilities from sheer scale.

LLM Landscape (2024)

GPT-4 / GPT-4oOpenAI — SOTA reasoning, multimodal
Claude 3.5 SonnetAnthropic — coding, long context, safe
Llama 3.1 405BMeta — open weights, near GPT-4 performance
Gemini 1.5 ProGoogle — 1M token context window
Mistral / MixtralEfficient MoE architecture, open source
Scaling Laws: Loss ∝ N^-α (compute-optimal training: Chinchilla paper shows train on 20× more tokens per parameter).
Fine-tuning LLMs with Hugging Face
from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments from peft import LoraConfig, get_peft_model # Load model + tokenizer model_name = "bert-base-uncased" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2) # LoRA config (for efficient fine-tuning) lora_config = LoraConfig(r=16, lora_alpha=32, target_modules=["query","value"], lora_dropout=0.1, task_type="SEQ_CLS") model = get_peft_model(model, lora_config) model.print_trainable_parameters() # << 1% of total params! # Training args = TrainingArguments(output_dir="./output", num_train_epochs=3, per_device_train_batch_size=16, learning_rate=2e-4, evaluation_strategy="epoch", save_strategy="epoch") trainer = Trainer(model=model, args=args, train_dataset=train_ds, eval_dataset=val_ds) trainer.train()
LoRA Key Idea
W = W₀ + ΔW = W₀ + BA where B∈R^(d×r), A∈R^(r×k), r≪d

Instead of updating the full weight matrix W₀, learn a low-rank decomposition. With r=16, only 0.1-1% of parameters need training.

QLoRA

Quantize base model to 4-bit (NF4), apply LoRA on top. Enables fine-tuning 70B LLMs on a single A100 GPU. Use bitsandbytes library.

RAG: Retrieval Augmented Generation

RAG Pipeline

  1. Ingest: Chunk documents (512-1024 tokens)
  2. Embed: Convert chunks to dense vectors
  3. Index: Store in vector database (FAISS, Pinecone, Chroma)
  4. Query: Embed user question
  5. Retrieve: Top-k most similar chunks
  6. Generate: LLM answers using retrieved context

Why RAG > Fine-tuning (for knowledge)

  • Updated knowledge without retraining
  • Source attribution (cite the document)
  • Cheaper — no GPU needed for updates
  • Handles long documents beyond context window
  • Reduces hallucinations with grounded context
Advanced RAG: Hybrid search (dense + sparse BM25), re-ranking (cross-encoder), HyDE (hypothetical document embeddings), sentence window retrieval.
AI Agents

ReAct Pattern

Reason → Act → Observe → Repeat

LLM reasons about what to do, calls a tool (search, code exec, API), observes result, reasons again until task is complete.

# LangChain agent example from langchain.agents import create_react_agent from langchain.tools import Tool tools = [search_tool, calculator_tool, python_repl_tool] agent = create_react_agent(llm, tools, prompt) result = agent.invoke({"input": "What is 2024's GDP of India?"})

Agent Components

ToolsSearch, code execution, API calls, DB queries
MemoryConversation history, episodic, semantic
PlanningChain-of-Thought, Tree-of-Thought, MCTS
OrchestrationLangChain, LlamaIndex, AutoGen, CrewAI
Vector StoreFAISS, Pinecone, Chroma, Weaviate
Multi-agent: CrewAI and AutoGen let specialized agents collaborate — researcher + writer + critic pattern. Great for complex, multi-step tasks.