Math for AI — Prepflix Cheat Sheet

Linear Algebra

Core Operations

Dot Producta·b = Σaᵢbᵢ = |a||b|cos(θ)

Matrix MultiplyC[i,j] = Σₖ A[i,k]·B[k,j]

Transpose(Aᵀ)[i,j] = A[j,i]

InverseA·A⁻¹ = I (square, non-singular)

DeterminantScaling factor of linear transform

L1 Norm‖x‖₁ = Σ|xᵢ|

L2 Norm‖x‖₂ = √(Σxᵢ²)

Why It Matters in ML

Weights in neural networks are matrices
Matrix multiply = forward pass computation
Gradient vectors live in same space as weights
PCA uses eigendecomposition for feature extraction
Attention scores = QKᵀ (matrix multiply)
L1/L2 norms → Lasso / Ridge regularization

Key Intuition: A matrix is a function that transforms space — it rotates, scales, or projects vectors. Neural networks are chains of these transformations.

Eigenvalues & Eigenvectors

A·v = λ·v → (A − λI)·v = 0 → det(A − λI) = 0

PCA in 4 Steps

Standardize data (zero mean, unit variance)
Compute covariance matrix Σ
Eigendecompose: Σ = VΛVᵀ
Project: X_new = X · V[:, :k]

SVD: The Universal Decomposition

A = U Σ Vᵀ

U = left singular vectors, Σ = singular values (diagonal), V = right singular vectors. Used in recommendation systems (matrix factorization), image compression, and LSA.

Calculus & Gradient Descent

Key Rules

Chain Ruled/dx[f(g(x))] = f'(g(x))·g'(x)

Gradient∇f = [∂f/∂x₁, …, ∂f/∂xₙ]ᵀ

JacobianJ[i,j] = ∂fᵢ/∂xⱼ (vector→vector)

HessianH[i,j] = ∂²f/∂xᵢ∂xⱼ (curvature)

Taylor Expansionf(x+Δx) ≈ f(x) + ∇f·Δx

Gradient Descent Family

θ ← θ − α · ∇L(θ)

Batch GDEntire dataset per step

SGD1 sample per step, noisy

Mini-batch SGD32–512 samples, practical

AdamAdaptive LR + momentum

      Why Chain Rule = Backpropagation: Backprop computes ∂Loss/∂w for every weight by applying the chain rule backwards through the computation graph. This is literally all that backprop is.
    

Common Derivatives in ML

σ(x) = 1/(1+e⁻ˣ)
σ'(x) = σ(x)(1−σ(x))

tanh'(x) = 1 − tanh²(x)

ReLU'(x) = 1 if x>0, else 0

Probability Theory

Fundamentals

P(A|B) = P(A∩B) / P(B)

Bayes: P(A|B) = P(B|A)·P(A) / P(B)

P(A∪B) = P(A) + P(B) − P(A∩B)

E[X] = Σ xᵢP(xᵢ) | Var[X] = E[X²]−E[X]²

Key Distributions

GaussianN(μ,σ²): continuous, bell-shaped

BernoulliBinary outcome: p(1)=p, p(0)=1−p

CategoricalK-class: softmax output

PoissonCount events: λ^k·e^−λ / k!

UniformWeight init: U[−1/√n, 1/√n]

MLE vs MAP

MLE — Maximum Likelihood Estimation

θ_MLE = argmax P(data | θ)

Find parameters that make observed data most probable. Equivalent to minimizing cross-entropy loss. No prior — overfits with small data.

MAP — Maximum a Posteriori

θ_MAP = argmax P(θ | data) = argmax P(data|θ)·P(θ)

Incorporates prior belief P(θ). Gaussian prior → L2 regularization. Laplace prior → L1 regularization.

Statistics for ML

Key Concepts

p-valueP(data | H₀). Small p → reject H₀

Type I ErrorFalse Positive (α, significance level)

Type II ErrorFalse Negative (β, 1−β = power)

CLTSample means → N(μ, σ²/n) as n→∞

Confidence Intervalx̄ ± z·σ/√n (95%: z=1.96)

Effect SizeCohen's d = (μ₁−μ₂)/σ_pooled

A/B Testing Checklist

Define metric and minimum detectable effect
Calculate sample size (power analysis)
Run experiment for full duration (no peeking!)
Check for novelty effects and segment leakage
Use two-sample t-test for means, chi-square for rates
Correct for multiple comparisons (Bonferroni)
Report confidence interval, not just p-value

Interview Trap: "Statistical significance ≠ practical significance." Always report effect size and confidence intervals, not just p-value. A massive dataset can make tiny, meaningless differences significant.

Information Theory

Core Formulas

Shannon Entropy: H(p) = −Σ p(x) log p(x)

Cross-Entropy: H(p,q) = −Σ p(x) log q(x)

KL Divergence: D_KL(p‖q) = Σ p(x) log(p(x)/q(x))

Relationship: H(p,q) = H(p) + D_KL(p‖q)

Mutual Info: I(X;Y) = H(X) − H(X|Y)

ML Connections

Why Cross-Entropy = Loss?

Minimizing cross-entropy H(p,q) is exactly MLE — it measures how well predicted distribution q approximates true distribution p. KL divergence only adds a constant (H(p)), so minimizing cross-entropy ≡ minimizing KL divergence.

High EntropyUniform distribution, max uncertainty

Zero EntropyOne-hot, perfect certainty

KL ≥ 0Always ≥ 0, = 0 iff p = q

Interview Gold: Cross-entropy loss in classification = negative log-likelihood = minimizing KL divergence between predicted and true label distributions. All three are the same thing stated differently.

Quick Reference Card

Linear Algebra
• dot(A,B) = matrix multiply
• Eigenvec = direction preserved by A
• SVD → recommender systems
• L2 norm → Ridge regularization

Calculus
• Gradient = steepest ascent direction
• Chain rule = backpropagation
• Adam = momentum + adaptive LR
• Hessian = curvature (2nd order)

Prob/Stats/InfoTheory
• Bayes = prior × likelihood / evidence
• MLE → cross-entropy loss
• MAP → regularized loss
• Cross-entropy = KL divergence + H(p)