Preliminaries

• 1.Data ManipulationnewPTTFJAXMX
• 2.Data PreprocessingnewPTTFJAXMX
• 3.Linear AlgebranewPTTFJAXMX
• 4.CalculusnewPTTFJAXMX
• 5.Automatic DifferentiationnewPTTFJAXMX
• 6.Probability and StatisticsnewPTTFJAXMX
• 7.DocumentationnewPTTFJAXMX

Linear Regression in Neural Networks

• 1.Linear RegressionnewPTTFJAXMX
• 2.Object-Oriented Design for ImplementationnewPTTFJAXMX
• 3.Synthetic Regression DatanewPTTFJAXMX
• 4.Linear Regression Implementation from ScratchnewPTTFJAXMX
• 5.Concise Implementation of Linear RegressionnewPTTFJAXMX
• 6.GeneralizationnewPTTFJAXMX
• 7.Weight DecaynewPTTFJAXMX

Linear Classification in Neural Networks

• 1.Softmax RegressionnewPTTFJAXMX
• 2.The Image Classification DatasetnewPTTFJAXMX
• 3.The Base Classification ModelnewPTTFJAXMX
• 4.Softmax Regression Implementation from ScratchnewPTTFJAXMX
• 5.Concise Implementation of Softmax RegressionnewPTTFJAXMX
• 6.Generalization in ClassificationnewPTTFJAXMX
• 7.Environment and Distribution ShiftnewPTTFJAXMX

Multilayer Perceptron

• 1.Multilayer PerceptronsnewPTTFJAXMX
• 2.Implementation of Multilayer PerceptronsnewPTTFJAXMX
• 3.Forward Propagation, Backward Propagation, and Computational GraphsnewPTTFJAXMX
• 4.Numerical Stability and InitializationnewPTTFJAXMX
• 5.Generalization in Deep LearningnewPTTFJAXMX
• 6.DropoutnewPTTFJAXMX
• 7.Predicting House Prices on KagglenewPTTFJAXMX

Convolutional Neural Networks

• 1.From Fully Connected Layers to ConvolutionsPTTFJAXMX
• 2.Convolutions for ImagesPTTFJAXMX
• 3.Padding and StridePTTFJAXMX
• 4.Multiple Input and Multiple Output ChannelsPTTFJAXMX
• 5.PoolingPTTFJAXMX
• 6.Convolutional Neural Networks (LeNet)PTTFJAXMX

Modern Convnets

• 1.The ImageNet Moment: AlexNetPTTFJAXMX
• 2.Blocks, Bottlenecks, and Branches: VGG, NiN, GoogLeNetPTTFJAXMX
• 3.Normalization LayersPTTFJAXMX
• 4.Residual Networks: ResNet, ResNeXt, and DenseNetPTTFJAXMX
• 5.Efficient ConvNets: Depthwise Separability, Mobile Architectures, and Re-parameterizationPTTFJAXMX
• 6.Training Recipes MatterPTTFJAXMX
• 7.ConvNeXt: A ConvNet for the 2020sPTTFJAXMX
• 8.Design Spaces and the Big PicturePTTFJAXMX

Sequence Models

• 1.Working with SequencesPTTFJAXMX
• 2.From Text to TokensPTTFJAXMX
• 3.Language ModelsPTTFJAXMX
• 4.Recurrent Neural NetworksPTTFJAXMX
• 5.Implementing RNN Language ModelsPTTFJAXMX
• 6.Backpropagation Through TimePTTFJAXMX
• 7.Decoding and GenerationPTTFJAXMX

Optimization Algorithms

• 1.LandscapesnewPTTFJAXMX
• 2.Gradient DescentnewPTTFJAXMX
• 3.Stochastic Gradient DescentnewPTTFJAXMX
• 4.MinibatchesPTTFJAXMX
• 5.MomentumPTTFJAXMX
• 6.AdamnewPTTFJAXMX
• 7.AdamWnewPTTFJAXMX
• 8.SchedulesnewPTTFJAXMX
• 9.MuonnewPTTFJAXMX
• 10.Batch SizenewPTTFJAXMX
• 11.Scaling UpnewPTTFJAXMX
• 12.PracticenewPTTFJAXMX

Attention

• 1.Queries, Keys, and ValuesnewPTTFJAXMX
• 2.Attention Scoring and MaskingnewPTTFJAXMX
• 3.Multi-Head and Cross-AttentionnewPTTFJAXMX
• 4.Positional InformationnewPTTFJAXMX
• 5.The Cost of AttentionnewPTTFJAXMX
• 6.What Attention ComputesnewPTTFJAXMX

Transformers

• 1.The Transformer BlocknewPTTFJAXMX
• 2.A GPT from ScratchnewPTTFJAXMX
• 3.Generation and the KV CachenewPTTFJAXMX
• 4.Encoders, Decoders, and Cross-AttentionnewPTTFJAXMX
• 5.Vision TransformernewPTTFJAXMX
• 6.Mixture of ExpertsnewPTTFJAXMX
• 7.Scaling Laws and the Modern RecipenewPTTFJAXMX

State Space Models

• 1.Gated RecurrencePTTFJAXMX
• 2.Linear Recurrence and State Space ModelsPTTFJAXMX
• 3.Selective State Space ModelsPTTFJAXMX
• 4.The Matrix State: From Linear Attention to Mamba-2newPTTFJAXMX
• 5.DeltaNet: Memory That EditsnewPTTFJAXMX
• 6.Learning at Test TimenewPTTFJAXMX
• 7.Hybrid ArchitecturesnewPTTFJAXMX

Computational Performance

• 1.The Performance ModelPTTFJAXMX
• 2.HardwarePTTFJAXMX
• 3.Compute Graphs and CompilationPTTFJAXMX
• 4.Memory and PrecisionPTTFJAXMX
• 5.Multi-GPU from First PrinciplesPTTFJAXMX
• 6.Multi-GPU in PracticePTTFJAXMX
• 7.Case Study: Making a Transformer FastPTTFJAXMX

Reinforcement Learning

• 1.Markov Decision Process (MDP)PTTFJAXMX
• 2.Value IterationPTTFJAXMX
• 3.Q-LearningPTTFJAXMX

Generative Adversarial Networks

• 1.Generative Adversarial NetworksPTTFJAXMX
• 2.Deep Convolutional Generative Adversarial NetworksPTTFJAXMX

Natural Language Processing: Pretraining

• 1.Encoder-Decoder Models for Sequence TransductionPTTFJAXMX
• 2.The Dataset for Pretraining Word EmbeddingsPTTFJAXMX
• 3.Pretraining word2vecPTTFJAXMX
• 4.Subword EmbeddingPTTFJAXMX
• 5.Word Similarity and AnalogyPTTFJAXMX
• 6.Bidirectional Encoder Representations from Transformers (BERT)PTTFJAXMX
• 7.The Dataset for Pretraining BERTPTTFJAXMX
• 8.Pretraining BERTPTTFJAXMX

Natural Language Processing: Applications

• 1.Sentiment Analysis and the DatasetPTTFJAXMX
• 2.Sentiment Analysis: Using Recurrent Neural NetworksPTTFJAXMX
• 3.Sentiment Analysis: Using Convolutional Neural NetworksPTTFJAXMX
• 4.Natural Language Inference and the DatasetPTTFJAXMX
• 5.Natural Language Inference: Using AttentionPTTFJAXMX
• 6.Natural Language Inference: Fine-Tuning BERTPTTFJAXMX

Computer Vision

• 1.Image AugmentationPTTFJAXMX
• 2.Fine-TuningPTTFJAXMX
• 3.Object Detection and Bounding BoxesPTTFJAXMX
• 4.Anchor BoxesPTTFJAXMX
• 5.Multiscale Object DetectionPTTFJAXMX
• 6.The Object Detection DatasetPTTFJAXMX
• 7.Single Shot Multibox DetectionPTTFJAXMX
• 8.Region-based CNNs (R-CNNs)PTTFJAXMX
• 9.Semantic Segmentation and the DatasetPTTFJAXMX
• 10.Transposed ConvolutionPTTFJAXMX
• 11.Fully Convolutional NetworksPTTFJAXMX
• 12.Neural Style TransferPTTFJAXMX
• 13.Image Classification (CIFAR-10) on KagglePTTFJAXMX
• 14.Dog Breed Identification (ImageNet Dogs) on KagglePTTFJAXMX

Gaussian Processes

• 1.Gaussian Process PriorsPTTFJAXMX
• 2.Gaussian Process InferencePTTFJAXMX

Hyperparameter Optimization

• 1.What Is Hyperparameter Optimization?PTTFJAXMX
• 2.Hyperparameter Optimization APIPTTFJAXMX
• 3.Asynchronous Random SearchPTTFJAXMX
• 4.Multi-Fidelity Hyperparameter OptimizationPTTFJAXMX
• 5.Asynchronous Successive HalvingPTTFJAXMX

Recommender Systems

• 1.The MovieLens DatasetPTTFJAXMX
• 2.Matrix FactorizationPTTFJAXMX
• 3.AutoRec: Rating Prediction with AutoencodersPTTFJAXMX
• 4.Personalized Ranking for Recommender SystemsPTTFJAXMX
• 5.Neural Collaborative Filtering for Personalized RankingPTTFJAXMX
• 6.Sequence-Aware Recommender SystemsPTTFJAXMX
• 7.Feature-Rich Recommender SystemsPTTFJAXMX
• 8.Factorization MachinesPTTFJAXMX
• 9.Deep Factorization MachinesPTTFJAXMX

Linear Algebra

• 1.Geometry and Linear Algebraic OperationsnewPTTFJAXMX
• 2.EigendecompositionsnewPTTFJAXMX
• 3.Singular Value Decomposition and Low-Rank ApproximationnewPTTFJAXMX

Calculus and Automatic Differentiation

• 1.Single Variable CalculusnewPTTFJAXMX
• 2.Multivariable CalculusnewPTTFJAXMX
• 3.Matrix Calculus and Automatic DifferentiationnewPTTFJAXMX
• 4.Integral CalculusnewPTTFJAXMX

Optimization

• 1.Gradient-Based OptimizationnewPTTFJAXMX
• 2.Stochastic and Adaptive MethodsnewPTTFJAXMX
• 3.Convex Sets and Convex FunctionsnewPTTFJAXMX
• 4.Constrained Optimization and DualitynewPTTFJAXMX
• 5.Numerical Stability and ConditioningnewPTTFJAXMX

Probability and Statistical Learning

• 1.Random VariablesnewPTTFJAXMX
• 2.DistributionsnewPTTFJAXMX
• 3.Maximum LikelihoodnewPTTFJAXMX
• 4.Bayesian ComputationnewPTTFJAXMX
• 5.StatisticsnewPTTFJAXMX
• 6.Concentration and GeneralizationnewPTTFJAXMX
• 7.Naive BayesnewPTTFJAXMX

Information Theory and Divergences

• 1.Entropy, Cross-Entropy, and KL DivergencenewPTTFJAXMX
• 2.Divergences and Distances Between DistributionsnewPTTFJAXMX
• 3.Mutual Information and Representation LearningnewPTTFJAXMX

Dynamics: Differential Equations and Generative Flows

• 1.Ordinary Differential Equations and Numerical SolversnewPTTFJAXMX
• 2.Stochastic Differential EquationsnewPTTFJAXMX
• 3.The Fokker--Planck Equation and Probability FlownewPTTFJAXMX
• 4.Score Matching, Diffusion, and Flow MatchingnewPTTFJAXMX

Tools for Deep Learning

• 1.Utility Functions and ClassesPTTFJAXMX