7 Modern Convnets
This chapter studies the architectures and training practices that established convolutional networks in computer vision. Since roughly 2021, vision transformers have led many large-scale image classification benchmarks, beginning with Dosovitskiy et al. (2021) and the Swin Transformer (Liu et al. 2021); we cover them in Chapter 11. Convnets remain important where latency is constrained, training data is limited, or the prediction is dense, as in segmentation (Long et al. 2015) and object detection (Redmon and Farhadi 2018). With modern training procedures and matched compute, they can also match transformers on image classification (Smith et al. 2023).
The first group of sections covers the architectural developments of 2012–2015. AlexNet (Krizhevsky et al. 2012) established deep convolutional networks on ImageNet (Section 7.1). VGG, NiN, and GoogLeNet then introduced repeated blocks, \(1 \times 1\) channel mixing, global pooling, and multi-branch design (Section 7.2). Batch normalization (Ioffe and Szegedy 2015) improved optimization (Section 7.3), while residual connections (He et al. 2016) made substantially deeper networks practical (Section 7.4). Normalization and residual connections subsequently became standard components outside computer vision, including in transformers.
The later sections separate architectural choices from training and deployment constraints. Efficient networks use depthwise convolution and structural re-parameterization (Section 7.5). Modern training procedures raise the accuracy of an unchanged ResNet-50 by more than four percentage points (Wightman et al. 2021) (Section 7.6). ConvNeXt applies these practices in a controlled modernization of ResNet (Liu et al. 2022) (Section 7.7). Finally, RegNet studies distributions over network designs rather than selecting a single architecture (Radosavovic et al. 2020) (Section 7.8).
The sections are arranged roughly chronologically because each design responds to a limitation of its predecessors. This also makes it possible to distinguish improvements due to architecture from those due to optimization, data augmentation, and computational budget.
Resources and Further Reading
The references below trace the architectural developments covered in this chapter and current practice: landmark networks from AlexNet to ConvNeXt, the training recipes that confound naive comparisons between them, and efficient networks for deployment. All are freely accessible online except where noted.
Books
- Deep Learning for Coders with fastai and PyTorch — Howard & Gugger — free notebooks; builds ResNets from scratch and applies modern training techniques (augmentation, schedules, mixed precision) of Section 7.6 in working code.
- Understanding Deep Learning — Simon J. D. Prince — free PDF; Chapter 11 (Residual networks) analyzes why residual connections ease optimization — loss-surface and gradient-propagation arguments complementing Section 7.4.
Courses and video lectures
- Michigan EECS 498-007, Lecture: CNN Architectures — Justin Johnson — free videos; walks AlexNet → VGG → GoogLeNet → ResNet with parameter/FLOP accounting, the same tour as this chapter’s first half.
- MIT 6.5940: TinyML and Efficient Deep Learning — Song Han — free lectures; depthwise separability, quantization, pruning, and edge deployment — the systems side of Section 7.5.
- Hugging Face Computer Vision Course — free; modern practice with pretrained backbones (including ConvNeXt) and transfer learning, the “what you actually do in 2026” companion to this chapter.
Tutorials, notes, and surveys
- timm (pytorch-image-models) — Ross Wightman et al. — free; an implementation collection containing the architectures in this chapter, with trained weights and results tables comparing them under consistent evaluation.
- A Recipe for Training Neural Networks — Andrej Karpathy — free; the debugging-first training discipline that Section 7.6 systematizes.
- Zoom In: An Introduction to Circuits — Olah et al. (2020), Distill — free; opens trained vision models to inspect the features and circuits they learn, useful perspective once you can train the architectures in this chapter.
Foundational papers
All free on arXiv or the proceedings site; these are the primary sources this chapter retells, worth reading in the original:
- ImageNet Classification with Deep Convolutional Neural Networks — Krizhevsky, Sutskever & Hinton (2012), NeurIPS — AlexNet (Section 7.1).
- Batch Normalization — Ioffe & Szegedy (2015) — read together with its critique in Section 7.3.
- Deep Residual Learning for Image Recognition — He, Zhang, Ren & Sun (2015) — ResNet (Section 7.4), the most-cited paper in deep learning.
- Bag of Tricks for Image Classification with CNNs — He et al. (2019) — the first systematic demonstration that recipe details rival architecture changes.
- ResNet Strikes Back — Wightman, Touvron & Jégou (2021) — a controlled analysis of training recipes and architecture behind Section 7.6.
- A ConvNet for the 2020s — Liu et al. (2022) — ConvNeXt (Section 7.7), a controlled ablation worth studying as method, not just result.
- ConvNets Match Vision Transformers at Scale — Smith et al. (2023) — the scaling-law resolution of the convnet-vs-transformer debate closing this chapter.