29  Tools for Deep Learning

The rest of this book teaches models; this chapter teaches the workshop around them. It follows the trajectory most readers actually take: run the book’s notebooks well (Section 29.1), for free in the cloud (Section 29.2), then on rented GPUs when the free tiers run out (Section 29.3) or on hardware of your own (Section 29.4). From there it widens to the ecosystem you will work inside — where models, datasets, papers, and benchmarks actually live (Section 29.5) — and to the systems questions that dominate practice at scale: training across many accelerators (Section 29.6) and serving models to users (Section 29.7). It closes with the machinery of this book itself and how to contribute to it (Section 29.8).

Two warnings apply throughout. Prices, model names, and library versions are quoted as of mid-2026 and will age; we date them so you can discount them, and each section says where to find current numbers. The underlying reasoning — memory fit before speed, bandwidth bounds on generation, cost per result rather than per hour, reproducibility as a workflow rather than a wish — is the part meant to last.

The generated utility and d2l API documents remain searchable HTML reference pages. They are not part of the teaching sequence or the PDF edition.

Resources and Further Reading

The references below extend this chapter’s practical arc — from working effectively on one machine to training and serving at scale. All are freely accessible online.

Systems and scaling

• How to Scale Your Model — Austin et al. (Google DeepMind, 2025) — free online book; from rooflines and TPU/GPU anatomy to parallelism and inference arithmetic, this is the quantitative companion to Section 29.4, Section 29.6, and Section 29.7 — the best single place to learn to predict performance before measuring it.
• The Ultra-Scale Playbook — Hugging Face — free; the definitive hands-on guide to training on GPU clusters, distilled from thousands of instrumented runs on up to 512 GPUs; picks up exactly where Section 29.6 stops, with the memory budgets and parallelism trade-offs worked out in detail.
• Stanford CS336: Language Modeling from Scratch — free lectures and assignments; builds a language model end to end including the systems layer — tokenization through distributed training and inference — the university-course companion to this chapter.
• GPU MODE lecture series — free videos, slides, and notebooks, from CUDA fundamentals through FlashAttention and quantization; where practitioners actually learn kernel-level thinking, with an active community attached.

Performance thinking, in blog-post form

• Making Deep Learning Go Brrrr From First Principles — Horace He — free; the compute-versus-bandwidth-versus-overhead taxonomy that underlies Section 29.4, in a few thousand vivid words.
• Transformer Inference Arithmetic — kipply — free; KV-cache sizing and bandwidth-bound decoding worked out by hand — every number in Section 29.7 can be re-derived with the methods here.
• Accelerating Generative AI with PyTorch: GPT, Fast — free, with the gpt-fast code; a ~10× inference speedup built step by step in under 1,000 lines of native PyTorch — compilation, quantization, speculative decoding, and tensor parallelism made concrete.
• Which GPU for Deep Learning? — Tim Dettmers — free; frozen since 2023, so ignore the part numbers, but still the classic exposition of how to reason about GPU choice — the ancestor of Section 29.4’s approach.

Surveys

• A Survey on Efficient Inference for Large Language Models — Zhou et al. (2024) — free; the standard taxonomy of data-, model-, and system-level inference optimization, with comparative experiments.
• A Survey on Large Language Model Acceleration Based on KV Cache Management — Li et al. (2025) — free; everything the field knows about the KV cache that Section 29.7 introduces, with a maintained paper repository.

Staying current

• r/LocalLLaMA and the llama.cpp benchmark discussions — free; where new models, quantizations, and hardware numbers surface first, with reproduction attempts attached (see Section 29.5 for the fuller information diet).
• Pro Git — Chacon & Straub — free book; chapters 1–3 and 6 cover everything Section 29.8 assumes about Git and pull requests, properly.