30  Tools for Deep Learning

This chapter covers the tools and systems used around deep learning models. It begins with reproducible notebook use (Section 30.1), hosted notebooks (Section 30.2), rented accelerators (Section 30.3), and local hardware (Section 30.4). It then surveys sources for models, datasets, papers, and benchmarks (Section 30.5), before introducing distributed training (Section 30.6) and model serving (Section 30.7). The final section explains how this book is built and how to contribute to it (Section 30.8).

Prices, model names, quotas, and library versions are dated to mid-2026 and must be checked before use. The durable principles are to establish memory fit before optimizing speed, account for bandwidth limits during generation, compare cost per completed result, and treat reproducibility as part of the workflow.

The generated utility and d2l API documents remain searchable HTML reference pages. They are not part of the teaching sequence or the PDF edition.

Resources and Further Reading

The references below extend this chapter’s practical arc — from working effectively on one machine to training and serving at scale. All are freely accessible online.

Systems and scaling

• How to Scale Your Model — Austin et al. (Google DeepMind, 2025) — free online book; from rooflines and TPU/GPU anatomy to parallelism and inference arithmetic, this is the quantitative companion to Section 30.4, Section 30.6, and Section 30.7 — a quantitative treatment of predicting performance before measuring it.
• The Ultra-Scale Playbook — Hugging Face — free; a hands-on guide to GPU-cluster training, based on thousands of instrumented runs on up to 512 GPUs; it extends the discussion in Section 30.6 with detailed memory budgets and parallelism trade-offs.
• Stanford CS336: Language Modeling from Scratch — free lectures and assignments; builds a language model end to end including the systems layer — tokenization through distributed training and inference — the university-course companion to this chapter.
• GPU MODE lecture series — free videos, slides, and notebooks, from CUDA fundamentals through FlashAttention and quantization; with material on kernel-level performance analysis and an active community.

Performance thinking, in blog-post form

• Making Deep Learning Go Brrrr From First Principles — Horace He — free; the compute-versus-bandwidth-versus-overhead taxonomy that underlies Section 30.4, in a concise article.
• Transformer Inference Arithmetic — kipply — free; KV-cache sizing and bandwidth-bound decoding worked out by hand — its methods support the KV-cache and bandwidth calculations in Section 30.7.
• Accelerating Generative AI with PyTorch: GPT, Fast — free, with the gpt-fast code; a ~10× inference speedup built step by step in under 1,000 lines of native PyTorch — compilation, quantization, speculative decoding, and tensor parallelism made concrete.
• Which GPU for Deep Learning? — Tim Dettmers — free; last updated in 2023; its hardware examples are dated, but its method for reasoning about GPU choice complements Section 30.4.

Surveys

• A Survey on Efficient Inference for Large Language Models — Zhou et al. (2024) — free; a taxonomy of data-, model-, and system-level inference optimization, with comparative experiments.
• A Survey on Large Language Model Acceleration Based on KV Cache Management — Li et al. (2025) — free; a survey of KV-cache management methods introduced in Section 30.7, with a maintained paper repository.

Staying current

• r/LocalLLaMA and the llama.cpp benchmark discussions — free; a source of release reports, quantization results, hardware measurements, and reproduction attempts (see Section 30.5 for the fuller information diet).
• Pro Git — Chacon & Straub — free book; chapters 1–3 and 6 cover the Git and pull-request concepts assumed by Section 30.8.