29.10  The d2l API Document

This section displays classes and functions (sorted alphabetically) in the d2l package, showing where they are defined in the book so you can find more detailed implementations and explanations. See also the source code on the GitHub repository.

29.10.1 Classes

PyTorch

class Accumulator

Accumulator(n)

For accumulating sums over n variables. Section 29.9

• add(*args)
• reset()

class Animator

Animator(xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), nrows=1, ncols=1, figsize=(3.5, 2.5))

For plotting data in animation. Section 29.9

• add(x, y)

class BERTEncoder(nn.Module)

BERTEncoder(vocab_size, num_hiddens, ffn_num_hiddens, num_heads, num_blks, dropout, max_len=1000, **kwargs)

BERT encoder. Section 17.9

• forward(tokens, segments, valid_lens)

class BERTModel(nn.Module)

BERTModel(vocab_size, num_hiddens, ffn_num_hiddens, num_heads, num_blks, dropout, max_len=1000)

The BERT model. Section 17.9

• forward(tokens, segments, valid_lens=None, pred_positions=None)

class BPETokenizer

BPETokenizer(vocab_size=1024, pattern=None, specials=('<pad>', '<bos>', '<eos>'))

Byte-level BPE tokenizer trained by iterated most-frequent-pair Section 8.2

• pad()
• bos()
• eos()
• train(text) — Learn vocab_size - 256 merges, most frequent pair first.
• encode(text, allow_special=False)
• decode(ids)
• from_tiktoken(name) — Load a published bytes->rank table (e.g. ‘gpt2’) into our encoder. Section 8.2
• to_tokens(ids)

class BananasDataset(torch.utils.data.Dataset)

BananasDataset(is_train)

A customized dataset to load the banana detection dataset. Section 19.6

class Benchmark

Benchmark(f, warmup=3, repeats=10, desc='time')

Time a callable: warmup, then device-synchronized average seconds. Section 13.1

class Classifier(d2l.Module)

The base class of classification models. Section 3.3

• validation_step(batch)
• accuracy(Y_hat, Y, averaged=True) — Compute the fraction of correct predictions. Section 3.3
• loss(Y_hat, Y, averaged=True)
• layer_summary(X_shape)

class DataModule(d2l.HyperParameters)

DataModule(root='../data', num_workers=4)

The base class of data. Section 2.2

• get_dataloader(train)
• train_dataloader()
• val_dataloader()
• get_tensorloader(tensors, train, indices=slice(0, None))

class Decoder(nn.Module)

The base decoder interface for the encoder-decoder architecture. Section 17.1.1

• init_state(enc_all_outputs, *args)
• forward(X, state)

class DotProductAttention(nn.Module)

DotProductAttention(dropout)

Scaled dot product attention. Section 10.2

• forward(queries, keys, values, valid_lens=None)

class Encoder(nn.Module)

The base encoder interface for the encoder-decoder architecture. Section 17.1.1

• forward(X, *args)

class EncoderDecoder(d2l.Classifier)

EncoderDecoder(encoder, decoder)

The base class for the encoder-decoder architecture. Section 17.1.1

• forward(enc_X, dec_X, *args)
• predict_step(batch, device, num_steps, save_attention_weights=False)

class FashionMNIST(d2l.DataModule)

FashionMNIST(batch_size=64, resize=(28, 28))

The Fashion-MNIST dataset. Section 3.2

• text_labels(indices) — Return text labels. Section 3.2
• get_dataloader(train)
• visualize(batch, nrows=1, ncols=8, labels=None)

class FeedForward(nn.Module)

FeedForward(num_hiddens, act='swiglu', bias=False)

Position-wise FFN: GELU MLP or SwiGLU at matched parameter count. Section 11.1

• forward(X)

class GPT(nn.Module)

GPT(vocab_size, num_hiddens=256, num_heads=8, num_blks=6, max_len=1024, pos='rope', norm='rms', act='swiglu', pre_norm=True, bias=False, dropout=0)

Decoder-only transformer language model built from configurable Section 11.2

• forward(X)
• generate(prefix, num_tokens, temperature=1.0, top_k=None) — Sample a continuation of the token-id list prefix. Section 11.2

class GQAAttention(nn.Module)

GQAAttention(num_hiddens, num_heads, num_kv_heads, bias=False, rope=False, causal=True)

Causal multi-head attention with num_kv_heads shared KV heads. Section 11.3

• forward(queries, keys, values, valid_lens=None)

class GRU(d2l.RNN)

GRU(num_inputs, num_hiddens, num_layers=1, dropout=0)

The multilayer GRU model implemented with high-level APIs. Section 12.1.3

class HyperParameters

The base class of hyperparameters. Section 2.2

• save_hyperparameters(ignore=None) — Save function arguments into class attributes. Section 29.9

class LSTM(d2l.RNN)

LSTM(num_inputs, num_hiddens, num_layers=1, dropout=0)

The multilayer LSTM model implemented with high-level APIs. Section 12.1

• forward(inputs, H_C=None)

class LSTMScratch(d2l.Module)

LSTMScratch(num_inputs, num_hiddens, sigma=0.01)

The long short-term memory (LSTM) cell implemented from scratch. Section 12.1

• forward(inputs, H_C=None)

class LeNet(d2l.Classifier)

LeNet(lr=0.1, num_classes=10)

The LeNet-5 model. Section 6.6

class LinearRegression(d2l.Module)

LinearRegression(lr)

The linear regression model implemented with high-level APIs. Section 2.5

• forward(X)
• loss(y_hat, y)
• configure_optimizers()
• get_w_b()

class LinearRegressionScratch(d2l.Module)

LinearRegressionScratch(num_inputs, lr, sigma=0.01)

The linear regression model implemented from scratch. Section 2.4

• forward(X)
• loss(y_hat, y)
• configure_optimizers()

class MTFraEng(d2l.DataModule)

MTFraEng(batch_size, num_steps=20, num_train=1024, num_val=128, vocab_size=4000)

The English-French dataset, tokenized with a shared byte-level BPE. Section 17.1.2

• build(src_sentences, tgt_sentences)
• get_dataloader(train)

class Mamba(d2l.Module)

Mamba(num_inputs, num_blocks=2, num_states=4, dropout=0)

A stack of Mamba blocks with the recurrent-cell interface. Section 12.3

• forward(X, state=None)
• step(X, state=None) — Advance the stack one token: X is (batch, d); one state per block. Section 12.3

class MambaBlock(nn.Module)

MambaBlock(num_hiddens, num_states=4, expand=2, conv_width=4, dropout=0)

Conv + SiLU + selective SSM, gated, inside a pre-norm residual. Section 12.3

• forward(X)
• step(X, state=None) — Advance one token, carrying (conv buffer, SSM state). Section 12.3

class MaskLM(nn.Module)

MaskLM(vocab_size, num_hiddens, **kwargs)

The masked language model task of BERT. Section 17.9

• forward(X, pred_positions)

class MaskedSoftmaxCELoss(nn.CrossEntropyLoss)

The softmax cross-entropy loss with masks. Section 29.9

• forward(pred, label, valid_len)

class Module(d2l.nn_Module, d2l.HyperParameters)

Module(plot_train_per_epoch=2, plot_valid_per_epoch=1)

The base class of models. Section 2.2

• loss(y_hat, y)
• forward(X)
• plot(key, value, train) — Plot a point in animation.
• training_step(batch)
• validation_step(batch)
• configure_optimizers()
• apply_init(inputs, init=None)

class MultiHeadAttention(d2l.Module)

MultiHeadAttention(num_hiddens, num_heads, dropout, bias=False, **kwargs)

Multi-head attention. Section 10.3

• forward(queries, keys, values, valid_lens)
• transpose_qkv(X) — Transposition for parallel computation of multiple attention heads. Section 10.3
• transpose_output(X) — Reverse the operation of transpose_qkv. Section 10.3

class NextSentencePred(nn.Module)

NextSentencePred(**kwargs)

The next sentence prediction task of BERT. Section 17.9

• forward(X)

class PatchEmbedding(nn.Module)

PatchEmbedding(img_size=96, patch_size=16, num_hiddens=512)

Image-to-sequence stem of the vision transformer. Section 11.5

• forward(X)

class ProgressBoard(d2l.HyperParameters)

ProgressBoard(xlabel=None, ylabel=None, xlim=None, ylim=None, xscale='linear', yscale='linear', ls=['-', '--', '-.', ':'], colors=['C0', 'C1', 'C2', 'C3'], fig=None, axes=None, figsize=(3.5, 2.5), display=True)

The board that plots data points in animation. Section 2.2

• draw(x, y, label, every_n=1) — Schedule the point (x, y) to be plotted under label. Section 29.9
• flush() — Wait for every scheduled point to be drawn, then render the final Section 29.9

class RNN(d2l.Module)

RNN(num_inputs, num_hiddens)

The RNN model implemented with high-level APIs. Section 8.5.5

• forward(inputs, H=None)

class RNNLM(d2l.RNNLMScratch)

The RNN-based language model implemented with high-level APIs. Section 8.5.5

• init_params()
• embedding(X)
• output_layer(hiddens)

class RNNLMScratch(d2l.Classifier)

RNNLMScratch(rnn, vocab_size, lr=0.01)

The RNN-based language model implemented from scratch. Section 8.5

• init_params()
• training_step(batch)
• validation_step(batch)
• embedding(X)
• output_layer(rnn_outputs)
• forward(X, state=None)
• predict(prefix, num_tokens, tok, device=None, temperature=0.0, rng=None)

class RNNScratch(d2l.Module)

RNNScratch(num_inputs, num_hiddens, sigma=0.01)

The RNN model implemented from scratch. Section 8.5

• forward(inputs, state=None)

class RandomGenerator

RandomGenerator(sampling_weights)

Randomly draw among {1, …, n} according to n sampling weights. Section 17.4

• draw()

class ResNeXtBlock(nn.Module)

ResNeXtBlock(num_channels, groups, bot_mul, use_1x1conv=False, strides=1)

The ResNeXt block. Section 7.4

• forward(X)

class Residual(nn.Module)

Residual(num_channels, use_1x1conv=False, strides=1)

The Residual block of ResNet models. Section 7.4

• forward(X)

class S4D(nn.Module)

S4D(num_hiddens, num_states=4, dt_min=0.001, dt_max=0.1)

A diagonal state space layer: one SSM per feature channel. Section 12.2

• forward(u)
• step(u, x=None) — Advance one token: u is (batch, H); x is the (batch, H, N) state. Section 12.2

class SGD(d2l.HyperParameters)

SGD(params, lr)

Minibatch stochastic gradient descent. Section 2.4

• step()
• zero_grad()

class SNLIDataset(torch.utils.data.Dataset)

SNLIDataset(dataset, num_steps, vocab=None)

A customized dataset to load the SNLI dataset. Section 18.4

class SelectiveSSM(nn.Module)

SelectiveSSM(num_hiddens, num_states=4, dt_min=0.001, dt_max=0.1)

A diagonal SSM whose step size, input matrix, and read-out are Section 12.3

• forward(u)
• step(u, x=None) — Advance one token: u is (batch, H); x is the (batch, H, N) state. Section 12.3

class Seq2Seq(d2l.EncoderDecoder)

Seq2Seq(encoder, decoder, tgt_pad, lr)

The RNN encoder-decoder for sequence-to-sequence learning. Section 17.1.2

• validation_step(batch)
• configure_optimizers()

class Seq2SeqEncoder(d2l.Encoder)

Seq2SeqEncoder(vocab_size, embed_size, num_hiddens, num_layers, dropout=0)

The RNN encoder for sequence-to-sequence learning. Section 17.1.2

• forward(X, *args)

class SoftmaxRegression(d2l.Classifier)

SoftmaxRegression(num_outputs, lr)

The softmax regression model. Section 3.5

• forward(X)

class SyntheticRegressionData(d2l.DataModule)

SyntheticRegressionData(w, b, noise=0.01, num_train=1000, num_val=1000, batch_size=32)

Synthetic data for linear regression. Section 2.3

• get_dataloader(train)

class TimeMachine(d2l.DataModule)

TimeMachine(batch_size, num_steps, num_train=10000, num_val=5000, tokenization='bpe', vocab_size=1024)

The Time Machine dataset. Section 8.2

• build(raw_text, vocab=None)
• get_dataloader(train)

class Timer

Record multiple running times. Section 9.4

• start() — Start the timer.
• stop() — Stop the timer and record the time in a list.
• avg() — Return the average time.
• sum() — Return the sum of time.
• cumsum() — Return the accumulated time.

class TinyCharLM(nn.Module)

TinyCharLM(vocab_size, num_hiddens=128, num_heads=4, num_blks=2, pos='rope', max_len=512, bias=False)

Attention-only character-level language model. Section 10.4

• forward(X)
• attention_weights(X) — Per-block attention maps, each (batch, num_heads, T, T).

class TinyLM(nn.Module)

TinyLM(vocab_size, d_model=128, num_heads=2, num_blks=2, max_len=64)

A small decoder-only transformer language model. Section 9.6

• attention(blk, X)
• forward(X)

class TokenEmbedding

TokenEmbedding(embedding_name)

Token Embedding. Section 17.8

class Trainer(d2l.HyperParameters)

Trainer(max_epochs, num_gpus=0, gradient_clip_val=0)

The base class for training models with data. Section 2.2

• prepare_data(data)
• fit(model, data)
• fit_epoch()
• prepare_batch(batch)
• prepare_model(model)
• clip_gradients(grad_clip_val, model)

class TransformerBlock(nn.Module)

TransformerBlock(num_hiddens, num_heads, dropout=0, norm='rms', act='swiglu', pre_norm=True, bias=False, attn_factory=None, ffn_factory=None)

Configurable transformer block: attention + FFN on a residual Section 11.1

• forward(X, valid_lens=None)

class VOCSegDataset(torch.utils.data.Dataset)

VOCSegDataset(is_train, crop_size, voc_dir)

A customized dataset to load the VOC dataset. Section 19.9

• normalize_image(img)
• filter(imgs)

class ViT(d2l.Classifier)

ViT(img_size, patch_size, num_hiddens, mlp_num_hiddens, num_heads, num_blks, emb_dropout, blk_dropout, lr=0.1, use_bias=False, num_classes=10)

Vision transformer. Section 11.5

• forward(X)

class ViTBlock(nn.Module)

ViTBlock(num_hiddens, mlp_num_hiddens, num_heads, dropout, use_bias=False)

Pre-norm transformer block with a GELU MLP. Section 11.5

• forward(X, valid_lens=None)

class Vocab

Vocab(tokens=[], min_freq=0, reserved_tokens=[])

Vocabulary for text. Section 8.2

• to_tokens(indices)
• unk()

TensorFlow

class Accumulator

Accumulator(n)

For accumulating sums over n variables. Section 29.9

• add(*args)
• reset()

class Animator

Animator(xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), nrows=1, ncols=1, figsize=(3.5, 2.5))

For plotting data in animation. Section 29.9

• add(x, y)

class BERTEncoder(keras.layers.Layer)

BERTEncoder(vocab_size, num_hiddens, ffn_num_hiddens, num_heads, num_blks, dropout, max_len=1000, **kwargs)

BERT encoder. Section 17.9

• call(tokens, segments, valid_lens, training=False, **kwargs)

class BERTModel(keras.Model)

BERTModel(vocab_size, num_hiddens, ffn_num_hiddens, num_heads, num_blks, dropout, max_len=1000)

The BERT model. Section 17.9

• call(tokens, segments, valid_lens=None, pred_positions=None, training=False, **kwargs)

class BPETokenizer

BPETokenizer(vocab_size=1024, pattern=None, specials=('<pad>', '<bos>', '<eos>'))

Byte-level BPE tokenizer trained by iterated most-frequent-pair Section 8.2

• pad()
• bos()
• eos()
• train(text) — Learn vocab_size - 256 merges, most frequent pair first.
• encode(text, allow_special=False)
• decode(ids)
• from_tiktoken(name) — Load a published bytes->rank table (e.g. ‘gpt2’) into our encoder. Section 8.2
• to_tokens(ids)

class BananasDataset

BananasDataset(is_train)

A customized dataset to load the banana detection dataset. Section 19.6

class Classifier(d2l.Module)

The base class of classification models. Section 3.3

• validation_step(batch)
• accuracy(Y_hat, Y, averaged=True) — Compute the fraction of correct predictions. Section 3.3
• loss(Y_hat, Y, averaged=True)
• layer_summary(X_shape)

class DataModule(d2l.HyperParameters)

DataModule(root='../data', num_workers=4)

The base class of data. Section 2.2

• get_dataloader(train)
• train_dataloader()
• val_dataloader()
• get_tensorloader(tensors, train, indices=slice(0, None))

class Decoder(tf.keras.layers.Layer)

The base decoder interface for the encoder-decoder architecture. Section 17.1.1

• init_state(enc_all_outputs, *args)
• call(X, state)

class Encoder(tf.keras.layers.Layer)

The base encoder interface for the encoder-decoder architecture. Section 17.1.1

• call(X, *args)

class EncoderDecoder(d2l.Classifier)

EncoderDecoder(encoder, decoder)

The base class for the encoder-decoder architecture. Section 17.1.1

• call(enc_X, dec_X, *args, training=None)
• predict_step(batch, device, num_steps, save_attention_weights=False)

class FashionMNIST(d2l.DataModule)

FashionMNIST(batch_size=64, resize=(28, 28))

The Fashion-MNIST dataset. Section 3.2

• text_labels(indices) — Return text labels. Section 3.2
• get_dataloader(train)
• visualize(batch, nrows=1, ncols=8, labels=None)

class HyperParameters

The base class of hyperparameters. Section 2.2

• save_hyperparameters(ignore=None) — Save function arguments into class attributes. Section 29.9

class LeNet(d2l.Classifier)

LeNet(lr=0.1, num_classes=10)

The LeNet-5 model. Section 6.6

class LinearRegression(d2l.Module)

LinearRegression(lr)

The linear regression model implemented with high-level APIs. Section 2.5

• forward(X)
• loss(y_hat, y)
• configure_optimizers()
• get_w_b()

class LinearRegressionScratch(d2l.Module)

LinearRegressionScratch(num_inputs, lr, sigma=0.01)

The linear regression model implemented from scratch. Section 2.4

• forward(X)
• loss(y_hat, y)
• configure_optimizers()

class MTFraEng(d2l.DataModule)

MTFraEng(batch_size, num_steps=20, num_train=1024, num_val=128, vocab_size=4000)

The English-French dataset, tokenized with a shared byte-level BPE. Section 17.1.2

• build(src_sentences, tgt_sentences)
• get_dataloader(train)

class MaskLM(keras.layers.Layer)

MaskLM(vocab_size, num_hiddens, **kwargs)

The masked language model task of BERT. Section 17.9

• call(X, pred_positions, **kwargs)

class MaskedSoftmaxCELoss(tf.keras.losses.Loss)

MaskedSoftmaxCELoss(valid_len)

The softmax cross-entropy loss with masks. Section 29.9

• call(label, pred)

class Module(d2l.nn_Module, d2l.HyperParameters)

Module(plot_train_per_epoch=2, plot_valid_per_epoch=1)

The base class of models. Section 2.2

• loss(y_hat, y)
• forward(X)
• call(X, *args, training=None, **kwargs)
• plot(key, value, train) — Plot a point in animation.
• training_step(batch)
• validation_step(batch)
• configure_optimizers()

class NextSentencePred(keras.layers.Layer)

NextSentencePred(**kwargs)

The next sentence prediction task of BERT. Section 17.9

• call(X, **kwargs)

class ProgressBoard(d2l.HyperParameters)

ProgressBoard(xlabel=None, ylabel=None, xlim=None, ylim=None, xscale='linear', yscale='linear', ls=['-', '--', '-.', ':'], colors=['C0', 'C1', 'C2', 'C3'], fig=None, axes=None, figsize=(3.5, 2.5), display=True)

The board that plots data points in animation. Section 2.2

• draw(x, y, label, every_n=1) — Schedule the point (x, y) to be plotted under label. Section 29.9
• flush() — Wait for every scheduled point to be drawn, then render the final Section 29.9

class RNN(d2l.Module)

RNN(num_inputs, num_hiddens)

The RNN model implemented with high-level APIs. Section 8.5.5

• forward(inputs, H=None)

class RNNLM(d2l.RNNLMScratch)

The RNN-based language model implemented with high-level APIs. Section 8.5.5

• init_params()
• embedding(X)
• output_layer(hiddens)

class RNNLMScratch(d2l.Classifier)

RNNLMScratch(rnn, vocab_size, lr=0.01)

The RNN-based language model implemented from scratch. Section 8.5

• init_params()
• embedding(X)
• output_layer(rnn_outputs)
• forward(X, state=None)
• predict(prefix, num_tokens, tok, device=None, temperature=0.0, rng=None)

class RNNScratch(d2l.Module)

RNNScratch(num_inputs, num_hiddens, sigma=0.01)

The RNN model implemented from scratch. Section 8.5

• forward(inputs, state=None)

class RandomGenerator

RandomGenerator(sampling_weights)

Randomly draw among {1, …, n} according to n sampling weights. Section 17.4

• draw()

class ResNeXtBlock(tf.keras.Model)

ResNeXtBlock(num_channels, groups, bot_mul, use_1x1conv=False, strides=1)

The ResNeXt block. Section 7.4

• call(X)

class Residual(tf.keras.Model)

Residual(num_channels, use_1x1conv=False, strides=1)

The Residual block of ResNet models. Section 7.4

• call(X)

class SGD(d2l.HyperParameters)

SGD(lr)

Minibatch stochastic gradient descent. Section 2.4

• apply_gradients(grads_and_vars)

class SNLIDataset

SNLIDataset(dataset, num_steps, vocab=None)

A customized dataset to load the SNLI dataset. Section 18.4

class Seq2Seq(d2l.EncoderDecoder)

Seq2Seq(encoder, decoder, tgt_pad, lr)

The RNN encoder-decoder for sequence-to-sequence learning. Section 17.1.2

• validation_step(batch)
• configure_optimizers()

class Seq2SeqEncoder(d2l.Encoder)

Seq2SeqEncoder(vocab_size, embed_size, num_hiddens, num_layers, dropout=0)

The RNN encoder for sequence-to-sequence learning. Section 17.1.2

• call(X, *args)

class SoftmaxRegression(d2l.Classifier)

SoftmaxRegression(num_outputs, lr)

The softmax regression model. Section 3.5

• forward(X)

class SyntheticRegressionData(d2l.DataModule)

SyntheticRegressionData(w, b, noise=0.01, num_train=1000, num_val=1000, batch_size=32)

Synthetic data for linear regression. Section 2.3

• get_dataloader(train)

class TimeMachine(d2l.DataModule)

TimeMachine(batch_size, num_steps, num_train=10000, num_val=5000, tokenization='bpe', vocab_size=1024)

The Time Machine dataset. Section 8.2

• build(raw_text, vocab=None)
• get_dataloader(train)

class Timer

Record multiple running times. Section 9.4

• start() — Start the timer.
• stop() — Stop the timer and record the time in a list.
• avg() — Return the average time.
• sum() — Return the sum of time.
• cumsum() — Return the accumulated time.

class TokenEmbedding

TokenEmbedding(embedding_name)

Token Embedding. Section 17.8

class TrainCallback(tf.keras.callbacks.Callback)

TrainCallback(net, train_iter, test_iter, num_epochs, device_name)

A callback to visualize the training progress. Section 29.9

• on_epoch_begin(epoch, logs=None)
• on_epoch_end(epoch, logs)

class Trainer(d2l.HyperParameters)

Trainer(max_epochs, num_gpus=0, gradient_clip_val=0)

The base class for training models with data. Section 2.2

• prepare_data(data)
• prepare_model(model)
• fit(model, data)
• fit_epoch()
• prepare_batch(batch)
• clip_gradients(grad_clip_val, grads)

class VOCSegDataset

VOCSegDataset(is_train, crop_size, voc_dir)

A customized dataset to load the VOC dataset. Section 19.9

• normalize_image(img)
• filter(imgs)

class Vocab

Vocab(tokens=[], min_freq=0, reserved_tokens=[])

Vocabulary for text. Section 8.2

• to_tokens(indices)
• unk()

JAX

class Accumulator

Accumulator(n)

For accumulating sums over n variables. Section 29.9

• add(*args)
• reset()

class Animator

Animator(xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), nrows=1, ncols=1, figsize=(3.5, 2.5))

For plotting data in animation. Section 29.9

• add(x, y)

class ArrayDataLoader

ArrayDataLoader(*args, batch_size=32, shuffle=False, drop_last=False, **kwargs)

A simple data loader for JAX that batches arrays or dataset objects. Section 29.9

class BERTEncoder(nnx.Module)

BERTEncoder(vocab_size, num_hiddens, ffn_num_hiddens, num_heads, num_blks, dropout, max_len=1000, rngs=None)

BERT encoder. Section 17.9

class BERTModel(nnx.Module)

BERTModel(vocab_size, num_hiddens, ffn_num_hiddens, num_heads, num_blks, dropout, max_len=1000, rngs=None)

The BERT model. Section 17.9

class BPETokenizer

BPETokenizer(vocab_size=1024, pattern=None, specials=('<pad>', '<bos>', '<eos>'))

Byte-level BPE tokenizer trained by iterated most-frequent-pair Section 8.2

• pad()
• bos()
• eos()
• train(text) — Learn vocab_size - 256 merges, most frequent pair first.
• encode(text, allow_special=False)
• decode(ids)
• from_tiktoken(name) — Load a published bytes->rank table (e.g. ‘gpt2’) into our encoder. Section 8.2
• to_tokens(ids)

class BananasDataset

BananasDataset(is_train)

A customized dataset to load the banana detection dataset. Section 19.6

class Benchmark

Benchmark(f, warmup=3, repeats=10, desc='time')

Time a callable: warmup, then device-synchronized average seconds. Section 13.1

class Classifier(d2l.Module)

The base class of classification models. Section 3.3

• validation_step(batch)
• accuracy(Y_hat, Y, averaged=True) — Compute the fraction of correct predictions. Section 3.3
• loss(Y_hat, Y, averaged=True)
• layer_summary(X_shape)

class DataModule(d2l.HyperParameters)

DataModule(root='../data', num_workers=4)

The base class of data. Section 2.2

• get_dataloader(train)
• train_dataloader()
• val_dataloader()
• get_tensorloader(tensors, train, indices=slice(0, None))

class Decoder(nnx.Module)

The base decoder interface for the encoder-decoder architecture. Section 17.1.1

• init_state(enc_all_outputs, *args)

class DotProductAttention(nnx.Module)

DotProductAttention(dropout, rngs=None)

Scaled dot product attention. Section 10.2

class Encoder(nnx.Module)

The base encoder interface for the encoder-decoder architecture. Section 17.1.1

class EncoderDecoder(d2l.Classifier)

EncoderDecoder(encoder, decoder)

The base class for the encoder-decoder architecture. Section 17.1.1

• forward(enc_X, dec_X, *args)
• predict_step(batch, num_steps, save_attention_weights=False)

class FashionMNIST(d2l.DataModule)

FashionMNIST(batch_size=64, resize=(28, 28))

The Fashion-MNIST dataset. Section 3.2

• text_labels(indices) — Return text labels. Section 3.2
• get_dataloader(train)
• visualize(batch, nrows=1, ncols=8, labels=None)

class FeedForward(nnx.Module)

FeedForward(num_hiddens, act='swiglu', bias=False, rngs=None)

Position-wise FFN: GELU MLP or SwiGLU at matched parameter count. Section 11.1

class GPT(nnx.Module)

GPT(vocab_size, num_hiddens=256, num_heads=8, num_blks=6, max_len=1024, pos='rope', norm='rms', act='swiglu', pre_norm=True, bias=False, dropout=0, rngs=None)

Decoder-only transformer language model built from configurable Section 11.2

• generate(prefix, num_tokens, seed=0, temperature=1.0, top_k=None) — Sample a continuation of the token-id list prefix. Section 11.2

class GQAAttention(nnx.Module)

GQAAttention(num_hiddens, num_heads, num_kv_heads, bias=False, rope=False, causal=True, rngs=None)

Causal multi-head attention with num_kv_heads shared KV heads. Section 11.3

class GRU(d2l.RNN)

GRU(num_inputs, num_hiddens, num_layers=1, dropout=0, rngs=None)

The multilayer GRU model implemented with high-level APIs. Section 12.1.3

class HyperParameters

The base class of hyperparameters. Section 2.2

• save_hyperparameters(ignore=None) — Save function arguments into class attributes. Section 29.9

class LSTM(d2l.RNN)

LSTM(num_inputs, num_hiddens, num_layers=1, dropout=0, rngs=None)

The multilayer LSTM model implemented with high-level APIs. Section 12.1

class LSTMScratch(nnx.Module)

LSTMScratch(num_inputs, num_hiddens, sigma=0.01, rngs=None)

The long short-term memory (LSTM) cell implemented from scratch. Section 12.1

class LeNet(d2l.Classifier)

LeNet(lr=0.1, num_classes=10, kernel_init=None, rngs=None)

The LeNet-5 model. Section 6.6

class LinearRegression(d2l.Module)

LinearRegression(num_inputs, lr, rngs=None)

The linear regression model implemented with high-level APIs. Section 2.5

• forward(X)
• loss(y_hat, y)
• configure_optimizers()
• get_w_b()

class LinearRegressionScratch(d2l.Module)

LinearRegressionScratch(num_inputs, lr, sigma=0.01, rngs=None)

The linear regression model implemented from scratch. Section 2.4

• forward(X)
• loss(y_hat, y)
• configure_optimizers()

class MTFraEng(d2l.DataModule)

MTFraEng(batch_size, num_steps=20, num_train=1024, num_val=128, vocab_size=4000)

The English-French dataset, tokenized with a shared byte-level BPE. Section 17.1.2

• build(src_sentences, tgt_sentences)
• get_dataloader(train)

class Mamba(nnx.Module)

Mamba(num_inputs, num_blocks=2, num_states=4, dropout=0, rngs=None)

A stack of Mamba blocks with the recurrent-cell interface. Section 12.3

• step(X, state=None) — Advance the stack one token: X is (batch, d); one state per block. Section 12.3

class MambaBlock(nnx.Module)

MambaBlock(num_hiddens, num_states=4, expand=2, conv_width=4, dropout=0, rngs=None)

Conv + SiLU + selective SSM, gated, inside a pre-norm residual. Section 12.3

• step(X, state=None) — Advance one token, carrying (conv buffer, SSM state). Section 12.3

class MaskLM(nnx.Module)

MaskLM(vocab_size, num_hiddens, rngs=None)

The masked language model task of BERT. Section 17.9

class Module(d2l.nn_Module, d2l.HyperParameters)

Module(plot_train_per_epoch=2, plot_valid_per_epoch=1)

The base class of models. Section 2.2

• loss(y_hat, y)
• forward(X, *args, **kwargs)
• plot(key, value, train) — Plot a point in animation.
• training_step(batch)
• validation_step(batch)
• configure_optimizers()

class MultiHeadAttention(nnx.Module)

MultiHeadAttention(num_hiddens, num_heads, dropout, bias=False, rngs=None)

Multi-head attention. Section 10.3

• transpose_qkv(X) — Transposition for parallel computation of multiple attention heads. Section 10.3
• transpose_output(X) — Reverse the operation of transpose_qkv. Section 10.3

class NextSentencePred(nnx.Module)

NextSentencePred(num_hiddens, rngs=None)

The next sentence prediction task of BERT. Section 17.9

class PatchEmbedding(nnx.Module)

PatchEmbedding(img_size=96, patch_size=16, num_hiddens=512, num_channels=3, rngs=None)

Image-to-sequence stem of the vision transformer. Section 11.5

class ProgressBoard(d2l.HyperParameters)

ProgressBoard(xlabel=None, ylabel=None, xlim=None, ylim=None, xscale='linear', yscale='linear', ls=['-', '--', '-.', ':'], colors=['C0', 'C1', 'C2', 'C3'], fig=None, axes=None, figsize=(3.5, 2.5), display=True)

The board that plots data points in animation. Section 2.2

• draw(x, y, label, every_n=1) — Schedule the point (x, y) to be plotted under label. Section 29.9
• flush() — Wait for every scheduled point to be drawn, then render the final Section 29.9

class RNN(nnx.Module)

RNN(num_inputs, num_hiddens, rngs=None)

The RNN model implemented with high-level APIs. Section 8.5.5

class RNNLM(d2l.RNNLMScratch)

RNNLM(rnn, vocab_size, lr=0.01, rngs=None)

The RNN-based language model implemented with high-level APIs. Section 8.5.5

• embedding(X)
• output_layer(hiddens)

class RNNLMScratch(d2l.Classifier)

RNNLMScratch(rnn, vocab_size, lr=0.01, rngs=None)

The RNN-based language model implemented from scratch. Section 8.5

• training_step(batch)
• validation_step(batch)
• plot(key, value, train)
• embedding(X)
• output_layer(rnn_outputs)
• forward(X, state=None)
• predict(prefix, num_tokens, tok, device=None, temperature=0.0, rng=None)

class RNNScratch(nnx.Module)

RNNScratch(num_inputs, num_hiddens, sigma=0.01, rngs=None)

The RNN model implemented from scratch. Section 8.5

class RandomGenerator

RandomGenerator(sampling_weights)

Randomly draw among {1, …, n} according to n sampling weights. Section 17.4

• draw()

class ResNeXtBlock(nnx.Module)

ResNeXtBlock(num_channels, groups, bot_mul, use_1x1conv=False, strides=(1, 1), in_channels=None, rngs=None)

The ResNeXt block. Section 7.4

class ResNet18(nnx.Module)

ResNet18(num_classes=10, rngs=None)

A slightly modified ResNet-18 (small stem, no max-pool). Section 13.6

class Residual(nnx.Module)

Residual(num_channels, use_1x1conv=False, strides=(1, 1), in_channels=None, rngs=None)

The Residual block of ResNet models. Section 7.4

class S4D(nnx.Module)

S4D(num_hiddens, num_states=4, dt_min=0.001, dt_max=0.1, rngs=None)

A diagonal state space layer: one SSM per feature channel. Section 12.2

• step(u, x=None) — Advance one token: u is (batch, H); x is the (batch, H, N) state. Section 12.2

class SGD(d2l.HyperParameters)

SGD(lr)

Minibatch stochastic gradient descent. Section 2.4

• init(params)
• update(updates, state, params=None)

class SNLIDataset

SNLIDataset(dataset, num_steps, vocab=None)

A customized dataset to load the SNLI dataset. Section 18.4

class SelectiveSSM(nnx.Module)

SelectiveSSM(num_hiddens, num_states=4, dt_min=0.001, dt_max=0.1, rngs=None)

A diagonal SSM whose step size, input matrix, and read-out are Section 12.3

• step(u, x=None) — Advance one token: u is (batch, H); x is the (batch, H, N) state. Section 12.3

class Seq2Seq(d2l.EncoderDecoder)

Seq2Seq(encoder, decoder, tgt_pad, lr)

The RNN encoder-decoder for sequence-to-sequence learning. Section 17.1.2

• validation_step(batch)
• configure_optimizers()

class Seq2SeqEncoder(d2l.Encoder)

Seq2SeqEncoder(vocab_size, embed_size, num_hiddens, num_layers, dropout=0, rngs=None)

The RNN encoder for sequence-to-sequence learning. Section 17.1.2

class SyntheticRegressionData(d2l.DataModule)

SyntheticRegressionData(w, b, noise=0.01, num_train=1000, num_val=1000, batch_size=32, key=None)

Synthetic data for linear regression. Section 2.3

• get_dataloader(train)

class TensorFlowDataLoader

TensorFlowDataLoader(dataset)

Expose a tf.data.Dataset as re-iterable NumPy batches. Section 2.3

class TimeMachine(d2l.DataModule)

TimeMachine(batch_size, num_steps, num_train=10000, num_val=5000, tokenization='bpe', vocab_size=1024)

The Time Machine dataset. Section 8.2

• build(raw_text, vocab=None)
• get_dataloader(train)

class Timer

Record multiple running times. Section 9.4

• start() — Start the timer.
• stop() — Stop the timer and record the time in a list.
• avg() — Return the average time.
• sum() — Return the sum of time.
• cumsum() — Return the accumulated time.

class TinyCharLM(nnx.Module)

TinyCharLM(vocab_size, num_hiddens=128, num_heads=4, num_blks=2, pos='rope', max_len=512, bias=False, rngs=None)

Attention-only character-level language model. Section 10.4

• attention_weights(X) — Per-block attention maps, each (batch, num_heads, T, T).

class TinyLM(nnx.Module)

TinyLM(vocab_size, d_model=128, num_heads=2, num_blks=2, max_len=64, rngs=None)

A small decoder-only transformer language model. Section 9.6

• attention(blk, X)

class TokenEmbedding

TokenEmbedding(embedding_name)

Token Embedding. Section 17.8

class Trainer(d2l.HyperParameters)

Trainer(max_epochs, num_gpus=0, gradient_clip_val=0)

The base class for training models with data. Section 2.2

• prepare_data(data)
• prepare_model(model)
• fit(model, data)
• fit_epoch()
• prepare_batch(batch)
• clip_gradients(grad_clip_val, grads)

class TransformerBlock(nnx.Module)

TransformerBlock(num_hiddens, num_heads, dropout=0, norm='rms', act='swiglu', pre_norm=True, bias=False, attn_factory=None, ffn_factory=None, rngs=None)

Configurable transformer block: attention + FFN on a residual Section 11.1

class VOCSegDataset

VOCSegDataset(is_train, crop_size, voc_dir)

A customized dataset to load the VOC dataset. Section 19.9

• normalize_image(img)
• filter(imgs)

class ViT(d2l.Classifier)

ViT(img_size, patch_size, num_hiddens, mlp_num_hiddens, num_heads, num_blks, emb_dropout, blk_dropout, lr=0.1, use_bias=False, num_classes=10, num_channels=1, rngs=None)

Vision transformer. Section 11.5

• forward(X)

class ViTBlock(nnx.Module)

ViTBlock(num_hiddens, mlp_num_hiddens, num_heads, dropout, use_bias=False, rngs=None)

Pre-norm transformer block with a GELU MLP. Section 11.5

class Vocab

Vocab(tokens=[], min_freq=0, reserved_tokens=[])

Vocabulary for text. Section 8.2

• to_tokens(indices)
• unk()

MXNet

class Accumulator

Accumulator(n)

For accumulating sums over n variables. Section 29.9

• add(*args)
• reset()

class Animator

Animator(xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), nrows=1, ncols=1, figsize=(3.5, 2.5))

For plotting data in animation. Section 29.9

• add(x, y)

class BERTEncoder(nn.Block)

BERTEncoder(vocab_size, num_hiddens, ffn_num_hiddens, num_heads, num_blks, dropout, max_len=1000)

BERT encoder. Section 17.9

• forward(tokens, segments, valid_lens)

class BERTModel(nn.Block)

BERTModel(vocab_size, num_hiddens, ffn_num_hiddens, num_heads, num_blks, dropout, max_len=1000)

The BERT model. Section 17.9

• forward(tokens, segments, valid_lens=None, pred_positions=None)

class BPETokenizer

BPETokenizer(vocab_size=1024, pattern=None, specials=('<pad>', '<bos>', '<eos>'))

Byte-level BPE tokenizer trained by iterated most-frequent-pair Section 8.2

• pad()
• bos()
• eos()
• train(text) — Learn vocab_size - 256 merges, most frequent pair first.
• encode(text, allow_special=False)
• decode(ids)
• from_tiktoken(name) — Load a published bytes->rank table (e.g. ‘gpt2’) into our encoder. Section 8.2
• to_tokens(ids)

class BananasDataset(gluon.data.Dataset)

BananasDataset(is_train)

A customized dataset to load the banana detection dataset. Section 19.6

class Classifier(d2l.Module)

The base class of classification models. Section 3.3

• validation_step(batch)
• accuracy(Y_hat, Y, averaged=True) — Compute the fraction of correct predictions. Section 3.3
• loss(Y_hat, Y, averaged=True)
• layer_summary(X_shape)

class DataModule(d2l.HyperParameters)

DataModule(root='../data', num_workers=4)

The base class of data. Section 2.2

• get_dataloader(train)
• train_dataloader()
• val_dataloader()
• get_tensorloader(tensors, train, indices=slice(0, None))

class Decoder(nn.Block)

The base decoder interface for the encoder-decoder architecture. Section 17.1.1

• init_state(enc_all_outputs, *args)
• forward(X, state)

class Encoder(nn.Block)

The base encoder interface for the encoder-decoder architecture. Section 17.1.1

• forward(X, *args)

class EncoderDecoder(d2l.Classifier)

EncoderDecoder(encoder, decoder)

The base class for the encoder-decoder architecture. Section 17.1.1

• forward(enc_X, dec_X, *args)
• predict_step(batch, device, num_steps, save_attention_weights=False)

class FashionMNIST(d2l.DataModule)

FashionMNIST(batch_size=64, resize=(28, 28))

The Fashion-MNIST dataset. Section 3.2

• text_labels(indices) — Return text labels. Section 3.2
• get_dataloader(train)
• visualize(batch, nrows=1, ncols=8, labels=None)

class HyperParameters

The base class of hyperparameters. Section 2.2

• save_hyperparameters(ignore=None) — Save function arguments into class attributes. Section 29.9

class LeNet(d2l.Classifier)

LeNet(lr=0.1, num_classes=10)

The LeNet-5 model. Section 6.6

class LinearRegression(d2l.Module)

LinearRegression(lr)

The linear regression model implemented with high-level APIs. Section 2.5

• forward(X)
• loss(y_hat, y)
• configure_optimizers()
• get_w_b()

class LinearRegressionScratch(d2l.Module)

LinearRegressionScratch(num_inputs, lr, sigma=0.01)

The linear regression model implemented from scratch. Section 2.4

• forward(X)
• loss(y_hat, y)
• configure_optimizers()

class MTFraEng(d2l.DataModule)

MTFraEng(batch_size, num_steps=20, num_train=1024, num_val=128, vocab_size=4000)

The English-French dataset, tokenized with a shared byte-level BPE. Section 17.1.2

• build(src_sentences, tgt_sentences)
• get_dataloader(train)

class MaskLM(nn.Block)

MaskLM(vocab_size, num_hiddens)

The masked language model task of BERT. Section 17.9

• forward(X, pred_positions)

class MaskedSoftmaxCELoss(gluon.loss.SoftmaxCELoss)

The softmax cross-entropy loss with masks. Section 29.9

• forward(pred, label, valid_len)

class Module(d2l.nn_Module, d2l.HyperParameters)

Module(plot_train_per_epoch=2, plot_valid_per_epoch=1)

The base class of models. Section 2.2

• loss(y_hat, y)
• forward(X)
• plot(key, value, train) — Plot a point in animation.
• training_step(batch)
• validation_step(batch)
• configure_optimizers()
• get_scratch_params()
• parameters()
• set_scratch_params_device(device)

class NextSentencePred(nn.Block)

The next sentence prediction task of BERT. Section 17.9

• forward(X)

class ProgressBoard(d2l.HyperParameters)

ProgressBoard(xlabel=None, ylabel=None, xlim=None, ylim=None, xscale='linear', yscale='linear', ls=['-', '--', '-.', ':'], colors=['C0', 'C1', 'C2', 'C3'], fig=None, axes=None, figsize=(3.5, 2.5), display=True)

The board that plots data points in animation. Section 2.2

• draw(x, y, label, every_n=1) — Schedule the point (x, y) to be plotted under label. Section 29.9
• flush() — Wait for every scheduled point to be drawn, then render the final Section 29.9

class RNN(d2l.Module)

RNN(num_inputs, num_hiddens)

The RNN model implemented with high-level APIs. Section 8.5.5

• forward(inputs, H=None)

class RNNLM(d2l.RNNLMScratch)

The RNN-based language model implemented with high-level APIs. Section 8.5.5

• init_params()
• embedding(X)
• output_layer(hiddens)

class RNNLMScratch(d2l.Classifier)

RNNLMScratch(rnn, vocab_size, lr=0.01)

The RNN-based language model implemented from scratch. Section 8.5

• init_params()
• training_step(batch)
• validation_step(batch)
• embedding(X)
• output_layer(rnn_outputs)
• forward(X, state=None)
• predict(prefix, num_tokens, tok, device=None, temperature=0.0, rng=None)

class RNNScratch(d2l.Module)

RNNScratch(num_inputs, num_hiddens, sigma=0.01)

The RNN model implemented from scratch. Section 8.5

• forward(inputs, state=None)

class RandomGenerator

RandomGenerator(sampling_weights)

Randomly draw among {1, …, n} according to n sampling weights. Section 17.4

• draw()

class ResNeXtBlock(nn.Block)

ResNeXtBlock(num_channels, groups, bot_mul, use_1x1conv=False, strides=1)

The ResNeXt block. Section 7.4

• forward(X)

class Residual(nn.Block)

Residual(num_channels, use_1x1conv=False, strides=1)

The Residual block of ResNet models. Section 7.4

• forward(X)

class SGD(d2l.HyperParameters)

SGD(params, lr)

Minibatch stochastic gradient descent. Section 2.4

• step(_)

class SNLIDataset(gluon.data.Dataset)

SNLIDataset(dataset, num_steps, vocab=None)

A customized dataset to load the SNLI dataset. Section 18.4

class Seq2Seq(d2l.EncoderDecoder)

Seq2Seq(encoder, decoder, tgt_pad, lr)

The RNN encoder-decoder for sequence-to-sequence learning. Section 17.1.2

• validation_step(batch)
• configure_optimizers()

class Seq2SeqEncoder(d2l.Encoder)

Seq2SeqEncoder(vocab_size, embed_size, num_hiddens, num_layers, dropout=0)

The RNN encoder for sequence-to-sequence learning. Section 17.1.2

• forward(X, *args)

class SoftmaxRegression(d2l.Classifier)

SoftmaxRegression(num_outputs, lr)

The softmax regression model. Section 3.5

• forward(X)

class SyntheticRegressionData(d2l.DataModule)

SyntheticRegressionData(w, b, noise=0.01, num_train=1000, num_val=1000, batch_size=32)

Synthetic data for linear regression. Section 2.3

• get_dataloader(train)

class TimeMachine(d2l.DataModule)

TimeMachine(batch_size, num_steps, num_train=10000, num_val=5000, tokenization='bpe', vocab_size=1024)

The Time Machine dataset. Section 8.2

• build(raw_text, vocab=None)
• get_dataloader(train)

class Timer

Record multiple running times. Section 9.4

• start() — Start the timer.
• stop() — Stop the timer and record the time in a list.
• avg() — Return the average time.
• sum() — Return the sum of time.
• cumsum() — Return the accumulated time.

class TokenEmbedding

TokenEmbedding(embedding_name)

Token Embedding. Section 17.8

class Trainer(d2l.HyperParameters)

Trainer(max_epochs, num_gpus=0, gradient_clip_val=0)

The base class for training models with data. Section 2.2

• prepare_data(data)
• fit(model, data)
• fit_epoch()
• prepare_batch(batch)
• prepare_model(model)
• clip_gradients(grad_clip_val, model)

class VOCSegDataset(gluon.data.Dataset)

VOCSegDataset(is_train, crop_size, voc_dir)

A customized dataset to load the VOC dataset. Section 19.9

• normalize_image(img)
• filter(imgs)

class Vocab

Vocab(tokens=[], min_freq=0, reserved_tokens=[])

Vocabulary for text. Section 8.2

• to_tokens(indices)
• unk()

29.10.2 Functions

PyTorch

accuracy

accuracy(y_hat, y)

Compute the number of correct predictions. Section 29.9

add_to_class

add_to_class(Class)

Register functions as methods in created class. Section 2.2

assign_anchor_to_bbox

assign_anchor_to_bbox(ground_truth, anchors, device, iou_threshold=0.5)

Assign closest ground-truth bounding boxes to anchor boxes. Section 19.4

associative_scan

associative_scan(a, b, dim=0)

Parallel prefix scan for h_t = a_t * h_{t-1} + b_t with h_0 = 0. Section 12.2

batchify

batchify(data)

Return a minibatch of examples for skip-gram with negative sampling. Section 17.4

bbox_to_rect

bbox_to_rect(bbox, color)

Convert bounding box to matplotlib format. Section 19.3

beam_search

beam_search(step_fn, prefix, num_tokens, beam_size=4, alpha=0.75, eos_id=None)

Length-normalized beam search (score = log P / len^alpha). Section 8.7.3

bleu

bleu(pred_seq, label_seq, k)

Compute the BLEU. Section 17.1.4

box_center_to_corner

box_center_to_corner(boxes)

Convert from (center, width, height) to (upper-left, lower-right). Section 19.3

box_corner_to_center

box_corner_to_center(boxes)

Convert from (upper-left, lower-right) to (center, width, height). Section 19.3

box_iou

box_iou(boxes1, boxes2)

Compute pairwise IoU across two lists of anchor or bounding boxes. Section 19.4

build_array_nmt

build_array_nmt(lines, vocab, num_steps)

Transform text sequences of machine translation into minibatches. Section 29.9

check_len

check_len(a, n)

Check the length of a list. Section 8.5

check_shape

check_shape(a, shape)

Check the shape of a tensor. Section 8.5

chrf

chrf(pred, label, n=6, beta=2)

chrF (Popovic, 2015): character n-gram F-score. Section 17.1.4

copyfile

copyfile(filename, target_dir)

Copy a file into a target directory. Section 19.13

corr2d

corr2d(X, K)

Compute 2D cross-correlation. Section 6.2

cpu

cpu()

Get the CPU device. Section 5.7

download

download(url, folder='../data', sha1_hash=None)

Download a file to folder and return the local filepath. Section 29.9

download_extract

download_extract(name, folder=None)

Download and extract a zip/tar file. Section 29.9

evaluate_accuracy_gpu

evaluate_accuracy_gpu(net, data_iter, device=None)

Compute the accuracy for a model on a dataset using a GPU. Section 29.9

evaluate_loss

evaluate_loss(net, data_iter, loss)

Evaluate the loss of a model on the given dataset. Section 29.9

extract

extract(filename, folder=None)

Extract a zip/tar file into folder. Section 29.9

generate

generate(step_fn, prefix, num_tokens, eos_id=None, **strategy)

Autoregressive generation. step_fn(ids: list[int]) -> numpy logits Section 8.7.3

get_centers_and_contexts

get_centers_and_contexts(corpus, max_window_size)

Return center words and context words in skip-gram. Section 17.4

get_dataloader_workers

get_dataloader_workers()

Use 4 processes to read the data. Section 29.9

get_fashion_mnist_labels

get_fashion_mnist_labels(labels)

Return text labels for the Fashion-MNIST dataset. Section 29.9

get_negatives

get_negatives(all_contexts, vocab, counter, K)

Return noise words in negative sampling. Section 17.4

get_tokens_and_segments

get_tokens_and_segments(tokens_a, tokens_b=None)

Get tokens of the BERT input sequence and their segment IDs. Section 17.9

gpu

gpu(i=0)

Get a GPU device. Section 5.7

grad_clipping

grad_clipping(net, theta)

Clip the gradient. Section 29.9

init_cnn

init_cnn(module)

Initialize weights for CNNs. Section 6.6

init_seq2seq

init_seq2seq(module)

Initialize weights for sequence-to-sequence learning. Section 17.1.2

linreg

linreg(X, w, b)

The linear regression model. Section 29.9

load_array

load_array(data_arrays, batch_size, is_train=True)

Construct a PyTorch data iterator. Section 29.9

load_checkpoint

load_checkpoint(path, model, optimizer=None)

Restore the state written by save_checkpoint; return the raw dict. Section 5.6

load_data_bananas

load_data_bananas(batch_size)

Load the banana detection dataset. Section 19.6

load_data_fashion_mnist

load_data_fashion_mnist(batch_size, resize=None)

Download the Fashion-MNIST dataset and then load it into memory. Section 29.9

load_data_imdb

load_data_imdb(batch_size, num_steps=500)

Return data iterators and the vocabulary of the IMDb review dataset. Section 18.1

load_data_nmt

load_data_nmt(batch_size, num_steps, num_examples=600)

Return the iterator and the vocabularies of the translation dataset. Section 29.9

load_data_ptb

load_data_ptb(batch_size, max_window_size, num_noise_words)

Download the PTB dataset and then load it into memory. Section 17.4

load_data_snli

load_data_snli(batch_size, num_steps=50)

Download the SNLI dataset and return data iterators and vocabulary. Section 18.4

load_data_voc

load_data_voc(batch_size, crop_size)

Load the VOC semantic segmentation dataset. Section 19.9

load_data_wiki

load_data_wiki(batch_size, max_len)

Load the WikiText-2 dataset. Section 17.10

masked_softmax

masked_softmax(X, valid_lens)

Perform softmax operation by masking elements on the last axis. Section 10.2

multibox_detection

multibox_detection(cls_probs, offset_preds, anchors, nms_threshold=0.5, pos_threshold=0.009999999)

Predict bounding boxes using non-maximum suppression. Section 19.4

multibox_prior

multibox_prior(data, sizes, ratios)

Generate anchor boxes with different shapes centered on each pixel. Section 19.4

multibox_target

multibox_target(anchors, labels)

Label anchor boxes using ground-truth bounding boxes. Section 19.4

nms

nms(boxes, scores, iou_threshold)

Sort confidence scores of predicted bounding boxes. Section 19.4

num_gpus

num_gpus()

Get the number of available GPUs. Section 5.7

offset_boxes

offset_boxes(anchors, assigned_bb, eps=1e-06)

Transform for anchor box offsets. Section 19.4

offset_inverse

offset_inverse(anchors, offset_preds)

Predict bounding boxes based on anchor boxes with predicted offsets. Section 19.4

plot

plot(X, Y=None, xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), figsize=(3.5, 2.5), axes=None)

Plot data points. Section 1.4

predict_sentiment

predict_sentiment(net, vocab, sequence)

Predict the sentiment of a text sequence. Section 18.2

predict_seq2seq

predict_seq2seq(net, src_sentence, src_vocab, tgt_vocab, num_steps, device, save_attention_weights=False)

Predict for sequence to sequence. Section 29.9

predict_snli

predict_snli(net, vocab, premise, hypothesis)

Predict the logical relationship between the premise and hypothesis. Section 18.5

preprocess_nmt

preprocess_nmt(text)

Preprocess the English-French dataset. Section 29.9

read_csv_labels

read_csv_labels(fname)

Read fname to return a filename to label dictionary. Section 19.13

read_data_bananas

read_data_bananas(is_train=True)

Read the banana detection dataset images and labels. Section 19.6

read_data_nmt

read_data_nmt()

Load the English-French dataset. Section 29.9

read_imdb

read_imdb(data_dir, is_train)

Read the IMDb review dataset text sequences and labels. Section 18.1

read_ptb

read_ptb()

Load the PTB dataset into a list of text lines. Section 17.4

read_snli

read_snli(data_dir, is_train)

Read the SNLI dataset into premises, hypotheses, and labels. Section 18.4

read_voc_images

read_voc_images(voc_dir, is_train=True)

Read all VOC feature and label images. Section 19.9

reorg_test

reorg_test(data_dir)

Organize the testing set for data loading during prediction. Section 19.13

reorg_train_valid

reorg_train_valid(data_dir, labels, valid_ratio)

Split the validation set out of the original training set. Section 19.13

resnet18

resnet18(num_classes, in_channels=1)

A slightly modified ResNet-18 model. Section 13.6

sample_next

sample_next(logits, strategy='greedy', temperature=1.0, k=None, p=None, min_p=None, rng=None)

Choose the next token id from a 1-D numpy logits array. Section 8.7.3

save_checkpoint

save_checkpoint(path, model, optimizer, step, cfg=None)

Atomically write a resumable training checkpoint. Section 5.6

sequence_mask

sequence_mask(X, valid_len, value=0)

Mask irrelevant entries in sequences. Section 29.9

set_axes

set_axes(axes, xlabel, ylabel, xlim, ylim, xscale, yscale, legend)

Set the axes for matplotlib. Section 1.4

set_figsize

set_figsize(figsize=(3.5, 2.5))

Set the figure size for matplotlib. Section 1.4

sgd

sgd(params, lr, batch_size)

Minibatch stochastic gradient descent. Section 29.9

show_bboxes

show_bboxes(axes, bboxes, labels=None, colors=None)

Show bounding boxes. Section 19.4

show_heatmaps

show_heatmaps(matrices, xlabel, ylabel, titles=None, figsize=(2.5, 2.5), cmap='Reds')

Show heatmaps of matrices. Section 3.4

show_images

show_images(imgs, num_rows, num_cols, titles=None, scale=1.5)

Plot a list of images. Section 29.9

show_list_len_pair_hist

show_list_len_pair_hist(legend, xlabel, ylabel, xlist, ylist)

Plot the histogram for list length pairs. Section 17.1.2

show_trace_2d

show_trace_2d(f, results)

Show the trace of 2D variables during optimization. Section 9.2

split_batch

split_batch(X, y, devices)

Split X and y into multiple devices. Section 13.5

squared_loss

squared_loss(y_hat, y)

Squared loss. Section 29.9

subsample

subsample(sentences, vocab)

Subsample high-frequency words. Section 17.4

synthetic_data

synthetic_data(w, b, num_examples)

Generate y = Xw + b + noise. Section 29.9

tokenize

tokenize(lines, token='word')

Split text lines into word or character tokens. Section 29.9

tokenize_nmt

tokenize_nmt(text, num_examples=None)

Tokenize the English-French dataset. Section 29.9

train_2d

train_2d(trainer, steps=20, f_grad=None)

Optimize a 2D objective function with a customized trainer. Section 9.2

train_batch_ch13

train_batch_ch13(net, X, y, loss, trainer, devices)

Train for a minibatch with multiple GPUs (defined in Chapter 13). Section 19.1

train_ch13

train_ch13(net, train_iter, test_iter, loss, trainer, num_epochs, devices=d2l.try_all_gpus())

Train a model with multiple GPUs (defined in Chapter 13). Section 19.1

train_ch6

train_ch6(net, train_iter, test_iter, num_epochs, lr, device)

Train a model with a GPU (defined in Chapter 6). Section 29.9

train_lm

train_lm(model, data, optimizer, num_steps)

Train a model on next-token prediction for a fixed number of steps. Section 9.6

train_seq2seq

train_seq2seq(net, data_iter, lr, num_epochs, tgt_vocab, device)

Train a model for sequence to sequence. Section 29.9

truncate_pad

truncate_pad(line, num_steps, padding_token)

Truncate or pad sequences. Section 29.9

try_all_gpus

try_all_gpus()

Return all available GPUs, or [cpu(),] if no GPU exists. Section 5.7

try_gpu

try_gpu(i=0)

Return gpu(i) if exists, otherwise return cpu(). Section 5.7

update_D

update_D(X, Z, net_D, net_G, loss, trainer_D)

Update discriminator. Section 15.1

update_G

update_G(Z, net_D, net_G, loss, trainer_G)

Update generator. Section 15.1

use_svg_display

use_svg_display()

Use the svg format to display a plot in Jupyter. Section 1.4

voc_colormap2label

voc_colormap2label()

Build the mapping from RGB to class indices for VOC labels. Section 19.9

voc_label_indices

voc_label_indices(colormap, colormap2label)

Map any RGB values in VOC labels to their class indices. Section 19.9

voc_rand_crop

voc_rand_crop(feature, label, height, width)

Randomly crop both feature and label images. Section 19.9

TensorFlow

accuracy

accuracy(y_hat, y)

Compute the number of correct predictions. Section 29.9

add_to_class

add_to_class(Class)

Register functions as methods in created class. Section 2.2

assign_anchor_to_bbox

assign_anchor_to_bbox(ground_truth, anchors, device, iou_threshold=0.5)

Assign closest ground-truth bounding boxes to anchor boxes. Section 19.4

batchify

batchify(data)

Return a minibatch of examples for skip-gram with negative sampling. Section 17.4

bbox_to_rect

bbox_to_rect(bbox, color)

Convert bounding box to matplotlib format. Section 19.3

beam_search

beam_search(step_fn, prefix, num_tokens, beam_size=4, alpha=0.75, eos_id=None)

Length-normalized beam search (score = log P / len^alpha). Section 8.7.3

bleu

bleu(pred_seq, label_seq, k)

Compute the BLEU. Section 17.1.4

box_center_to_corner

box_center_to_corner(boxes)

Convert from (center, width, height) to (upper-left, lower-right). Section 19.3

box_corner_to_center

box_corner_to_center(boxes)

Convert from (upper-left, lower-right) to (center, width, height). Section 19.3

box_iou

box_iou(boxes1, boxes2)

Compute pairwise IoU across two lists of anchor or bounding boxes. Section 19.4

build_array_nmt

build_array_nmt(lines, vocab, num_steps)

Transform text sequences of machine translation into minibatches. Section 29.9

check_len

check_len(a, n)

Check the length of a list. Section 8.5

check_shape

check_shape(a, shape)

Check the shape of a tensor. Section 8.5

chrf

chrf(pred, label, n=6, beta=2)

chrF (Popovic, 2015): character n-gram F-score. Section 17.1.4

copyfile

copyfile(filename, target_dir)

Copy a file into a target directory. Section 19.13

corr2d

corr2d(X, K)

Compute 2D cross-correlation. Section 6.2

cpu

cpu()

Get the CPU device. Section 5.7

download

download(url, folder='../data', sha1_hash=None)

Download a file to folder and return the local filepath. Section 29.9

download_extract

download_extract(name, folder=None)

Download and extract a zip/tar file. Section 29.9

evaluate_accuracy

evaluate_accuracy(net, data_iter)

Compute the accuracy for a model on a dataset. Section 29.9

evaluate_loss

evaluate_loss(net, data_iter, loss)

Evaluate the loss of a model on the given dataset. Section 29.9

extract

extract(filename, folder=None)

Extract a zip/tar file into folder. Section 29.9

generate

generate(step_fn, prefix, num_tokens, eos_id=None, **strategy)

Autoregressive generation. step_fn(ids: list[int]) -> numpy logits Section 8.7.3

get_centers_and_contexts

get_centers_and_contexts(corpus, max_window_size)

Return center words and context words in skip-gram. Section 17.4

get_fashion_mnist_labels

get_fashion_mnist_labels(labels)

Return text labels for the Fashion-MNIST dataset. Section 29.9

get_negatives

get_negatives(all_contexts, vocab, counter, K)

Return noise words in negative sampling. Section 17.4

get_tokens_and_segments

get_tokens_and_segments(tokens_a, tokens_b=None)

Get tokens of the BERT input sequence and their segment IDs. Section 17.9

gpu

gpu(i=0)

Get a GPU device. Section 5.7

grad_clipping

grad_clipping(grads, theta)

Clip the gradient. Section 29.9

linreg

linreg(X, w, b)

The linear regression model. Section 29.9

load_array

load_array(data_arrays, batch_size, is_train=True)

Construct a TensorFlow data iterator. Section 29.9

load_data_bananas

load_data_bananas(batch_size)

Load the banana detection dataset. Section 19.6

load_data_fashion_mnist

load_data_fashion_mnist(batch_size, resize=None)

Download the Fashion-MNIST dataset and then load it into memory. Section 29.9

load_data_imdb

load_data_imdb(batch_size, num_steps=500)

Return data iterators and the vocabulary of the IMDb review dataset. Section 18.1

load_data_nmt

load_data_nmt(batch_size, num_steps, num_examples=600)

Return the iterator and the vocabularies of the translation dataset. Section 29.9

load_data_ptb

load_data_ptb(batch_size, max_window_size, num_noise_words)

Download the PTB dataset and then load it into memory. Section 17.4

load_data_snli

load_data_snli(batch_size, num_steps=50)

Download the SNLI dataset and return data iterators and vocabulary. Section 18.4

load_data_voc

load_data_voc(batch_size, crop_size)

Load the VOC semantic segmentation dataset. Section 19.9

load_data_wiki

load_data_wiki(batch_size, max_len)

Load the WikiText-2 dataset. Section 17.10

multibox_detection

multibox_detection(cls_probs, offset_preds, anchors, nms_threshold=0.5, pos_threshold=0.009999999)

Predict bounding boxes using non-maximum suppression. Section 19.4

multibox_prior

multibox_prior(data, sizes, ratios)

Generate anchor boxes with different shapes centered on each pixel. Section 19.4

multibox_target

multibox_target(anchors, labels)

Label anchor boxes using ground-truth bounding boxes. Section 19.4

nms

nms(boxes, scores, iou_threshold)

Sort confidence scores of predicted bounding boxes. Section 19.4

num_gpus

num_gpus()

Get the number of available GPUs. Section 5.7

offset_boxes

offset_boxes(anchors, assigned_bb, eps=1e-06)

Transform for anchor box offsets. Section 19.4

offset_inverse

offset_inverse(anchors, offset_preds)

Predict bounding boxes based on anchor boxes with predicted offsets. Section 19.4

plot

plot(X, Y=None, xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), figsize=(3.5, 2.5), axes=None)

Plot data points. Section 1.4

predict_sentiment

predict_sentiment(net, vocab, sequence)

Predict the sentiment of a text sequence. Section 18.2

predict_seq2seq

predict_seq2seq(net, src_sentence, src_vocab, tgt_vocab, num_steps, save_attention_weights=False)

Predict for sequence to sequence. Section 29.9

predict_snli

predict_snli(net, vocab, premise, hypothesis)

Predict the logical relationship between the premise and hypothesis. Section 18.5

preprocess_nmt

preprocess_nmt(text)

Preprocess the English-French dataset. Section 29.9

read_csv_labels

read_csv_labels(fname)

Read fname to return a filename to label dictionary. Section 19.13

read_data_bananas

read_data_bananas(is_train=True)

Read the banana detection dataset images and labels. Section 19.6

read_data_nmt

read_data_nmt()

Load the English-French dataset. Section 29.9

read_imdb

read_imdb(data_dir, is_train)

Read the IMDb review dataset text sequences and labels. Section 18.1

read_ptb

read_ptb()

Load the PTB dataset into a list of text lines. Section 17.4

read_snli

read_snli(data_dir, is_train)

Read the SNLI dataset into premises, hypotheses, and labels. Section 18.4

read_voc_images

read_voc_images(voc_dir, is_train=True)

Read all VOC feature and label images. Section 19.9

reorg_test

reorg_test(data_dir)

Organize the testing set for data loading during prediction. Section 19.13

reorg_train_valid

reorg_train_valid(data_dir, labels, valid_ratio)

Split the validation set out of the original training set. Section 19.13

resnet18

resnet18(num_classes, in_channels=1)

A slightly modified ResNet-18 model built with Keras. Section 13.6

sample_next

sample_next(logits, strategy='greedy', temperature=1.0, k=None, p=None, min_p=None, rng=None)

Choose the next token id from a 1-D numpy logits array. Section 8.7.3

sequence_mask

sequence_mask(X, valid_len, value=0)

Mask irrelevant entries in sequences. Section 29.9

set_axes

set_axes(axes, xlabel, ylabel, xlim, ylim, xscale, yscale, legend)

Set the axes for matplotlib. Section 1.4

set_figsize

set_figsize(figsize=(3.5, 2.5))

Set the figure size for matplotlib. Section 1.4

sgd

sgd(params, grads, lr, batch_size)

Minibatch stochastic gradient descent. Section 29.9

show_bboxes

show_bboxes(axes, bboxes, labels=None, colors=None)

Show bounding boxes. Section 19.4

show_heatmaps

show_heatmaps(matrices, xlabel, ylabel, titles=None, figsize=(2.5, 2.5), cmap='Reds')

Show heatmaps of matrices. Section 3.4

show_images

show_images(imgs, num_rows, num_cols, titles=None, scale=1.5)

Plot a list of images. Section 29.9

show_list_len_pair_hist

show_list_len_pair_hist(legend, xlabel, ylabel, xlist, ylist)

Plot the histogram for list length pairs. Section 17.1.2

split_batch

split_batch(X, y, devices)

Split X and y into multiple devices. Section 13.5

squared_loss

squared_loss(y_hat, y)

Squared loss. Section 29.9

subsample

subsample(sentences, vocab)

Subsample high-frequency words. Section 17.4

synthetic_data

synthetic_data(w, b, num_examples)

Generate y = Xw + b + noise. Section 29.9

tokenize

tokenize(lines, token='word')

Split text lines into word or character tokens. Section 29.9

tokenize_nmt

tokenize_nmt(text, num_examples=None)

Tokenize the English-French dataset. Section 29.9

train_2d

train_2d(trainer, steps=20, f_grad=None)

Optimize a 2D objective function with a customized trainer. Section 9.2

train_batch_ch13

train_batch_ch13(net, X, y, loss, optimizer)

Train for a minibatch with Keras (defined in Chapter 13). Section 19.1

train_ch13

train_ch13(net, train_iter, test_iter, loss, optimizer, num_epochs)

Train a model with Keras (defined in Chapter 13). Section 19.1

train_ch6

train_ch6(net_fn, train_iter, test_iter, num_epochs, lr, device)

Train a model with a GPU (defined in Chapter 6). Section 29.9

train_seq2seq

train_seq2seq(net, data_iter, lr, num_epochs, tgt_vocab, device)

Train a model for sequence to sequence. Section 29.9

truncate_pad

truncate_pad(line, num_steps, padding_token)

Truncate or pad sequences. Section 29.9

try_all_gpus

try_all_gpus()

Return all available GPUs, or [cpu(),] if no GPU exists. Section 5.7

try_gpu

try_gpu(i=0)

Return gpu(i) if exists, otherwise return cpu(). Section 5.7

update_D

update_D(X, Z, net_D, net_G, loss, optimizer_D)

Update discriminator. Section 15.1

update_G

update_G(Z, net_D, net_G, loss, optimizer_G)

Update generator. Section 15.1

use_svg_display

use_svg_display()

Use the svg format to display a plot in Jupyter. Section 1.4

voc_colormap2label

voc_colormap2label()

Build the mapping from RGB to class indices for VOC labels. Section 19.9

voc_label_indices

voc_label_indices(colormap, colormap2label)

Map any RGB values in VOC labels to their class indices. Section 19.9

voc_rand_crop

voc_rand_crop(feature, label, height, width)

Randomly crop both feature and label images. Section 19.9

JAX

add_to_class

add_to_class(Class)

Register functions as methods in created class. Section 2.2

assign_anchor_to_bbox

assign_anchor_to_bbox(ground_truth, anchors, device, iou_threshold=0.5)

Assign closest ground-truth bounding boxes to anchor boxes. Section 19.4

associative_scan

associative_scan(a, b)

Parallel prefix scan for h_t = a_t * h_{t-1} + b_t with h_0 = 0. Section 12.2

batchify

batchify(data)

Return a minibatch of examples for skip-gram with negative sampling. Section 17.4

bbox_to_rect

bbox_to_rect(bbox, color)

Convert bounding box to matplotlib format. Section 19.3

beam_search

beam_search(step_fn, prefix, num_tokens, beam_size=4, alpha=0.75, eos_id=None)

Length-normalized beam search (score = log P / len^alpha). Section 8.7.3

bleu

bleu(pred_seq, label_seq, k)

Compute the BLEU. Section 17.1.4

box_center_to_corner

box_center_to_corner(boxes)

Convert from (center, width, height) to (upper-left, lower-right). Section 19.3

box_corner_to_center

box_corner_to_center(boxes)

Convert from (upper-left, lower-right) to (center, width, height). Section 19.3

box_iou

box_iou(boxes1, boxes2)

Compute pairwise IoU across two lists of anchor or bounding boxes. Section 19.4

check_len

check_len(a, n)

Check the length of a list. Section 8.5

check_shape

check_shape(a, shape)

Check the shape of a tensor. Section 8.5

chrf

chrf(pred, label, n=6, beta=2)

chrF (Popovic, 2015): character n-gram F-score. Section 17.1.4

copyfile

copyfile(filename, target_dir)

Copy a file into a target directory. Section 19.13

corr2d

corr2d(X, K)

Compute 2D cross-correlation. Section 6.2

cpu

cpu()

Get the CPU device. Section 5.7

download

download(url, folder='../data', sha1_hash=None)

Download a file to folder and return the local filepath. Section 29.9

download_extract

download_extract(name, folder=None)

Download and extract a zip/tar file. Section 29.9

evaluate_loss

evaluate_loss(net, data_iter, loss)

Evaluate the loss of a model on the given dataset. Section 29.9

extract

extract(filename, folder=None)

Extract a zip/tar file into folder. Section 29.9

generate

generate(step_fn, prefix, num_tokens, eos_id=None, **strategy)

Autoregressive generation. step_fn(ids: list[int]) -> numpy logits Section 8.7.3

get_centers_and_contexts

get_centers_and_contexts(corpus, max_window_size)

Return center words and context words in skip-gram. Section 17.4

get_negatives

get_negatives(all_contexts, vocab, counter, K)

Return noise words in negative sampling. Section 17.4

get_tokens_and_segments

get_tokens_and_segments(tokens_a, tokens_b=None)

Get tokens of the BERT input sequence and their segment IDs. Section 17.9

gpu

gpu(i=0)

Get a GPU device. Section 5.7

linreg

linreg(X, w, b)

The linear regression model. Section 29.9

load_array

load_array(data_arrays, batch_size, is_train=True)

Construct a JAX-compatible data iterator. Section 29.9

load_data_bananas

load_data_bananas(batch_size)

Load the banana detection dataset. Section 19.6

load_data_imdb

load_data_imdb(batch_size, num_steps=500)

Return data iterators and the vocabulary of the IMDb review dataset. Section 18.1

load_data_ptb

load_data_ptb(batch_size, max_window_size, num_noise_words)

Download the PTB dataset and then load it into memory. Section 17.4

load_data_snli

load_data_snli(batch_size, num_steps=50)

Download the SNLI dataset and return data iterators and vocabulary. Section 18.4

load_data_voc

load_data_voc(batch_size, crop_size)

Load the VOC semantic segmentation dataset. Section 19.9

load_data_wiki

load_data_wiki(batch_size, max_len)

Load the WikiText-2 dataset. Section 17.10

masked_softmax

masked_softmax(X, valid_lens)

Perform softmax operation by masking elements on the last axis. Section 10.2

multibox_detection

multibox_detection(cls_probs, offset_preds, anchors, nms_threshold=0.5, pos_threshold=0.009999999)

Predict bounding boxes using non-maximum suppression. Section 19.4

multibox_prior

multibox_prior(data, sizes, ratios)

Generate anchor boxes with different shapes centered on each pixel. Section 19.4

multibox_target

multibox_target(anchors, labels)

Label anchor boxes using ground-truth bounding boxes. Section 19.4

nms

nms(boxes, scores, iou_threshold)

Sort confidence scores of predicted bounding boxes. Section 19.4

num_gpus

num_gpus()

Get the number of available GPUs. Section 5.7

offset_boxes

offset_boxes(anchors, assigned_bb, eps=1e-06)

Transform for anchor box offsets. Section 19.4

offset_inverse

offset_inverse(anchors, offset_preds)

Predict bounding boxes based on anchor boxes with predicted offsets. Section 19.4

plot

plot(X, Y=None, xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), figsize=(3.5, 2.5), axes=None)

Plot data points. Section 1.4

predict_sentiment

predict_sentiment(net, vocab, sequence)

Predict the sentiment of a text sequence. Section 18.2

predict_snli

predict_snli(net, vocab, premise, hypothesis)

Predict the logical relationship between the premise and hypothesis. Section 18.5

read_csv_labels

read_csv_labels(fname)

Read fname to return a filename to label dictionary. Section 19.13

read_data_bananas

read_data_bananas(is_train=True)

Read the banana detection dataset images and labels. Section 19.6

read_imdb

read_imdb(data_dir, is_train)

Read the IMDb review dataset text sequences and labels. Section 18.1

read_ptb

read_ptb()

Load the PTB dataset into a list of text lines. Section 17.4

read_snli

read_snli(data_dir, is_train)

Read the SNLI dataset into premises, hypotheses, and labels. Section 18.4

read_voc_images

read_voc_images(voc_dir, is_train=True)

Read all VOC feature and label images. Section 19.9

reorg_test

reorg_test(data_dir)

Organize the testing set for data loading during prediction. Section 19.13

reorg_train_valid

reorg_train_valid(data_dir, labels, valid_ratio)

Split the validation set out of the original training set. Section 19.13

sample_next

sample_next(logits, strategy='greedy', temperature=1.0, k=None, p=None, min_p=None, rng=None)

Choose the next token id from a 1-D numpy logits array. Section 8.7.3

set_axes

set_axes(axes, xlabel, ylabel, xlim, ylim, xscale, yscale, legend)

Set the axes for matplotlib. Section 1.4

set_figsize

set_figsize(figsize=(3.5, 2.5))

Set the figure size for matplotlib. Section 1.4

show_bboxes

show_bboxes(axes, bboxes, labels=None, colors=None)

Show bounding boxes. Section 19.4

show_heatmaps

show_heatmaps(matrices, xlabel, ylabel, titles=None, figsize=(2.5, 2.5), cmap='Reds')

Show heatmaps of matrices. Section 3.4

show_images

show_images(imgs, num_rows, num_cols, titles=None, scale=1.5)

Plot a list of images. Section 29.9

show_list_len_pair_hist

show_list_len_pair_hist(legend, xlabel, ylabel, xlist, ylist)

Plot the histogram for list length pairs. Section 17.1.2

show_trace_2d

show_trace_2d(f, results)

Show the trace of 2D variables during optimization. Section 9.2

split_batch

split_batch(X, y, num_devices)

Reshape X and y onto a leading device axis of size num_devices. Section 13.5

squared_loss

squared_loss(y_hat, y)

Squared loss. Section 29.9

subsample

subsample(sentences, vocab)

Subsample high-frequency words. Section 17.4

tokenize

tokenize(lines, token='word')

Split text lines into word or character tokens. Section 29.9

train_2d

train_2d(trainer, steps=20, f_grad=None)

Optimize a 2D objective function with a customized trainer. Section 9.2

train_batch_ch13

train_batch_ch13(net, optimizer, X, y)

Train for a minibatch with JAX (defined in Chapter 13). Section 19.1

train_ch13

train_ch13(net, train_iter, test_iter, optimizer, num_epochs)

Train a model with JAX (defined in Chapter 13). Section 19.1

train_lm

train_lm(model, data, optimizer, num_steps)

Train a model on next-token prediction for a fixed number of steps. Section 9.6

truncate_pad

truncate_pad(line, num_steps, padding_token)

Truncate or pad sequences. Section 29.9

try_all_gpus

try_all_gpus()

Return all available GPUs, or [cpu(),] if no GPU exists. Section 5.7

try_gpu

try_gpu(i=0)

Return gpu(i) if exists, otherwise return cpu(). Section 5.7

update_D

update_D(X, Z, net_D, net_G, optimizer_D)

Update discriminator. Section 15.1

update_G

update_G(Z, net_D, net_G, optimizer_G)

Update generator. Section 15.1

use_svg_display

use_svg_display()

Use the svg format to display a plot in Jupyter. Section 1.4

voc_colormap2label

voc_colormap2label()

Build the mapping from RGB to class indices for VOC labels. Section 19.9

voc_label_indices

voc_label_indices(colormap, colormap2label)

Map any RGB values in VOC labels to their class indices. Section 19.9

voc_rand_crop

voc_rand_crop(feature, label, height, width)

Randomly crop both feature and label images. Section 19.9

MXNet

accuracy

accuracy(y_hat, y)

Compute the number of correct predictions. Section 29.9

add_to_class

add_to_class(Class)

Register functions as methods in created class. Section 2.2

assign_anchor_to_bbox

assign_anchor_to_bbox(ground_truth, anchors, device, iou_threshold=0.5)

Assign closest ground-truth bounding boxes to anchor boxes. Section 19.4

batchify

batchify(data)

Return a minibatch of examples for skip-gram with negative sampling. Section 17.4

bbox_to_rect

bbox_to_rect(bbox, color)

Convert bounding box to matplotlib format. Section 19.3

beam_search

beam_search(step_fn, prefix, num_tokens, beam_size=4, alpha=0.75, eos_id=None)

Length-normalized beam search (score = log P / len^alpha). Section 8.7.3

bleu

bleu(pred_seq, label_seq, k)

Compute the BLEU. Section 17.1.4

box_center_to_corner

box_center_to_corner(boxes)

Convert from (center, width, height) to (upper-left, lower-right). Section 19.3

box_corner_to_center

box_corner_to_center(boxes)

Convert from (upper-left, lower-right) to (center, width, height). Section 19.3

box_iou

box_iou(boxes1, boxes2)

Compute pairwise IoU across two lists of anchor or bounding boxes. Section 19.4

build_array_nmt

build_array_nmt(lines, vocab, num_steps)

Transform text sequences of machine translation into minibatches. Section 29.9

check_len

check_len(a, n)

Check the length of a list. Section 8.5

check_shape

check_shape(a, shape)

Check the shape of a tensor. Section 8.5

chrf

chrf(pred, label, n=6, beta=2)

chrF (Popovic, 2015): character n-gram F-score. Section 17.1.4

copyfile

copyfile(filename, target_dir)

Copy a file into a target directory. Section 19.13

corr2d

corr2d(X, K)

Compute 2D cross-correlation. Section 6.2

cpu

cpu()

Get the CPU device. Section 5.7

download

download(url, folder='../data', sha1_hash=None)

Download a file to folder and return the local filepath. Section 29.9

download_extract

download_extract(name, folder=None)

Download and extract a zip/tar file. Section 29.9

evaluate_accuracy

evaluate_accuracy(net, data_iter)

Compute the accuracy for a model on a dataset. Section 29.9

evaluate_accuracy_gpu

evaluate_accuracy_gpu(net, data_iter, device=None)

Compute the accuracy for a model on a dataset using a GPU. Section 29.9

evaluate_accuracy_gpus

evaluate_accuracy_gpus(net, data_iter, split_f=d2l.split_batch)

Compute the accuracy for a model on a dataset using multiple GPUs. Section 13.6

evaluate_loss

evaluate_loss(net, data_iter, loss)

Evaluate the loss of a model on the given dataset. Section 29.9

extract

extract(filename, folder=None)

Extract a zip/tar file into folder. Section 29.9

generate

generate(step_fn, prefix, num_tokens, eos_id=None, **strategy)

Autoregressive generation. step_fn(ids: list[int]) -> numpy logits Section 8.7.3

get_centers_and_contexts

get_centers_and_contexts(corpus, max_window_size)

Return center words and context words in skip-gram. Section 17.4

get_dataloader_workers

get_dataloader_workers()

Use 4 processes to read the data except for Windows. Section 29.9

get_fashion_mnist_labels

get_fashion_mnist_labels(labels)

Return text labels for the Fashion-MNIST dataset. Section 29.9

get_negatives

get_negatives(all_contexts, vocab, counter, K)

Return noise words in negative sampling. Section 17.4

get_tokens_and_segments

get_tokens_and_segments(tokens_a, tokens_b=None)

Get tokens of the BERT input sequence and their segment IDs. Section 17.9

gpu

gpu(i=0)

Get a GPU device. Section 5.7

linreg

linreg(X, w, b)

The linear regression model. Section 29.9

load_array

load_array(data_arrays, batch_size, is_train=True)

Construct a Gluon data iterator. Section 29.9

load_checkpoint

load_checkpoint(prefix, model, trainer=None)

Restore the state written by save_checkpoint; return the metadata. Section 5.6

load_data_bananas

load_data_bananas(batch_size)

Load the banana detection dataset. Section 19.6

load_data_fashion_mnist

load_data_fashion_mnist(batch_size, resize=None)

Download the Fashion-MNIST dataset and then load it into memory. Section 29.9

load_data_imdb

load_data_imdb(batch_size, num_steps=500)

Return data iterators and the vocabulary of the IMDb review dataset. Section 18.1

load_data_nmt

load_data_nmt(batch_size, num_steps, num_examples=600)

Return the iterator and the vocabularies of the translation dataset. Section 29.9

load_data_ptb

load_data_ptb(batch_size, max_window_size, num_noise_words)

Download the PTB dataset and then load it into memory. Section 17.4

load_data_snli

load_data_snli(batch_size, num_steps=50)

Download the SNLI dataset and return data iterators and vocabulary. Section 18.4

load_data_voc

load_data_voc(batch_size, crop_size)

Load the VOC semantic segmentation dataset. Section 19.9

load_data_wiki

load_data_wiki(batch_size, max_len)

Load the WikiText-2 dataset. Section 17.10

multibox_detection

multibox_detection(cls_probs, offset_preds, anchors, nms_threshold=0.5, pos_threshold=0.009999999)

Predict bounding boxes using non-maximum suppression. Section 19.4

multibox_prior

multibox_prior(data, sizes, ratios)

Generate anchor boxes with different shapes centered on each pixel. Section 19.4

multibox_target

multibox_target(anchors, labels)

Label anchor boxes using ground-truth bounding boxes. Section 19.4

nms

nms(boxes, scores, iou_threshold)

Sort confidence scores of predicted bounding boxes. Section 19.4

num_gpus

num_gpus()

Get the number of available GPUs. Section 5.7

offset_boxes

offset_boxes(anchors, assigned_bb, eps=1e-06)

Transform for anchor box offsets. Section 19.4

offset_inverse

offset_inverse(anchors, offset_preds)

Predict bounding boxes based on anchor boxes with predicted offsets. Section 19.4

plot

plot(X, Y=None, xlabel=None, ylabel=None, legend=None, xlim=None, ylim=None, xscale='linear', yscale='linear', fmts=('-', 'm--', 'g-.', 'r:'), figsize=(3.5, 2.5), axes=None)

Plot data points. Section 1.4

predict_sentiment

predict_sentiment(net, vocab, sequence)

Predict the sentiment of a text sequence. Section 18.2

predict_seq2seq

predict_seq2seq(net, src_sentence, src_vocab, tgt_vocab, num_steps, device, save_attention_weights=False)

Predict for sequence to sequence. Section 29.9

predict_snli

predict_snli(net, vocab, premise, hypothesis)

Predict the logical relationship between the premise and hypothesis. Section 18.5

preprocess_nmt

preprocess_nmt(text)

Preprocess the English-French dataset. Section 29.9

read_csv_labels

read_csv_labels(fname)

Read fname to return a filename to label dictionary. Section 19.13

read_data_bananas

read_data_bananas(is_train=True)

Read the banana detection dataset images and labels. Section 19.6

read_data_nmt

read_data_nmt()

Load the English-French dataset. Section 29.9

read_imdb

read_imdb(data_dir, is_train)

Read the IMDb review dataset text sequences and labels. Section 18.1

read_ptb

read_ptb()

Load the PTB dataset into a list of text lines. Section 17.4

read_snli

read_snli(data_dir, is_train)

Read the SNLI dataset into premises, hypotheses, and labels. Section 18.4

read_voc_images

read_voc_images(voc_dir, is_train=True)

Read all VOC feature and label images. Section 19.9

reorg_test

reorg_test(data_dir)

Organize the testing set for data loading during prediction. Section 19.13

reorg_train_valid

reorg_train_valid(data_dir, labels, valid_ratio)

Split the validation set out of the original training set. Section 19.13

resnet18

resnet18(num_classes)

A slightly modified ResNet-18 model. Section 13.6

sample_next

sample_next(logits, strategy='greedy', temperature=1.0, k=None, p=None, min_p=None, rng=None)

Choose the next token id from a 1-D numpy logits array. Section 8.7.3

save_checkpoint

save_checkpoint(prefix, model, trainer, step, cfg=None)

Replace each component of a resumable MXNet checkpoint. Section 5.6

set_axes

set_axes(axes, xlabel, ylabel, xlim, ylim, xscale, yscale, legend)

Set the axes for matplotlib. Section 1.4

set_figsize

set_figsize(figsize=(3.5, 2.5))

Set the figure size for matplotlib. Section 1.4

sgd

sgd(params, lr, batch_size)

Minibatch stochastic gradient descent. Section 29.9

show_bboxes

show_bboxes(axes, bboxes, labels=None, colors=None)

Show bounding boxes. Section 19.4

show_heatmaps

show_heatmaps(matrices, xlabel, ylabel, titles=None, figsize=(2.5, 2.5), cmap='Reds')

Show heatmaps of matrices. Section 3.4

show_images

show_images(imgs, num_rows, num_cols, titles=None, scale=1.5)

Plot a list of images. Section 29.9

show_list_len_pair_hist

show_list_len_pair_hist(legend, xlabel, ylabel, xlist, ylist)

Plot the histogram for list length pairs. Section 17.1.2

split_batch

split_batch(X, y, devices)

Split X and y into multiple devices. Section 13.5

split_batch_multi_inputs

split_batch_multi_inputs(X, y, devices)

Split multi-input X and y into multiple devices. Section 18.5

squared_loss

squared_loss(y_hat, y)

Squared loss. Section 29.9

subsample

subsample(sentences, vocab)

Subsample high-frequency words. Section 17.4

synthetic_data

synthetic_data(w, b, num_examples)

Generate y = Xw + b + noise. Section 29.9

tokenize

tokenize(lines, token='word')

Split text lines into word or character tokens. Section 29.9

tokenize_nmt

tokenize_nmt(text, num_examples=None)

Tokenize the English-French dataset. Section 29.9

train_2d

train_2d(trainer, steps=20, f_grad=None)

Optimize a 2D objective function with a customized trainer. Section 9.2

train_batch_ch13

train_batch_ch13(net, features, labels, loss, trainer, devices, split_f=d2l.split_batch)

Train for a minibatch with multiple GPUs (defined in Chapter 13). Section 19.1

train_ch13

train_ch13(net, train_iter, test_iter, loss, trainer, num_epochs, devices=d2l.try_all_gpus(), split_f=d2l.split_batch)

Train a model with multiple GPUs (defined in Chapter 13). Section 19.1

train_ch6

train_ch6(net, train_iter, test_iter, num_epochs, lr, device)

Train a model with a GPU (defined in Chapter 6). Section 29.9

train_seq2seq

train_seq2seq(net, data_iter, lr, num_epochs, tgt_vocab, device)

Train a model for sequence to sequence. Section 29.9

truncate_pad

truncate_pad(line, num_steps, padding_token)

Truncate or pad sequences. Section 29.9

try_all_gpus

try_all_gpus()

Return all available GPUs, or [cpu(),] if no GPU exists. Section 5.7

try_gpu

try_gpu(i=0)

Return gpu(i) if exists, otherwise return cpu(). Section 5.7

update_D

update_D(X, Z, net_D, net_G, loss, trainer_D)

Update discriminator. Section 15.1

update_G

update_G(Z, net_D, net_G, loss, trainer_G)

Update generator. Section 15.1

use_svg_display

use_svg_display()

Use the svg format to display a plot in Jupyter. Section 1.4

voc_colormap2label

voc_colormap2label()

Build the mapping from RGB to class indices for VOC labels. Section 19.9

voc_label_indices

voc_label_indices(colormap, colormap2label)

Map any RGB values in VOC labels to their class indices. Section 19.9

voc_rand_crop

voc_rand_crop(feature, label, height, width)

Randomly crop both feature and label images. Section 19.9