Known ground truth

Synthetic Regression Data

Synthetic data with a known answer

Before we train a model we need data. For pedagogy, we’ll synthesize it — known weights, known noise, and a guaranteed correct answer to compare against:

\mathbf{y} = \mathbf{X} \mathbf{w} + b + \boldsymbol{\epsilon}, \quad \boldsymbol{\epsilon} \sim \mathcal{N}(0, \sigma^2 I).

This chapter:

• Subclass DataModule to generate the synthetic batch.
• Roll a hand-written minibatch sampler (to see how it works).
• Swap in the framework’s built-in dataloader (the version we’ll actually use).

Synthetic data module

A DataModule subclass that draws features and computes labels in __init__:

%matplotlib inline
from d2l import tensorflow as d2l
import tensorflow as tf
import random

class SyntheticRegressionData(d2l.DataModule):
    """Synthetic data for linear regression."""
    def __init__(self, w, b, noise=0.01, num_train=1000, num_val=1000, 
                 batch_size=32):
        super().__init__()
        self.save_hyperparameters()
        n = num_train + num_val
        self.X = tf.random.normal((n, w.shape[0]))
        noise = tf.random.normal((n, 1)) * noise
        self.y = d2l.matmul(self.X, d2l.reshape(w, (-1, 1))) + b + noise

Instantiate with the true w = [2, -3.4], b = 4.2:

data = SyntheticRegressionData(w=d2l.tensor([2, -3.4]), b=4.2)

Inspecting one example

Each row of features is a vector in \mathbb{R}^2; the corresponding label is a scalar:

print('features:', data.X[0],'\nlabel:', data.y[0])

features: tf.Tensor([-1.3305755  1.3084109], shape=(2,), dtype=float32) 
label: tf.Tensor([-2.9095104], shape=(1,), dtype=float32)

A handwritten dataloader

get_dataloader shuffles indices, then yields minibatches of size batch_size:

def get_dataloader(self, train):
    if train:
        indices = list(range(0, self.num_train))
        # The examples are read in random order
        random.shuffle(indices)
    else:
        indices = list(range(self.num_train, self.num_train+self.num_val))
    for i in range(0, len(indices), self.batch_size):
        j = tf.constant(indices[i : i+self.batch_size])
        yield tf.gather(self.X, j), tf.gather(self.y, j)

X, y = next(iter(data.train_dataloader()))
print('X shape:', X.shape, '\ny shape:', y.shape)

X shape: (32, 2) 
y shape: (32, 1)

Educational, but slow — Python loops over indices, no prefetching, no parallelism.

The framework dataloader

For real work, wrap features and labels in the framework’s built-in dataset / dataloader (workers, prefetch, GPU pinning):

@d2l.add_to_class(d2l.DataModule)
def get_tensorloader(self, tensors, train, indices=slice(0, None)):
    tensors = tuple(a[indices] for a in tensors)
    shuffle_buffer = tensors[0].shape[0] if train else 1
    return tf.data.Dataset.from_tensor_slices(tensors).shuffle(
        buffer_size=shuffle_buffer).batch(self.batch_size)

@d2l.add_to_class(SyntheticRegressionData)
def get_dataloader(self, train):
    i = slice(0, self.num_train) if train else slice(self.num_train, None)
    return self.get_tensorloader((self.X, self.y), train, i)

Same minibatch interface

Identical iteration protocol from the caller’s POV:

X, y = next(iter(data.train_dataloader()))
print('X shape:', X.shape, '\ny shape:', y.shape)

X shape: (32, 2) 
y shape: (32, 1)

len(dl) reports the number of batches per epoch — convenient for progress bars:

len(data.train_dataloader())

32

Recap

• Synthetic data → ground-truth w, b you can compare against later.
• DataModule subclasses encapsulate “where do batches come from?” once, reusable across models.
• Hand-rolled iterator vs. framework dataloader — same protocol; framework version wins on speed and ergonomics.