{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "---\n",
    "execute:\n",
    "  enabled: false\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-1 fw=all -->\n",
    "\n",
    "# Image Classification (CIFAR-10) on Kaggle\n",
    "\n",
    "So far, we have been using high-level APIs of deep learning frameworks to directly obtain image datasets in tensor format.\n",
    "However, custom image datasets\n",
    "often come in the form of image files.\n",
    "In this section, we will start from\n",
    "raw image files,\n",
    "and organize, read, then transform them\n",
    "into tensor format step by step.\n",
    "\n",
    "We experimented with the CIFAR-10 dataset in that section,\n",
    "which is an important dataset in computer vision.\n",
    "In this section,\n",
    "we will apply the knowledge we learned\n",
    "in previous sections\n",
    "to practice the Kaggle competition of\n",
    "CIFAR-10 image classification.\n",
    "The web address of the competition is https://www.kaggle.com/c/cifar-10\n",
    "\n",
    "the figure shows the information on the competition's webpage.\n",
    "In order to submit the results,\n",
    "you need to register a Kaggle account.\n",
    "\n",
    "![CIFAR-10 image classification competition webpage information. The competition dataset can be obtained by clicking the \"Data\" tab.](../img/kaggle-cifar10.png)"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-image-classification-cifar-10-on-kaggle"
   },
   "source": [
    "import collections\n",
    "from d2l import jax as d2l\n",
    "import jax\n",
    "from jax import numpy as jnp\n",
    "from flax import nnx\n",
    "import optax\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import math\n",
    "import os\n",
    "import pandas as pd\n",
    "import shutil"
   ],
   "execution_count": 1,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-2 fw=all -->\n",
    "\n",
    "## Obtaining and Organizing the Dataset\n",
    "\n",
    "The competition dataset is divided into\n",
    "a training set and a test set,\n",
    "which contain 50000 and 300000 images, respectively.\n",
    "In the test set,\n",
    "10000 images will be used for evaluation,\n",
    "while the remaining 290000 images will not\n",
    "be evaluated:\n",
    "they are included just\n",
    "to make it hard\n",
    "to cheat with\n",
    "*manually* labeled results of the test set.\n",
    "The images in this dataset\n",
    "are all png color (RGB channels) image files,\n",
    "whose height and width are both 32 pixels.\n",
    "The images cover a total of 10 categories, namely airplanes, cars, birds, cats, deer, dogs, frogs, horses, boats, and trucks.\n",
    "The upper-left corner of the figure shows some images of airplanes, cars, and birds in the dataset.\n",
    "\n",
    "\n",
    "### Downloading the Dataset\n",
    "\n",
    "After logging in to Kaggle, we can click the \"Data\" tab on the CIFAR-10 image classification competition webpage shown in the figure and download the dataset by clicking the \"Download All\" button.\n",
    "After unzipping the downloaded file in `../data`, and unzipping `train.7z` and `test.7z` inside it, you will find the entire dataset in the following paths:\n",
    "\n",
    "* `../data/cifar-10/train/[1-50000].png`\n",
    "* `../data/cifar-10/test/[1-300000].png`\n",
    "* `../data/cifar-10/trainLabels.csv`\n",
    "* `../data/cifar-10/sampleSubmission.csv`\n",
    "\n",
    "where the `train` and `test` directories contain the training and testing images, respectively, `trainLabels.csv` provides labels for the training images, and `sample_submission.csv` is a sample submission file.\n",
    "\n",
    "To make it easier to get started, we provide a small-scale sample of the dataset that\n",
    "contains the first 1000 training images and 5 random testing images.\n",
    "To use the full dataset of the Kaggle competition, you need to set the following `demo` variable to `False`."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-downloading-the-dataset"
   },
   "source": [
    "\n",
    "d2l.DATA_HUB['cifar10_tiny'] = (d2l.DATA_URL + 'kaggle_cifar10_tiny.zip',\n",
    "                                '2068874e4b9a9f0fb07ebe0ad2b29754449ccacd')\n",
    "\n",
    "# If you use the full dataset downloaded for the Kaggle competition, set\n",
    "# `demo` to False\n",
    "demo = True\n",
    "\n",
    "if demo:\n",
    "    data_dir = d2l.download_extract('cifar10_tiny')\n",
    "else:\n",
    "    data_dir = '../data/cifar-10/'"
   ],
   "execution_count": 2,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-3 fw=all -->\n",
    "\n",
    "### Organizing the Dataset\n",
    "\n",
    "We need to organize datasets to facilitate model training and testing.\n",
    "Let's first read the labels from the csv file.\n",
    "The following function returns a dictionary that maps\n",
    "the non-extension part of the filename to its label."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-organizing-the-dataset-1"
   },
   "source": [
    "\n",
    "def read_csv_labels(fname):\n",
    "    \"\"\"Read `fname` to return a filename to label dictionary.\"\"\"\n",
    "    with open(fname, 'r') as f:\n",
    "        # Skip the file header line (column name)\n",
    "        lines = f.readlines()[1:]\n",
    "    tokens = [l.rstrip().split(',') for l in lines]\n",
    "    return dict(((name, label) for name, label in tokens))\n",
    "\n",
    "labels = read_csv_labels(os.path.join(data_dir, 'trainLabels.csv'))\n",
    "print('# training examples:', len(labels))\n",
    "print('# classes:', len(set(labels.values())))"
   ],
   "execution_count": 3,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "# training examples: 1000\n# classes: 10\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-4 fw=all -->\n",
    "\n",
    "Next, we define the `reorg_train_valid` function to split the validation set out of the original training set.\n",
    "The argument `valid_ratio` in this function is the ratio of the number of examples in the validation set to the number of examples in the original training set.\n",
    "More concretely,\n",
    "let $n$ be the number of images of the class with the least examples, and $r$ be the ratio.\n",
    "The validation set will split out\n",
    "$\\max(\\lfloor nr\\rfloor,1)$ images for each class.\n",
    "Let's use `valid_ratio=0.1` as an example. Since the original training set has 50000 images,\n",
    "there will be 45000 images used for training in the path `train_valid_test/train`,\n",
    "while the other 5000 images will be split out\n",
    "as validation set in the path `train_valid_test/valid`. After organizing the dataset, images of the same class will be placed under the same folder."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-organizing-the-dataset-2"
   },
   "source": [
    "\n",
    "def copyfile(filename, target_dir):\n",
    "    \"\"\"Copy a file into a target directory.\"\"\"\n",
    "    os.makedirs(target_dir, exist_ok=True)\n",
    "    shutil.copy(filename, target_dir)\n",
    "\n",
    "\n",
    "def reorg_train_valid(data_dir, labels, valid_ratio):\n",
    "    \"\"\"Split the validation set out of the original training set.\"\"\"\n",
    "    # The number of examples of the class that has the fewest examples in the\n",
    "    # training dataset\n",
    "    n = collections.Counter(labels.values()).most_common()[-1][1]\n",
    "    # The number of examples per class for the validation set\n",
    "    n_valid_per_label = max(1, math.floor(n * valid_ratio))\n",
    "    label_count = {}\n",
    "    for train_file in os.listdir(os.path.join(data_dir, 'train')):\n",
    "        label = labels[train_file.split('.')[0]]\n",
    "        fname = os.path.join(data_dir, 'train', train_file)\n",
    "        copyfile(fname, os.path.join(data_dir, 'train_valid_test',\n",
    "                                     'train_valid', label))\n",
    "        if label not in label_count or label_count[label] < n_valid_per_label:\n",
    "            copyfile(fname, os.path.join(data_dir, 'train_valid_test',\n",
    "                                         'valid', label))\n",
    "            label_count[label] = label_count.get(label, 0) + 1\n",
    "        else:\n",
    "            copyfile(fname, os.path.join(data_dir, 'train_valid_test',\n",
    "                                         'train', label))\n",
    "    return n_valid_per_label"
   ],
   "execution_count": 4,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-5 fw=all -->\n",
    "\n",
    "The `reorg_test` function below organizes the testing set for data loading during prediction."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-organizing-the-dataset-3"
   },
   "source": [
    "\n",
    "def reorg_test(data_dir):\n",
    "    \"\"\"Organize the testing set for data loading during prediction.\"\"\"\n",
    "    for test_file in os.listdir(os.path.join(data_dir, 'test')):\n",
    "        copyfile(os.path.join(data_dir, 'test', test_file),\n",
    "                 os.path.join(data_dir, 'train_valid_test', 'test',\n",
    "                              'unknown'))"
   ],
   "execution_count": 5,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-6 fw=all -->\n",
    "\n",
    "Finally, we use a function to invoke\n",
    "the `read_csv_labels`, `reorg_train_valid`, and `reorg_test` functions defined above."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-organizing-the-dataset-4"
   },
   "source": [
    "def reorg_cifar10_data(data_dir, valid_ratio):\n",
    "    labels = read_csv_labels(os.path.join(data_dir, 'trainLabels.csv'))\n",
    "    reorg_train_valid(data_dir, labels, valid_ratio)\n",
    "    reorg_test(data_dir)"
   ],
   "execution_count": 6,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-7 fw=all -->\n",
    "\n",
    "Here we only set the batch size to 32 for the small-scale sample of the dataset.\n",
    "When training and testing\n",
    "the complete dataset of the Kaggle competition,\n",
    "`batch_size` should be set to a larger integer, such as 128.\n",
    "We split out 10% of the training examples as the validation set for tuning hyperparameters."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-organizing-the-dataset-5"
   },
   "source": [
    "batch_size = 32 if demo else 128\n",
    "valid_ratio = 0.1\n",
    "reorg_cifar10_data(data_dir, valid_ratio)"
   ],
   "execution_count": 7,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-8 fw=all -->\n",
    "\n",
    "## Image Augmentation\n",
    "\n",
    "We use image augmentation to address overfitting.\n",
    "For example, images can be flipped horizontally at random during training.\n",
    "We can also perform standardization for the three RGB channels of color images. Below lists some of these operations that you can tweak."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-image-augmentation-1"
   },
   "source": [
    "CIFAR_MEAN = np.array([0.4914, 0.4822, 0.4465], dtype=np.float32)\n",
    "CIFAR_STD = np.array([0.2023, 0.1994, 0.2010], dtype=np.float32)\n",
    "\n",
    "def transform_train_fn(image, label):\n",
    "    \"\"\"Training augmentation: resize, random crop, flip, normalize.\"\"\"\n",
    "    image = tf.cast(image, tf.float32)\n",
    "    image = tf.image.resize(image, [40, 40])\n",
    "    image = tf.image.random_crop(image, size=[32, 32, 3])\n",
    "    image = tf.image.random_flip_left_right(image)\n",
    "    image = image / 255.0\n",
    "    image = (image - CIFAR_MEAN) / CIFAR_STD\n",
    "    return image, label\n",
    "\n",
    "def transform_test_fn(image, label):\n",
    "    \"\"\"Test preprocessing: normalize only.\"\"\"\n",
    "    image = tf.cast(image, tf.float32) / 255.0\n",
    "    image = (image - CIFAR_MEAN) / CIFAR_STD\n",
    "    return image, label"
   ],
   "execution_count": 8,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-9 fw=all -->\n",
    "\n",
    "During testing,\n",
    "we only perform standardization on images\n",
    "so as to\n",
    "remove randomness in the evaluation results."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-image-augmentation-2"
   },
   "source": [
    "# Test transform is defined above as transform_test_fn"
   ],
   "execution_count": 9,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-10 fw=all -->\n",
    "\n",
    "## Reading the Dataset\n",
    "\n",
    "Next, we read the organized dataset consisting of raw image files. Each example includes an image and a label."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-reading-the-dataset-1"
   },
   "source": [
    "def _load_image_folder_tf(folder_path):\n",
    "    \"\"\"Load images from a class-subfolder directory into a tf.data.Dataset\n",
    "    of (image, label) where image is uint8 [H, W, 3] and label is int.\"\"\"\n",
    "    ds = tf.keras.utils.image_dataset_from_directory(\n",
    "        folder_path, label_mode='int', image_size=(32, 32),\n",
    "        batch_size=None, shuffle=False)\n",
    "    return ds\n",
    "\n",
    "train_ds = _load_image_folder_tf(\n",
    "    os.path.join(data_dir, 'train_valid_test', 'train'))\n",
    "train_valid_ds = _load_image_folder_tf(\n",
    "    os.path.join(data_dir, 'train_valid_test', 'train_valid'))\n",
    "valid_ds = _load_image_folder_tf(\n",
    "    os.path.join(data_dir, 'train_valid_test', 'valid'))\n",
    "test_ds = _load_image_folder_tf(\n",
    "    os.path.join(data_dir, 'train_valid_test', 'test'))"
   ],
   "execution_count": 10,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Found 920 files belonging to 10 classes.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Found 1000 files belonging to 10 classes.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Found 80 files belonging to 10 classes.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Found 5 files belonging to 1 classes.\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-11 fw=all -->\n",
    "\n",
    "During training,\n",
    "we need to specify all the image augmentation operations defined above.\n",
    "When the validation set\n",
    "is used for model evaluation during hyperparameter tuning,\n",
    "no randomness from image augmentation should be introduced.\n",
    "Before final prediction,\n",
    "we train the model on the combined training set and validation set to make full use of all the labeled data."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-reading-the-dataset-2"
   },
   "source": [
    "train_iter = (train_ds.map(transform_train_fn, num_parallel_calls=tf.data.AUTOTUNE)\n",
    "              .shuffle(10000).batch(batch_size, drop_remainder=True)\n",
    "              .prefetch(tf.data.AUTOTUNE))\n",
    "train_valid_iter = (train_valid_ds.map(transform_train_fn,\n",
    "                    num_parallel_calls=tf.data.AUTOTUNE)\n",
    "                    .shuffle(10000).batch(batch_size, drop_remainder=True)\n",
    "                    .prefetch(tf.data.AUTOTUNE))\n",
    "valid_iter = (valid_ds.map(transform_test_fn, num_parallel_calls=tf.data.AUTOTUNE)\n",
    "              .batch(batch_size, drop_remainder=True)\n",
    "              .prefetch(tf.data.AUTOTUNE))\n",
    "test_iter = (test_ds.map(transform_test_fn, num_parallel_calls=tf.data.AUTOTUNE)\n",
    "             .batch(batch_size, drop_remainder=False)\n",
    "             .prefetch(tf.data.AUTOTUNE))"
   ],
   "execution_count": 11,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-12 fw=all -->\n",
    "\n",
    "## Defining the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-13 fw=jax -->\n",
    "\n",
    "We define the ResNet-18 model described in\n",
    "that section using Flax."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-defining-the-model-3"
   },
   "source": [
    "class Residual(nnx.Module):\n",
    "    def __init__(self, in_channels, num_channels, use_1x1conv=False,\n",
    "                 strides=1, *, rngs):\n",
    "        stride = (strides, strides)\n",
    "        self.conv1 = nnx.Conv(in_channels, num_channels, (3, 3),\n",
    "                              strides=stride, padding='same', rngs=rngs)\n",
    "        self.bn1 = nnx.BatchNorm(num_channels, rngs=rngs)\n",
    "        self.conv2 = nnx.Conv(num_channels, num_channels, (3, 3),\n",
    "                              padding='same', rngs=rngs)\n",
    "        self.bn2 = nnx.BatchNorm(num_channels, rngs=rngs)\n",
    "        self.conv3 = (nnx.Conv(in_channels, num_channels, (1, 1),\n",
    "                               strides=stride, rngs=rngs)\n",
    "                      if use_1x1conv else None)\n",
    "\n",
    "    def __call__(self, X):\n",
    "        Y = nnx.relu(self.bn1(self.conv1(X)))\n",
    "        Y = self.bn2(self.conv2(Y))\n",
    "        if self.conv3 is not None:\n",
    "            X = self.conv3(X)\n",
    "        return nnx.relu(Y + X)\n",
    "\n",
    "class ResNet18(nnx.Module):\n",
    "    def __init__(self, num_classes=10, *, rngs):\n",
    "        self.stem = nnx.List([\n",
    "            nnx.Conv(3, 64, (3, 3), padding='same', rngs=rngs),\n",
    "            nnx.BatchNorm(64, rngs=rngs), nnx.relu])\n",
    "        channels = [64, 128, 256, 512]\n",
    "        blocks, in_channels = [], 64\n",
    "        for i, num_channels in enumerate(channels):\n",
    "            for j in range(2):\n",
    "                strides = 2 if i > 0 and j == 0 else 1\n",
    "                use_1x1conv = i > 0 and j == 0\n",
    "                blocks.append(Residual(in_channels, num_channels,\n",
    "                                       use_1x1conv, strides, rngs=rngs))\n",
    "                in_channels = num_channels\n",
    "        self.blocks = nnx.List(blocks)\n",
    "        self.head = nnx.Linear(512, num_classes, rngs=rngs)\n",
    "\n",
    "    def __call__(self, X):\n",
    "        for layer in self.stem:\n",
    "            X = layer(X)\n",
    "        for block in self.blocks:\n",
    "            X = block(X)\n",
    "        X = jnp.mean(X, axis=(1, 2))  # Global average pooling\n",
    "        return self.head(X)\n",
    "\n",
    "def get_net():\n",
    "    return ResNet18(num_classes=10, rngs=nnx.Rngs(0))\n",
    "\n",
    "def loss_fn(logits, labels):\n",
    "    return optax.softmax_cross_entropy_with_integer_labels(logits, labels)"
   ],
   "execution_count": 12,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-14 fw=all -->\n",
    "\n",
    "## Defining the Training Function\n",
    "\n",
    "We will select models and tune hyperparameters according to the model's performance on the validation set.\n",
    "In the following, we define the model training function `train`."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-defining-the-training-function"
   },
   "source": [
    "def train(net, train_iter, valid_iter, num_epochs, lr, wd, lr_period,\n",
    "          lr_decay):\n",
    "    # `optax.exponential_decay.transition_steps` and Keras's\n",
    "    # `ExponentialDecay.decay_steps` count *gradient-update steps*, not\n",
    "    # epochs — unlike PyTorch's `StepLR(step_size=lr_period)`, which the\n",
    "    # PT tab steps once per epoch. Scale by `num_batches` so all four\n",
    "    # frameworks decay the LR every `lr_period` *epochs*.\n",
    "    num_batches = int(train_iter.cardinality().numpy())\n",
    "    schedule = optax.exponential_decay(\n",
    "        init_value=lr, transition_steps=lr_period * num_batches,\n",
    "        decay_rate=lr_decay, staircase=True)\n",
    "    tx = optax.chain(optax.add_decayed_weights(wd),\n",
    "                     optax.sgd(schedule, momentum=0.9))\n",
    "    optimizer = nnx.Optimizer(net, tx, wrt=nnx.Param)\n",
    "    train_net = nnx.view(net, use_running_average=False)\n",
    "    eval_net = nnx.view(net, use_running_average=True)\n",
    "    timer = d2l.Timer()\n",
    "    legend = ['train loss', 'train acc']\n",
    "    if valid_iter is not None:\n",
    "        legend.append('valid acc')\n",
    "    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n",
    "                            legend=legend)\n",
    "\n",
    "    @nnx.jit\n",
    "    def train_step(net, optimizer, X, y):\n",
    "        def compute_loss(net):\n",
    "            logits = net(X)\n",
    "            l = loss_fn(logits, y)\n",
    "            return l.mean(), (l.sum(), (logits.argmax(axis=-1) == y).sum())\n",
    "        (_, (l_sum, acc)), grads = nnx.value_and_grad(\n",
    "            compute_loss, has_aux=True)(net)\n",
    "        optimizer.update(net, grads)\n",
    "        return l_sum, acc\n",
    "\n",
    "    @nnx.jit\n",
    "    def eval_step(net, X, y):\n",
    "        logits = net(X)\n",
    "        return (logits.argmax(axis=-1) == y).sum()\n",
    "\n",
    "    for epoch in range(num_epochs):\n",
    "        loss_sum = jnp.array(0.0)\n",
    "        train_correct = jnp.array(0)\n",
    "        num_examples = 0\n",
    "        timer.start()\n",
    "        for i, (features, labels) in enumerate(train_iter):\n",
    "            X = jnp.array(features.numpy())  # Already NHWC from tf.data\n",
    "            y = jnp.array(labels.numpy())\n",
    "            l, acc = train_step(train_net, optimizer, X, y)\n",
    "            loss_sum += l\n",
    "            train_correct += acc\n",
    "            num_examples += len(labels)\n",
    "        loss_sum, train_correct = jax.device_get((loss_sum, train_correct))\n",
    "        timer.stop()\n",
    "        train_loss = float(loss_sum) / num_examples\n",
    "        train_acc = float(train_correct) / num_examples\n",
    "        if valid_iter is not None:\n",
    "            valid_correct = jnp.array(0)\n",
    "            valid_examples = 0\n",
    "            for features, labels in valid_iter:\n",
    "                X = jnp.array(features.numpy())  # Already NHWC\n",
    "                y = jnp.array(labels.numpy())\n",
    "                valid_correct += eval_step(eval_net, X, y)\n",
    "                valid_examples += len(labels)\n",
    "            valid_acc = float(jax.device_get(valid_correct)) / valid_examples\n",
    "        animator.add(epoch + 1, (train_loss, train_acc,\n",
    "                                 valid_acc if valid_iter is not None else None))\n",
    "    measures = (f'train loss {train_loss:.3f}, '\n",
    "                f'train acc {train_acc:.3f}')\n",
    "    if valid_iter is not None:\n",
    "        measures += f', valid acc {valid_acc:.3f}'\n",
    "    print(measures + f'\\n{num_examples * num_epochs / timer.sum():.1f}'\n",
    "          f' examples/sec')\n",
    "    return net"
   ],
   "execution_count": 13,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-15 fw=all -->\n",
    "\n",
    "## Training and Validating the Model\n",
    "\n",
    "Now, we can train and validate the model.\n",
    "All the following hyperparameters can be tuned.\n",
    "For example, we can increase the number of epochs.\n",
    "When `lr_period` and `lr_decay` are set to 4 and 0.9, respectively, the learning rate of the optimization algorithm will be multiplied by 0.9 after every 4 epochs. Just for ease of demonstration,\n",
    "we only train 20 epochs here.\n",
    "When `demo=True`, the tiny sample has only a few validation images per class,\n",
    "so the validation accuracy is a noisy smoke test of the pipeline rather than\n",
    "a meaningful competition score. Use the full dataset (`demo=False`) and the\n",
    "longer recipe in the exercises to assess model quality."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-training-and-validating-the-model"
   },
   "source": [
    "num_epochs, lr, wd = 20, 0.005, 5e-4\n",
    "lr_period, lr_decay = 4, 0.9\n",
    "net = get_net()\n",
    "net = train(net, train_iter, valid_iter, num_epochs, lr, wd, lr_period,\n",
    "            lr_decay)"
   ],
   "execution_count": 14,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "train loss 0.715, train acc 0.749, valid acc 0.312\n1177.5 examples/sec\n"
     ]
    },
    {
     "output_type": "display_data",
     "data": {
      "image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"186.772183pt\" viewBox=\"0 0 238.965625 186.772183\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n <metadata>\n  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n   <cc:Work>\n    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n    <dc:date>2026-07-19T14:24:46.232772</dc:date>\n    <dc:format>image/svg+xml</dc:format>\n    <dc:creator>\n     <cc:Agent>\n      <dc:title>Matplotlib v3.10.8, https://matplotlib.org/</dc:title>\n     </cc:Agent>\n    </dc:creator>\n   </cc:Work>\n  </rdf:RDF>\n </metadata>\n <defs>\n  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n </defs>\n <g id=\"figure_1\">\n  <g id=\"patch_1\">\n   <path d=\"M 0 186.772183 \nL 238.965625 186.772183 \nL 238.965625 0 \nL 0 0 \nz\n\" style=\"fill: #ffffff\"/>\n  </g>\n  <g id=\"axes_1\">\n   <g id=\"patch_2\">\n    <path d=\"M 30.103125 149.215933 \nL 225.403125 149.215933 \nL 225.403125 10.615933 \nL 30.103125 10.615933 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g id=\"matplotlib.axis_1\">\n    <g id=\"xtick_1\">\n     <g id=\"line2d_1\">\n      <path d=\"M 71.218914 149.215933 \nL 71.218914 10.615933 \n\" clip-path=\"url(#p0aa99ab2cb)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_2\">\n      <defs>\n       <path id=\"mcf17f94d21\" d=\"M 0 0 \nL 0 3.5 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#mcf17f94d21\" x=\"71.218914\" y=\"149.215933\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_1\">\n      <!-- 5 -->\n      <g transform=\"translate(68.037664 163.81437) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \nL 3169 4666 \nL 3169 4134 \nL 1269 4134 \nL 1269 2991 \nQ 1406 3038 1543 3061 \nQ 1681 3084 1819 3084 \nQ 2600 3084 3056 2656 \nQ 3513 2228 3513 1497 \nQ 3513 744 3044 326 \nQ 2575 -91 1722 -91 \nQ 1428 -91 1123 -41 \nQ 819 9 494 109 \nL 494 744 \nQ 775 591 1075 516 \nQ 1375 441 1709 441 \nQ 2250 441 2565 725 \nQ 2881 1009 2881 1497 \nQ 2881 1984 2565 2268 \nQ 2250 2553 1709 2553 \nQ 1456 2553 1204 2497 \nQ 953 2441 691 2322 \nL 691 4666 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_2\">\n     <g id=\"line2d_3\">\n      <path d=\"M 122.613651 149.215933 \nL 122.613651 10.615933 \n\" clip-path=\"url(#p0aa99ab2cb)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_4\">\n      <g>\n       <use xlink:href=\"#mcf17f94d21\" x=\"122.613651\" y=\"149.215933\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_2\">\n      <!-- 10 -->\n      <g transform=\"translate(116.251151 163.81437) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-31\" d=\"M 794 531 \nL 1825 531 \nL 1825 4091 \nL 703 3866 \nL 703 4441 \nL 1819 4666 \nL 2450 4666 \nL 2450 531 \nL 3481 531 \nL 3481 0 \nL 794 0 \nL 794 531 \nz\n\" transform=\"scale(0.015625)\"/>\n        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \nQ 1547 4250 1301 3770 \nQ 1056 3291 1056 2328 \nQ 1056 1369 1301 889 \nQ 1547 409 2034 409 \nQ 2525 409 2770 889 \nQ 3016 1369 3016 2328 \nQ 3016 3291 2770 3770 \nQ 2525 4250 2034 4250 \nz\nM 2034 4750 \nQ 2819 4750 3233 4129 \nQ 3647 3509 3647 2328 \nQ 3647 1150 3233 529 \nQ 2819 -91 2034 -91 \nQ 1250 -91 836 529 \nQ 422 1150 422 2328 \nQ 422 3509 836 4129 \nQ 1250 4750 2034 4750 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(63.623047 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_3\">\n     <g id=\"line2d_5\">\n      <path d=\"M 174.008388 149.215933 \nL 174.008388 10.615933 \n\" clip-path=\"url(#p0aa99ab2cb)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_6\">\n      <g>\n       <use xlink:href=\"#mcf17f94d21\" x=\"174.008388\" y=\"149.215933\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_3\">\n      <!-- 15 -->\n      <g transform=\"translate(167.645888 163.81437) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-35\" transform=\"translate(63.623047 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_4\">\n     <g id=\"line2d_7\">\n      <path d=\"M 225.403125 149.215933 \nL 225.403125 10.615933 \n\" clip-path=\"url(#p0aa99ab2cb)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_8\">\n      <g>\n       <use xlink:href=\"#mcf17f94d21\" x=\"225.403125\" y=\"149.215933\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_4\">\n      <!-- 20 -->\n      <g transform=\"translate(219.040625 163.81437) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \nL 3431 531 \nL 3431 0 \nL 469 0 \nL 469 531 \nQ 828 903 1448 1529 \nQ 2069 2156 2228 2338 \nQ 2531 2678 2651 2914 \nQ 2772 3150 2772 3378 \nQ 2772 3750 2511 3984 \nQ 2250 4219 1831 4219 \nQ 1534 4219 1204 4116 \nQ 875 4013 500 3803 \nL 500 4441 \nQ 881 4594 1212 4672 \nQ 1544 4750 1819 4750 \nQ 2544 4750 2975 4387 \nQ 3406 4025 3406 3419 \nQ 3406 3131 3298 2873 \nQ 3191 2616 2906 2266 \nQ 2828 2175 2409 1742 \nQ 1991 1309 1228 531 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-32\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(63.623047 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"text_5\">\n     <!-- epoch -->\n     <g transform=\"translate(112.525 177.492495) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \nL 3597 1613 \nL 953 1613 \nQ 991 1019 1311 708 \nQ 1631 397 2203 397 \nQ 2534 397 2845 478 \nQ 3156 559 3463 722 \nL 3463 178 \nQ 3153 47 2828 -22 \nQ 2503 -91 2169 -91 \nQ 1331 -91 842 396 \nQ 353 884 353 1716 \nQ 353 2575 817 3079 \nQ 1281 3584 2069 3584 \nQ 2775 3584 3186 3129 \nQ 3597 2675 3597 1894 \nz\nM 3022 2063 \nQ 3016 2534 2758 2815 \nQ 2500 3097 2075 3097 \nQ 1594 3097 1305 2825 \nQ 1016 2553 972 2059 \nL 3022 2063 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \nL 1159 -1331 \nL 581 -1331 \nL 581 3500 \nL 1159 3500 \nL 1159 2969 \nQ 1341 3281 1617 3432 \nQ 1894 3584 2278 3584 \nQ 2916 3584 3314 3078 \nQ 3713 2572 3713 1747 \nQ 3713 922 3314 415 \nQ 2916 -91 2278 -91 \nQ 1894 -91 1617 61 \nQ 1341 213 1159 525 \nz\nM 3116 1747 \nQ 3116 2381 2855 2742 \nQ 2594 3103 2138 3103 \nQ 1681 3103 1420 2742 \nQ 1159 2381 1159 1747 \nQ 1159 1113 1420 752 \nQ 1681 391 2138 391 \nQ 2594 391 2855 752 \nQ 3116 1113 3116 1747 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \nQ 1497 3097 1228 2736 \nQ 959 2375 959 1747 \nQ 959 1119 1226 758 \nQ 1494 397 1959 397 \nQ 2419 397 2687 759 \nQ 2956 1122 2956 1747 \nQ 2956 2369 2687 2733 \nQ 2419 3097 1959 3097 \nz\nM 1959 3584 \nQ 2709 3584 3137 3096 \nQ 3566 2609 3566 1747 \nQ 3566 888 3137 398 \nQ 2709 -91 1959 -91 \nQ 1206 -91 779 398 \nQ 353 888 353 1747 \nQ 353 2609 779 3096 \nQ 1206 3584 1959 3584 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \nL 3122 2828 \nQ 2878 2963 2633 3030 \nQ 2388 3097 2138 3097 \nQ 1578 3097 1268 2742 \nQ 959 2388 959 1747 \nQ 959 1106 1268 751 \nQ 1578 397 2138 397 \nQ 2388 397 2633 464 \nQ 2878 531 3122 666 \nL 3122 134 \nQ 2881 22 2623 -34 \nQ 2366 -91 2075 -91 \nQ 1284 -91 818 406 \nQ 353 903 353 1747 \nQ 353 2603 823 3093 \nQ 1294 3584 2113 3584 \nQ 2378 3584 2631 3529 \nQ 2884 3475 3122 3366 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \nL 3513 0 \nL 2938 0 \nL 2938 2094 \nQ 2938 2591 2744 2837 \nQ 2550 3084 2163 3084 \nQ 1697 3084 1428 2787 \nQ 1159 2491 1159 1978 \nL 1159 0 \nL 581 0 \nL 581 4863 \nL 1159 4863 \nL 1159 2956 \nQ 1366 3272 1645 3428 \nQ 1925 3584 2291 3584 \nQ 2894 3584 3203 3211 \nQ 3513 2838 3513 2113 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-65\"/>\n      <use xlink:href=\"#DejaVuSans-70\" transform=\"translate(61.523438 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" transform=\"translate(125 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(186.181641 0)\"/>\n      <use xlink:href=\"#DejaVuSans-68\" transform=\"translate(241.162109 0)\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_2\">\n    <g id=\"ytick_1\">\n     <g id=\"line2d_9\">\n      <path d=\"M 30.103125 122.822658 \nL 225.403125 122.822658 \n\" clip-path=\"url(#p0aa99ab2cb)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_10\">\n      <defs>\n       <path id=\"m53736d9e81\" d=\"M 0 0 \nL -3.5 0 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#m53736d9e81\" x=\"30.103125\" y=\"122.822658\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_6\">\n      <!-- 0.5 -->\n      <g transform=\"translate(7.2 126.621877) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \nL 1344 794 \nL 1344 0 \nL 684 0 \nL 684 794 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-30\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-35\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_2\">\n     <g id=\"line2d_11\">\n      <path d=\"M 30.103125 94.866798 \nL 225.403125 94.866798 \n\" clip-path=\"url(#p0aa99ab2cb)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_12\">\n      <g>\n       <use xlink:href=\"#m53736d9e81\" x=\"30.103125\" y=\"94.866798\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_7\">\n      <!-- 1.0 -->\n      <g transform=\"translate(7.2 98.666017) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_3\">\n     <g id=\"line2d_13\">\n      <path d=\"M 30.103125 66.910939 \nL 225.403125 66.910939 \n\" clip-path=\"url(#p0aa99ab2cb)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_14\">\n      <g>\n       <use xlink:href=\"#m53736d9e81\" x=\"30.103125\" y=\"66.910939\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_8\">\n      <!-- 1.5 -->\n      <g transform=\"translate(7.2 70.710157) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-35\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_4\">\n     <g id=\"line2d_15\">\n      <path d=\"M 30.103125 38.955079 \nL 225.403125 38.955079 \n\" clip-path=\"url(#p0aa99ab2cb)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_16\">\n      <g>\n       <use xlink:href=\"#m53736d9e81\" x=\"30.103125\" y=\"38.955079\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_9\">\n      <!-- 2.0 -->\n      <g transform=\"translate(7.2 42.754297) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-32\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_5\">\n     <g id=\"line2d_17\">\n      <path d=\"M 30.103125 10.999219 \nL 225.403125 10.999219 \n\" clip-path=\"url(#p0aa99ab2cb)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_18\">\n      <g>\n       <use xlink:href=\"#m53736d9e81\" x=\"30.103125\" y=\"10.999219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_10\">\n      <!-- 2.5 -->\n      <g transform=\"translate(7.2 14.798438) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-32\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-35\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"line2d_19\">\n    <path d=\"M 30.103125 16.915933 \nL 40.382072 43.346346 \nL 50.66102 46.400484 \nL 60.939967 49.350622 \nL 71.218914 62.357619 \nL 81.497862 67.205952 \nL 91.776809 70.555994 \nL 102.055757 73.479874 \nL 112.334704 76.030702 \nL 122.613651 80.109124 \nL 132.892599 85.266509 \nL 143.171546 88.333379 \nL 153.450493 91.996917 \nL 163.729441 94.013269 \nL 174.008388 97.637966 \nL 184.287336 99.274481 \nL 194.566283 102.858015 \nL 204.84523 106.299621 \nL 215.124178 109.617571 \nL 225.403125 110.774356 \n\" clip-path=\"url(#p0aa99ab2cb)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n   </g>\n   <g id=\"line2d_20\">\n    <path d=\"M 30.103125 140.607078 \nL 40.382072 133.867719 \nL 50.66102 133.368507 \nL 60.939967 130.622842 \nL 71.218914 127.315564 \nL 81.497862 126.067535 \nL 91.776809 124.257892 \nL 102.055757 121.449826 \nL 112.334704 124.133089 \nL 122.613651 120.201796 \nL 132.892599 118.828964 \nL 143.171546 116.95692 \nL 153.450493 116.208102 \nL 163.729441 115.334482 \nL 174.008388 114.460861 \nL 184.287336 112.963226 \nL 194.566283 112.588817 \nL 204.84523 111.091181 \nL 215.124178 109.156736 \nL 225.403125 108.90713 \n\" clip-path=\"url(#p0aa99ab2cb)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n   </g>\n   <g id=\"line2d_21\">\n    <path d=\"M 30.103125 142.915933 \nL 40.382072 138.54783 \nL 50.66102 140.295071 \nL 60.939967 142.042312 \nL 71.218914 133.306106 \nL 81.497862 130.685244 \nL 91.776809 132.432485 \nL 102.055757 137.674209 \nL 112.334704 124.5699 \nL 122.613651 135.053347 \nL 132.892599 129.811623 \nL 143.171546 123.696279 \nL 153.450493 129.811623 \nL 163.729441 125.44352 \nL 174.008388 128.938003 \nL 184.287336 127.190761 \nL 194.566283 128.064382 \nL 204.84523 128.064382 \nL 215.124178 133.306106 \nL 225.403125 133.306106 \n\" clip-path=\"url(#p0aa99ab2cb)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n   </g>\n   <g id=\"patch_3\">\n    <path d=\"M 30.103125 149.215933 \nL 30.103125 10.615933 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_4\">\n    <path d=\"M 225.403125 149.215933 \nL 225.403125 10.615933 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_5\">\n    <path d=\"M 30.103125 149.215933 \nL 225.403125 149.215933 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_6\">\n    <path d=\"M 30.103125 10.615933 \nL 225.403125 10.615933 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"legend_1\">\n    <g id=\"patch_7\">\n     <path d=\"M 140.634375 62.650308 \nL 218.403125 62.650308 \nQ 220.403125 62.650308 220.403125 60.650308 \nL 220.403125 17.615933 \nQ 220.403125 15.615933 218.403125 15.615933 \nL 140.634375 15.615933 \nQ 138.634375 15.615933 138.634375 17.615933 \nL 138.634375 60.650308 \nQ 138.634375 62.650308 140.634375 62.650308 \nz\n\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n    </g>\n    <g id=\"line2d_22\">\n     <path d=\"M 142.634375 23.71437 \nL 152.634375 23.71437 \nL 162.634375 23.71437 \n\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n    </g>\n    <g id=\"text_11\">\n     <!-- train loss -->\n     <g transform=\"translate(170.634375 27.21437) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \nL 1172 3500 \nL 2356 3500 \nL 2356 3053 \nL 1172 3053 \nL 1172 1153 \nQ 1172 725 1289 603 \nQ 1406 481 1766 481 \nL 2356 481 \nL 2356 0 \nL 1766 0 \nQ 1100 0 847 248 \nQ 594 497 594 1153 \nL 594 3053 \nL 172 3053 \nL 172 3500 \nL 594 3500 \nL 594 4494 \nL 1172 4494 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \nQ 2534 3019 2420 3045 \nQ 2306 3072 2169 3072 \nQ 1681 3072 1420 2755 \nQ 1159 2438 1159 1844 \nL 1159 0 \nL 581 0 \nL 581 3500 \nL 1159 3500 \nL 1159 2956 \nQ 1341 3275 1631 3429 \nQ 1922 3584 2338 3584 \nQ 2397 3584 2469 3576 \nQ 2541 3569 2628 3553 \nL 2631 2963 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \nQ 1497 1759 1228 1600 \nQ 959 1441 959 1056 \nQ 959 750 1161 570 \nQ 1363 391 1709 391 \nQ 2188 391 2477 730 \nQ 2766 1069 2766 1631 \nL 2766 1759 \nL 2194 1759 \nz\nM 3341 1997 \nL 3341 0 \nL 2766 0 \nL 2766 531 \nQ 2569 213 2275 61 \nQ 1981 -91 1556 -91 \nQ 1019 -91 701 211 \nQ 384 513 384 1019 \nQ 384 1609 779 1909 \nQ 1175 2209 1959 2209 \nL 2766 2209 \nL 2766 2266 \nQ 2766 2663 2505 2880 \nQ 2244 3097 1772 3097 \nQ 1472 3097 1187 3025 \nQ 903 2953 641 2809 \nL 641 3341 \nQ 956 3463 1253 3523 \nQ 1550 3584 1831 3584 \nQ 2591 3584 2966 3190 \nQ 3341 2797 3341 1997 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \nL 1178 3500 \nL 1178 0 \nL 603 0 \nL 603 3500 \nz\nM 603 4863 \nL 1178 4863 \nL 1178 4134 \nL 603 4134 \nL 603 4863 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \nL 3513 0 \nL 2938 0 \nL 2938 2094 \nQ 2938 2591 2744 2837 \nQ 2550 3084 2163 3084 \nQ 1697 3084 1428 2787 \nQ 1159 2491 1159 1978 \nL 1159 0 \nL 581 0 \nL 581 3500 \nL 1159 3500 \nL 1159 2956 \nQ 1366 3272 1645 3428 \nQ 1925 3584 2291 3584 \nQ 2894 3584 3203 3211 \nQ 3513 2838 3513 2113 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \nL 1178 4863 \nL 1178 0 \nL 603 0 \nL 603 4863 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \nL 2834 2853 \nQ 2591 2978 2328 3040 \nQ 2066 3103 1784 3103 \nQ 1356 3103 1142 2972 \nQ 928 2841 928 2578 \nQ 928 2378 1081 2264 \nQ 1234 2150 1697 2047 \nL 1894 2003 \nQ 2506 1872 2764 1633 \nQ 3022 1394 3022 966 \nQ 3022 478 2636 193 \nQ 2250 -91 1575 -91 \nQ 1294 -91 989 -36 \nQ 684 19 347 128 \nL 347 722 \nQ 666 556 975 473 \nQ 1284 391 1588 391 \nQ 1994 391 2212 530 \nQ 2431 669 2431 922 \nQ 2431 1156 2273 1281 \nQ 2116 1406 1581 1522 \nL 1381 1569 \nQ 847 1681 609 1914 \nQ 372 2147 372 2553 \nQ 372 3047 722 3315 \nQ 1072 3584 1716 3584 \nQ 2034 3584 2315 3537 \nQ 2597 3491 2834 3397 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-74\"/>\n      <use xlink:href=\"#DejaVuSans-72\" transform=\"translate(39.208984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(80.322266 0)\"/>\n      <use xlink:href=\"#DejaVuSans-69\" transform=\"translate(141.601562 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6e\" transform=\"translate(169.384766 0)\"/>\n      <use xlink:href=\"#DejaVuSans-20\" transform=\"translate(232.763672 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" transform=\"translate(264.550781 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" transform=\"translate(292.333984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-73\" transform=\"translate(353.515625 0)\"/>\n      <use xlink:href=\"#DejaVuSans-73\" transform=\"translate(405.615234 0)\"/>\n     </g>\n    </g>\n    <g id=\"line2d_23\">\n     <path d=\"M 142.634375 38.392495 \nL 152.634375 38.392495 \nL 162.634375 38.392495 \n\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n    </g>\n    <g id=\"text_12\">\n     <!-- train acc -->\n     <g transform=\"translate(170.634375 41.892495) scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-74\"/>\n      <use xlink:href=\"#DejaVuSans-72\" transform=\"translate(39.208984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(80.322266 0)\"/>\n      <use xlink:href=\"#DejaVuSans-69\" transform=\"translate(141.601562 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6e\" transform=\"translate(169.384766 0)\"/>\n      <use xlink:href=\"#DejaVuSans-20\" transform=\"translate(232.763672 0)\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(264.550781 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(325.830078 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(380.810547 0)\"/>\n     </g>\n    </g>\n    <g id=\"line2d_24\">\n     <path d=\"M 142.634375 53.07062 \nL 152.634375 53.07062 \nL 162.634375 53.07062 \n\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n    </g>\n    <g id=\"text_13\">\n     <!-- valid acc -->\n     <g transform=\"translate(170.634375 56.57062) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-76\" d=\"M 191 3500 \nL 800 3500 \nL 1894 563 \nL 2988 3500 \nL 3597 3500 \nL 2284 0 \nL 1503 0 \nL 191 3500 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-64\" d=\"M 2906 2969 \nL 2906 4863 \nL 3481 4863 \nL 3481 0 \nL 2906 0 \nL 2906 525 \nQ 2725 213 2448 61 \nQ 2172 -91 1784 -91 \nQ 1150 -91 751 415 \nQ 353 922 353 1747 \nQ 353 2572 751 3078 \nQ 1150 3584 1784 3584 \nQ 2172 3584 2448 3432 \nQ 2725 3281 2906 2969 \nz\nM 947 1747 \nQ 947 1113 1208 752 \nQ 1469 391 1925 391 \nQ 2381 391 2643 752 \nQ 2906 1113 2906 1747 \nQ 2906 2381 2643 2742 \nQ 2381 3103 1925 3103 \nQ 1469 3103 1208 2742 \nQ 947 2381 947 1747 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-76\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(59.179688 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" transform=\"translate(120.458984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-69\" transform=\"translate(148.242188 0)\"/>\n      <use xlink:href=\"#DejaVuSans-64\" transform=\"translate(176.025391 0)\"/>\n      <use xlink:href=\"#DejaVuSans-20\" transform=\"translate(239.501953 0)\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(271.289062 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(332.568359 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(387.548828 0)\"/>\n     </g>\n    </g>\n   </g>\n  </g>\n </g>\n <defs>\n  <clipPath id=\"p0aa99ab2cb\">\n   <rect x=\"30.103125\" y=\"10.615933\" width=\"195.3\" height=\"138.6\"/>\n  </clipPath>\n </defs>\n</svg>\n"
     },
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-16 fw=all -->\n",
    "\n",
    "## Classifying the Testing Set and Submitting Results on Kaggle\n",
    "\n",
    "After obtaining a promising model with hyperparameters,\n",
    "we use all the labeled data (including the validation set) to retrain the model and classify the testing set."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kaggle-cifar10-classifying-the-testing-set-and-submitting-results-on-kaggle"
   },
   "source": [
    "net, preds = get_net(), []\n",
    "net = train(net, train_valid_iter, None, num_epochs, lr, wd, lr_period,\n",
    "            lr_decay)\n",
    "\n",
    "for X, _ in test_iter:\n",
    "    X_jax = jnp.array(X.numpy())  # Already NHWC from tf.data\n",
    "    y_hat = nnx.view(net, use_running_average=True)(X_jax)\n",
    "    preds.extend(np.array(y_hat.argmax(axis=-1)))\n",
    "# Get class names from the train_valid dataset directory\n",
    "class_names = sorted(os.listdir(\n",
    "    os.path.join(data_dir, 'train_valid_test', 'train_valid')))\n",
    "sorted_ids = list(range(1, sum(1 for _ in test_ds) + 1))\n",
    "sorted_ids.sort(key=lambda x: str(x))\n",
    "df = pd.DataFrame({'id': sorted_ids, 'label': preds})\n",
    "df['label'] = df['label'].apply(lambda x: class_names[x])\n",
    "df.to_csv('submission.csv', index=False)"
   ],
   "execution_count": 15,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "train loss 0.631, train acc 0.777\n1380.0 examples/sec\n"
     ]
    },
    {
     "output_type": "display_data",
     "data": {
      "image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n <metadata>\n  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n   <cc:Work>\n    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n    <dc:date>2026-07-19T14:25:08.533200</dc:date>\n    <dc:format>image/svg+xml</dc:format>\n    <dc:creator>\n     <cc:Agent>\n      <dc:title>Matplotlib v3.10.8, https://matplotlib.org/</dc:title>\n     </cc:Agent>\n    </dc:creator>\n   </cc:Work>\n  </rdf:RDF>\n </metadata>\n <defs>\n  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n </defs>\n <g id=\"figure_1\">\n  <g id=\"patch_1\">\n   <path d=\"M 0 183.35625 \nL 238.965625 183.35625 \nL 238.965625 0 \nL 0 0 \nz\n\" style=\"fill: #ffffff\"/>\n  </g>\n  <g id=\"axes_1\">\n   <g id=\"patch_2\">\n    <path d=\"M 30.103125 145.8 \nL 225.403125 145.8 \nL 225.403125 7.2 \nL 30.103125 7.2 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g id=\"matplotlib.axis_1\">\n    <g id=\"xtick_1\">\n     <g id=\"line2d_1\">\n      <path d=\"M 71.218914 145.8 \nL 71.218914 7.2 \n\" clip-path=\"url(#pb0c00fea5f)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_2\">\n      <defs>\n       <path id=\"mfa85df3487\" d=\"M 0 0 \nL 0 3.5 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#mfa85df3487\" x=\"71.218914\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_1\">\n      <!-- 5 -->\n      <g transform=\"translate(68.037664 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \nL 3169 4666 \nL 3169 4134 \nL 1269 4134 \nL 1269 2991 \nQ 1406 3038 1543 3061 \nQ 1681 3084 1819 3084 \nQ 2600 3084 3056 2656 \nQ 3513 2228 3513 1497 \nQ 3513 744 3044 326 \nQ 2575 -91 1722 -91 \nQ 1428 -91 1123 -41 \nQ 819 9 494 109 \nL 494 744 \nQ 775 591 1075 516 \nQ 1375 441 1709 441 \nQ 2250 441 2565 725 \nQ 2881 1009 2881 1497 \nQ 2881 1984 2565 2268 \nQ 2250 2553 1709 2553 \nQ 1456 2553 1204 2497 \nQ 953 2441 691 2322 \nL 691 4666 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_2\">\n     <g id=\"line2d_3\">\n      <path d=\"M 122.613651 145.8 \nL 122.613651 7.2 \n\" clip-path=\"url(#pb0c00fea5f)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_4\">\n      <g>\n       <use xlink:href=\"#mfa85df3487\" x=\"122.613651\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_2\">\n      <!-- 10 -->\n      <g transform=\"translate(116.251151 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-31\" d=\"M 794 531 \nL 1825 531 \nL 1825 4091 \nL 703 3866 \nL 703 4441 \nL 1819 4666 \nL 2450 4666 \nL 2450 531 \nL 3481 531 \nL 3481 0 \nL 794 0 \nL 794 531 \nz\n\" transform=\"scale(0.015625)\"/>\n        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \nQ 1547 4250 1301 3770 \nQ 1056 3291 1056 2328 \nQ 1056 1369 1301 889 \nQ 1547 409 2034 409 \nQ 2525 409 2770 889 \nQ 3016 1369 3016 2328 \nQ 3016 3291 2770 3770 \nQ 2525 4250 2034 4250 \nz\nM 2034 4750 \nQ 2819 4750 3233 4129 \nQ 3647 3509 3647 2328 \nQ 3647 1150 3233 529 \nQ 2819 -91 2034 -91 \nQ 1250 -91 836 529 \nQ 422 1150 422 2328 \nQ 422 3509 836 4129 \nQ 1250 4750 2034 4750 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(63.623047 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_3\">\n     <g id=\"line2d_5\">\n      <path d=\"M 174.008388 145.8 \nL 174.008388 7.2 \n\" clip-path=\"url(#pb0c00fea5f)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_6\">\n      <g>\n       <use xlink:href=\"#mfa85df3487\" x=\"174.008388\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_3\">\n      <!-- 15 -->\n      <g transform=\"translate(167.645888 160.398438) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-35\" transform=\"translate(63.623047 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_4\">\n     <g id=\"line2d_7\">\n      <path d=\"M 225.403125 145.8 \nL 225.403125 7.2 \n\" clip-path=\"url(#pb0c00fea5f)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_8\">\n      <g>\n       <use xlink:href=\"#mfa85df3487\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_4\">\n      <!-- 20 -->\n      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \nL 3431 531 \nL 3431 0 \nL 469 0 \nL 469 531 \nQ 828 903 1448 1529 \nQ 2069 2156 2228 2338 \nQ 2531 2678 2651 2914 \nQ 2772 3150 2772 3378 \nQ 2772 3750 2511 3984 \nQ 2250 4219 1831 4219 \nQ 1534 4219 1204 4116 \nQ 875 4013 500 3803 \nL 500 4441 \nQ 881 4594 1212 4672 \nQ 1544 4750 1819 4750 \nQ 2544 4750 2975 4387 \nQ 3406 4025 3406 3419 \nQ 3406 3131 3298 2873 \nQ 3191 2616 2906 2266 \nQ 2828 2175 2409 1742 \nQ 1991 1309 1228 531 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-32\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(63.623047 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"text_5\">\n     <!-- epoch -->\n     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \nL 3597 1613 \nL 953 1613 \nQ 991 1019 1311 708 \nQ 1631 397 2203 397 \nQ 2534 397 2845 478 \nQ 3156 559 3463 722 \nL 3463 178 \nQ 3153 47 2828 -22 \nQ 2503 -91 2169 -91 \nQ 1331 -91 842 396 \nQ 353 884 353 1716 \nQ 353 2575 817 3079 \nQ 1281 3584 2069 3584 \nQ 2775 3584 3186 3129 \nQ 3597 2675 3597 1894 \nz\nM 3022 2063 \nQ 3016 2534 2758 2815 \nQ 2500 3097 2075 3097 \nQ 1594 3097 1305 2825 \nQ 1016 2553 972 2059 \nL 3022 2063 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \nL 1159 -1331 \nL 581 -1331 \nL 581 3500 \nL 1159 3500 \nL 1159 2969 \nQ 1341 3281 1617 3432 \nQ 1894 3584 2278 3584 \nQ 2916 3584 3314 3078 \nQ 3713 2572 3713 1747 \nQ 3713 922 3314 415 \nQ 2916 -91 2278 -91 \nQ 1894 -91 1617 61 \nQ 1341 213 1159 525 \nz\nM 3116 1747 \nQ 3116 2381 2855 2742 \nQ 2594 3103 2138 3103 \nQ 1681 3103 1420 2742 \nQ 1159 2381 1159 1747 \nQ 1159 1113 1420 752 \nQ 1681 391 2138 391 \nQ 2594 391 2855 752 \nQ 3116 1113 3116 1747 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \nQ 1497 3097 1228 2736 \nQ 959 2375 959 1747 \nQ 959 1119 1226 758 \nQ 1494 397 1959 397 \nQ 2419 397 2687 759 \nQ 2956 1122 2956 1747 \nQ 2956 2369 2687 2733 \nQ 2419 3097 1959 3097 \nz\nM 1959 3584 \nQ 2709 3584 3137 3096 \nQ 3566 2609 3566 1747 \nQ 3566 888 3137 398 \nQ 2709 -91 1959 -91 \nQ 1206 -91 779 398 \nQ 353 888 353 1747 \nQ 353 2609 779 3096 \nQ 1206 3584 1959 3584 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \nL 3122 2828 \nQ 2878 2963 2633 3030 \nQ 2388 3097 2138 3097 \nQ 1578 3097 1268 2742 \nQ 959 2388 959 1747 \nQ 959 1106 1268 751 \nQ 1578 397 2138 397 \nQ 2388 397 2633 464 \nQ 2878 531 3122 666 \nL 3122 134 \nQ 2881 22 2623 -34 \nQ 2366 -91 2075 -91 \nQ 1284 -91 818 406 \nQ 353 903 353 1747 \nQ 353 2603 823 3093 \nQ 1294 3584 2113 3584 \nQ 2378 3584 2631 3529 \nQ 2884 3475 3122 3366 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \nL 3513 0 \nL 2938 0 \nL 2938 2094 \nQ 2938 2591 2744 2837 \nQ 2550 3084 2163 3084 \nQ 1697 3084 1428 2787 \nQ 1159 2491 1159 1978 \nL 1159 0 \nL 581 0 \nL 581 4863 \nL 1159 4863 \nL 1159 2956 \nQ 1366 3272 1645 3428 \nQ 1925 3584 2291 3584 \nQ 2894 3584 3203 3211 \nQ 3513 2838 3513 2113 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-65\"/>\n      <use xlink:href=\"#DejaVuSans-70\" transform=\"translate(61.523438 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" transform=\"translate(125 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(186.181641 0)\"/>\n      <use xlink:href=\"#DejaVuSans-68\" transform=\"translate(241.162109 0)\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_2\">\n    <g id=\"ytick_1\">\n     <g id=\"line2d_9\">\n      <path d=\"M 30.103125 120.661931 \nL 225.403125 120.661931 \n\" clip-path=\"url(#pb0c00fea5f)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_10\">\n      <defs>\n       <path id=\"m5cb45c3d2c\" d=\"M 0 0 \nL -3.5 0 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#m5cb45c3d2c\" x=\"30.103125\" y=\"120.661931\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_6\">\n      <!-- 0.5 -->\n      <g transform=\"translate(7.2 124.46115) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \nL 1344 794 \nL 1344 0 \nL 684 0 \nL 684 794 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-30\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-35\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_2\">\n     <g id=\"line2d_11\">\n      <path d=\"M 30.103125 90.3253 \nL 225.403125 90.3253 \n\" clip-path=\"url(#pb0c00fea5f)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_12\">\n      <g>\n       <use xlink:href=\"#m5cb45c3d2c\" x=\"30.103125\" y=\"90.3253\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_7\">\n      <!-- 1.0 -->\n      <g transform=\"translate(7.2 94.124519) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_3\">\n     <g id=\"line2d_13\">\n      <path d=\"M 30.103125 59.988669 \nL 225.403125 59.988669 \n\" clip-path=\"url(#pb0c00fea5f)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_14\">\n      <g>\n       <use xlink:href=\"#m5cb45c3d2c\" x=\"30.103125\" y=\"59.988669\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_8\">\n      <!-- 1.5 -->\n      <g transform=\"translate(7.2 63.787888) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-35\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_4\">\n     <g id=\"line2d_15\">\n      <path d=\"M 30.103125 29.652038 \nL 225.403125 29.652038 \n\" clip-path=\"url(#pb0c00fea5f)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_16\">\n      <g>\n       <use xlink:href=\"#m5cb45c3d2c\" x=\"30.103125\" y=\"29.652038\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_9\">\n      <!-- 2.0 -->\n      <g transform=\"translate(7.2 33.451257) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-32\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"line2d_17\">\n    <path d=\"M 30.103125 13.5 \nL 40.382072 28.721332 \nL 50.66102 36.098611 \nL 60.939967 47.010462 \nL 71.218914 57.813262 \nL 81.497862 62.046641 \nL 91.776809 65.120045 \nL 102.055757 68.13422 \nL 112.334704 75.519391 \nL 122.613651 79.009644 \nL 132.892599 81.567242 \nL 143.171546 83.325016 \nL 153.450493 89.479193 \nL 163.729441 93.024807 \nL 174.008388 94.130183 \nL 184.287336 97.092118 \nL 194.566283 98.519937 \nL 204.84523 106.998448 \nL 215.124178 108.856799 \nL 225.403125 112.68498 \n\" clip-path=\"url(#pb0c00fea5f)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n   </g>\n   <g id=\"line2d_18\">\n    <path d=\"M 30.103125 139.5 \nL 40.382072 133.383744 \nL 50.66102 130.998404 \nL 60.939967 129.71399 \nL 71.218914 123.658896 \nL 81.497862 124.515172 \nL 91.776809 122.863783 \nL 102.055757 120.478443 \nL 112.334704 117.970778 \nL 122.613651 117.05334 \nL 132.892599 116.502877 \nL 143.171546 114.851487 \nL 153.450493 112.833123 \nL 163.729441 111.120571 \nL 174.008388 110.875921 \nL 184.287336 110.203133 \nL 194.566283 109.346857 \nL 204.84523 105.921753 \nL 215.124178 106.166404 \nL 225.403125 103.842226 \n\" clip-path=\"url(#pb0c00fea5f)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n   </g>\n   <g id=\"line2d_19\"/>\n   <g id=\"patch_3\">\n    <path d=\"M 30.103125 145.8 \nL 30.103125 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_4\">\n    <path d=\"M 225.403125 145.8 \nL 225.403125 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_5\">\n    <path d=\"M 30.103125 145.8 \nL 225.403125 145.8 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_6\">\n    <path d=\"M 30.103125 7.2 \nL 225.403125 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"legend_1\">\n    <g id=\"patch_7\">\n     <path d=\"M 140.634375 44.55625 \nL 218.403125 44.55625 \nQ 220.403125 44.55625 220.403125 42.55625 \nL 220.403125 14.2 \nQ 220.403125 12.2 218.403125 12.2 \nL 140.634375 12.2 \nQ 138.634375 12.2 138.634375 14.2 \nL 138.634375 42.55625 \nQ 138.634375 44.55625 140.634375 44.55625 \nz\n\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n    </g>\n    <g id=\"line2d_20\">\n     <path d=\"M 142.634375 20.298438 \nL 152.634375 20.298438 \nL 162.634375 20.298438 \n\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n    </g>\n    <g id=\"text_10\">\n     <!-- train loss -->\n     <g transform=\"translate(170.634375 23.798438) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \nL 1172 3500 \nL 2356 3500 \nL 2356 3053 \nL 1172 3053 \nL 1172 1153 \nQ 1172 725 1289 603 \nQ 1406 481 1766 481 \nL 2356 481 \nL 2356 0 \nL 1766 0 \nQ 1100 0 847 248 \nQ 594 497 594 1153 \nL 594 3053 \nL 172 3053 \nL 172 3500 \nL 594 3500 \nL 594 4494 \nL 1172 4494 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \nQ 2534 3019 2420 3045 \nQ 2306 3072 2169 3072 \nQ 1681 3072 1420 2755 \nQ 1159 2438 1159 1844 \nL 1159 0 \nL 581 0 \nL 581 3500 \nL 1159 3500 \nL 1159 2956 \nQ 1341 3275 1631 3429 \nQ 1922 3584 2338 3584 \nQ 2397 3584 2469 3576 \nQ 2541 3569 2628 3553 \nL 2631 2963 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \nQ 1497 1759 1228 1600 \nQ 959 1441 959 1056 \nQ 959 750 1161 570 \nQ 1363 391 1709 391 \nQ 2188 391 2477 730 \nQ 2766 1069 2766 1631 \nL 2766 1759 \nL 2194 1759 \nz\nM 3341 1997 \nL 3341 0 \nL 2766 0 \nL 2766 531 \nQ 2569 213 2275 61 \nQ 1981 -91 1556 -91 \nQ 1019 -91 701 211 \nQ 384 513 384 1019 \nQ 384 1609 779 1909 \nQ 1175 2209 1959 2209 \nL 2766 2209 \nL 2766 2266 \nQ 2766 2663 2505 2880 \nQ 2244 3097 1772 3097 \nQ 1472 3097 1187 3025 \nQ 903 2953 641 2809 \nL 641 3341 \nQ 956 3463 1253 3523 \nQ 1550 3584 1831 3584 \nQ 2591 3584 2966 3190 \nQ 3341 2797 3341 1997 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \nL 1178 3500 \nL 1178 0 \nL 603 0 \nL 603 3500 \nz\nM 603 4863 \nL 1178 4863 \nL 1178 4134 \nL 603 4134 \nL 603 4863 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \nL 3513 0 \nL 2938 0 \nL 2938 2094 \nQ 2938 2591 2744 2837 \nQ 2550 3084 2163 3084 \nQ 1697 3084 1428 2787 \nQ 1159 2491 1159 1978 \nL 1159 0 \nL 581 0 \nL 581 3500 \nL 1159 3500 \nL 1159 2956 \nQ 1366 3272 1645 3428 \nQ 1925 3584 2291 3584 \nQ 2894 3584 3203 3211 \nQ 3513 2838 3513 2113 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \nL 1178 4863 \nL 1178 0 \nL 603 0 \nL 603 4863 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \nL 2834 2853 \nQ 2591 2978 2328 3040 \nQ 2066 3103 1784 3103 \nQ 1356 3103 1142 2972 \nQ 928 2841 928 2578 \nQ 928 2378 1081 2264 \nQ 1234 2150 1697 2047 \nL 1894 2003 \nQ 2506 1872 2764 1633 \nQ 3022 1394 3022 966 \nQ 3022 478 2636 193 \nQ 2250 -91 1575 -91 \nQ 1294 -91 989 -36 \nQ 684 19 347 128 \nL 347 722 \nQ 666 556 975 473 \nQ 1284 391 1588 391 \nQ 1994 391 2212 530 \nQ 2431 669 2431 922 \nQ 2431 1156 2273 1281 \nQ 2116 1406 1581 1522 \nL 1381 1569 \nQ 847 1681 609 1914 \nQ 372 2147 372 2553 \nQ 372 3047 722 3315 \nQ 1072 3584 1716 3584 \nQ 2034 3584 2315 3537 \nQ 2597 3491 2834 3397 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-74\"/>\n      <use xlink:href=\"#DejaVuSans-72\" transform=\"translate(39.208984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(80.322266 0)\"/>\n      <use xlink:href=\"#DejaVuSans-69\" transform=\"translate(141.601562 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6e\" transform=\"translate(169.384766 0)\"/>\n      <use xlink:href=\"#DejaVuSans-20\" transform=\"translate(232.763672 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" transform=\"translate(264.550781 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" transform=\"translate(292.333984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-73\" transform=\"translate(353.515625 0)\"/>\n      <use xlink:href=\"#DejaVuSans-73\" transform=\"translate(405.615234 0)\"/>\n     </g>\n    </g>\n    <g id=\"line2d_21\">\n     <path d=\"M 142.634375 34.976562 \nL 152.634375 34.976562 \nL 162.634375 34.976562 \n\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n    </g>\n    <g id=\"text_11\">\n     <!-- train acc -->\n     <g transform=\"translate(170.634375 38.476562) scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-74\"/>\n      <use xlink:href=\"#DejaVuSans-72\" transform=\"translate(39.208984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(80.322266 0)\"/>\n      <use xlink:href=\"#DejaVuSans-69\" transform=\"translate(141.601562 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6e\" transform=\"translate(169.384766 0)\"/>\n      <use xlink:href=\"#DejaVuSans-20\" transform=\"translate(232.763672 0)\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(264.550781 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(325.830078 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(380.810547 0)\"/>\n     </g>\n    </g>\n   </g>\n  </g>\n </g>\n <defs>\n  <clipPath id=\"pb0c00fea5f\">\n   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n  </clipPath>\n </defs>\n</svg>\n"
     },
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-17 fw=all -->\n",
    "\n",
    "The above code\n",
    "will generate a `submission.csv` file,\n",
    "whose format\n",
    "meets the requirement of the Kaggle competition.\n",
    "The method\n",
    "for submitting results to Kaggle\n",
    "is similar to that in that section.\n",
    "\n",
    "## Summary\n",
    "\n",
    "* We can read datasets containing raw image files after organizing them into the required format."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-18 fw=jax -->\n",
    "\n",
    "* We can use convolutional neural networks and image augmentation in an image classification competition."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-19 fw=all -->\n",
    "\n",
    "## Exercises\n",
    "\n",
    "1. Use the complete CIFAR-10 dataset for this Kaggle competition. Set hyperparameters as `batch_size = 128`, `num_epochs = 100`, `lr = 0.1`, `lr_period = 50`, and `lr_decay = 0.1`.  See what accuracy and ranking you can achieve in this competition. Can you further improve them?\n",
    "1. What accuracy can you get when not using image augmentation?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=kaggle-cifar10-md-20 fw=jax -->\n",
    "\n",
    "[Discussions](https://d2l.discourse.group/t/1479)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "name": "d2l-jax",
   "display_name": "d2l (JAX)",
   "language": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
