{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "---\n",
    "execute:\n",
    "  enabled: false\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=vision-transformer-md-1 fw=all -->\n",
    "\n",
    "# Vision Transformer\n",
    "\n",
    "Everything this chapter has built so far consumed text. Yet nothing in the\n",
    "transformer block *asks* what its tokens are: attention mixes vectors, the\n",
    "feed-forward network transforms them, and neither cares whether those vectors\n",
    "started life as subwords. This section makes the point concrete by feeding the\n",
    "block pieces of an image.\n",
    "\n",
    "For years the presumed answer to \"transformers for vision?\" was that images\n",
    "are different. CNNs (that section) owned computer vision, and\n",
    "their structural commitments, locality and translation equivariance\n",
    "(that section), looked like exactly what images demand. Early\n",
    "attempts kept those commitments: @ramachandran2019stand replaced\n",
    "convolutions with local self-attention, whose specialized attention patterns\n",
    "were hard to run fast on accelerators, and @cordonnier2020relationship\n",
    "proved that self-attention *can* learn to behave like convolution,\n",
    "demonstrating it with $2 \\times 2$ patches too small for realistic images.\n",
    "The *vision transformer* (ViT) of @Dosovitskiy.Beyer.Kolesnikov.ea.2021\n",
    "dropped the caution: cut the image into $16 \\times 16$ patches, embed each\n",
    "patch as a token, and run an unmodified transformer encoder over the\n",
    "resulting sequence. Trained on enough images (300 million of them), it beat\n",
    "the best CNNs of the day.\n",
    "\n",
    "We build that model here and train it on Fashion-MNIST, where the headline\n",
    "result is that it *loses* to a CNN of the same parameter count. The loss is\n",
    "the lesson: a convolutional network gets locality and translation\n",
    "equivariance by construction, while a transformer must learn them from data,\n",
    "and 60,000 images are not enough to close the gap that 300 million close\n",
    "easily. Along the way we open the trained model and check whether it has\n",
    "started learning what it was never told — that its 36 tokens came from a\n",
    "$6 \\times 6$ grid."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "vision-transformer"
   },
   "source": [
    "%matplotlib inline\n",
    "from d2l import torch as d2l\n",
    "import torch\n",
    "from torch import nn"
   ],
   "execution_count": 1,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=vision-transformer-md-2 fw=all -->\n",
    "\n",
    "## Patches as Tokens\n",
    "\n",
    "the figure depicts the architecture. It consists of a stem that\n",
    "patchifies images, a body of transformer blocks, and a head that turns one\n",
    "token's final representation into a class label.\n",
    "\n",
    "![The vision transformer architecture. In this example, an image is split into nine patches. A special “&lt;cls&gt;” token and the nine flattened image patches are transformed via patch embedding and $\\mathit{n}$ transformer blocks into ten representations, respectively. The “&lt;cls&gt;” representation is further transformed into the output label.](../img/vit.svg)\n",
    "\n",
    "Consider an input image with height $h$, width $w$, and $c$ channels. With\n",
    "patch height and width both set to $p$, the image is split into a sequence of\n",
    "$m = hw/p^2$ patches, each flattened to a vector of length $cp^2$. A special\n",
    "“&lt;cls&gt;” (class) token and the $m$ flattened patches are linearly\n",
    "projected into $m + 1$ vectors and summed with learnable positional\n",
    "embeddings. From here on the model is the encoder wiring of\n",
    "that section: a stack of transformer blocks with no causal mask,\n",
    "every token attending to every other, mapping $m + 1$ input vectors to\n",
    "$m + 1$ output representations of the same dimension. Since the “&lt;cls&gt;”\n",
    "token attends to all the image patches, its representation at the output of\n",
    "the stack summarizes the whole image and is the one the classification head\n",
    "reads.\n",
    "\n",
    "Splitting an image into patches and linearly projecting the flattened patches\n",
    "sounds like two operations, but they combine into one: a convolution whose\n",
    "kernel size and stride both equal the patch size. Each output position of\n",
    "such a convolution sees exactly one patch and computes exactly its linear\n",
    "projection."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "vision-transformer-patches-as-tokens-1"
   },
   "source": [
    "class PatchEmbedding(nn.Module):\n",
    "    \"\"\"Image-to-sequence stem of the vision transformer.\"\"\"\n",
    "    def __init__(self, img_size=96, patch_size=16, num_hiddens=512):\n",
    "        super().__init__()\n",
    "        def _make_tuple(x):\n",
    "            if not isinstance(x, (list, tuple)):\n",
    "                return (x, x)\n",
    "            return x\n",
    "        img_size, patch_size = _make_tuple(img_size), _make_tuple(patch_size)\n",
    "        # A partial trailing patch would silently shrink the token grid\n",
    "        assert img_size[0] % patch_size[0] == 0 and \\\n",
    "            img_size[1] % patch_size[1] == 0, \\\n",
    "            'image size must be divisible by the patch size'\n",
    "        self.num_patches = (img_size[0] // patch_size[0]) * (\n",
    "            img_size[1] // patch_size[1])\n",
    "        self.conv = nn.LazyConv2d(num_hiddens, kernel_size=patch_size,\n",
    "                                  stride=patch_size)\n",
    "\n",
    "    def forward(self, X):\n",
    "        # Output shape: (batch size, no. of patches, no. of channels)\n",
    "        return self.conv(X).flatten(2).transpose(1, 2)"
   ],
   "execution_count": 2,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=vision-transformer-md-3 fw=all -->\n",
    "\n",
    "In the following example, taking images with height and width of `img_size`\n",
    "as inputs, the patch embedding outputs `(img_size//patch_size)**2` patches\n",
    "that are linearly projected to vectors of length `num_hiddens`."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "vision-transformer-patches-as-tokens-2"
   },
   "source": [
    "img_size, patch_size, num_hiddens, batch_size = 96, 16, 512, 4\n",
    "patch_emb = PatchEmbedding(img_size, patch_size, num_hiddens)\n",
    "X = d2l.zeros(batch_size, 3, img_size, img_size)\n",
    "d2l.check_shape(patch_emb(X),\n",
    "                (batch_size, (img_size//patch_size)**2, num_hiddens))"
   ],
   "execution_count": 3,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=vision-transformer-md-4 fw=all -->\n",
    "\n",
    "This is the entire interface between vision and the transformer. Everything\n",
    "that follows treats the patch vectors exactly as the rest of this chapter\n",
    "treats token embeddings.\n",
    "\n",
    "## The Vision Transformer Block\n",
    "\n",
    "The body of the model is the transformer block of\n",
    "that section in its pre-norm configuration: normalization\n",
    "sits on each branch, right before multi-head attention and before the MLP,\n",
    "and the residual stream runs uninterrupted. ViT adopted this arrangement just\n",
    "as its training advantages were being established\n",
    "[@baevski2018adaptive; @wang2019learning; @xiong2020layer], and its success\n",
    "helped make pre-norm the default it is today.\n",
    "\n",
    "Two details are specific to the vision variant. The MLP uses the Gaussian\n",
    "error linear unit (GELU), a smooth relative of ReLU\n",
    "[@Hendrycks.Gimpel.2016], and applies dropout after each of its two\n",
    "linear layers — at ViT scale, unlike at LLM scale, regularization earns its\n",
    "keep. And because every patch is a valid token that may look at every other,\n",
    "the block needs no mask: the `valid_lens` argument of the\n",
    "`d2l.MultiHeadAttention` from that section stays\n",
    "`None` throughout this section."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "vision-transformer-the-vision-transformer-block-1"
   },
   "source": [
    "class ViTBlock(nn.Module):\n",
    "    \"\"\"Pre-norm transformer block with a GELU MLP.\"\"\"\n",
    "    def __init__(self, num_hiddens, mlp_num_hiddens, num_heads, dropout,\n",
    "                 use_bias=False):\n",
    "        super().__init__()\n",
    "        self.ln1 = nn.LayerNorm(num_hiddens)\n",
    "        self.attention = d2l.MultiHeadAttention(num_hiddens, num_heads,\n",
    "                                                dropout, use_bias)\n",
    "        self.ln2 = nn.LayerNorm(num_hiddens)\n",
    "        self.mlp = nn.Sequential(\n",
    "            nn.LazyLinear(mlp_num_hiddens), nn.GELU(), nn.Dropout(dropout),\n",
    "            nn.LazyLinear(num_hiddens), nn.Dropout(dropout))\n",
    "\n",
    "    def forward(self, X, valid_lens=None):\n",
    "        X = X + self.attention(*([self.ln1(X)] * 3), valid_lens)\n",
    "        return X + self.mlp(self.ln2(X))"
   ],
   "execution_count": 4,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=vision-transformer-md-5 fw=all -->\n",
    "\n",
    "As with every transformer block, the output shape equals the input shape, so\n",
    "blocks stack without further ceremony."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "vision-transformer-the-vision-transformer-block-2"
   },
   "source": [
    "X = d2l.ones((2, 100, 24))\n",
    "encoder_blk = ViTBlock(24, 48, 8, 0.5)\n",
    "encoder_blk.eval()\n",
    "d2l.check_shape(encoder_blk(X), X.shape)"
   ],
   "execution_count": 5,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=vision-transformer-md-6 fw=all -->\n",
    "\n",
    "## The Full Model\n",
    "\n",
    "Assembling the pieces takes one class. Input images pass through a\n",
    "`PatchEmbedding` instance; the “&lt;cls&gt;” token embedding, a learnable\n",
    "parameter initialized to zeros, is prepended to the resulting sequence. The\n",
    "sum with the positional embeddings goes through dropout, then through\n",
    "`num_blks` stacked `ViTBlock` instances, and finally the head projects the\n",
    "“&lt;cls&gt;” token's representation to the class logits.\n",
    "\n",
    "The positional embeddings deserve a pause. The sinusoidal and rotary\n",
    "constructions of that section encode positions along\n",
    "a line, but a patch has a row *and* a column, and it is not obvious what a\n",
    "hand-designed two-dimensional code should look like. ViT sidesteps the\n",
    "question: the position embeddings are a freely learnable parameter,\n",
    "initialized to small Gaussian noise as in the original recipe, one vector\n",
    "per sequence position. At\n",
    "initialization, the model therefore has no idea that patch 7 sits directly\n",
    "left of patch 8 and directly above patch 13 — as far as it is concerned the\n",
    "patches are an unordered set of vectors. Whatever spatial structure the\n",
    "trained model uses, it must discover during training. We will check how far\n",
    "it gets."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "vision-transformer-the-full-model"
   },
   "source": [
    "class ViT(d2l.Classifier):\n",
    "    \"\"\"Vision transformer.\"\"\"\n",
    "    def __init__(self, img_size, patch_size, num_hiddens, mlp_num_hiddens,\n",
    "                 num_heads, num_blks, emb_dropout, blk_dropout, lr=0.1,\n",
    "                 use_bias=False, num_classes=10):\n",
    "        super().__init__()\n",
    "        self.save_hyperparameters()\n",
    "        self.patch_embedding = PatchEmbedding(\n",
    "            img_size, patch_size, num_hiddens)\n",
    "        self.cls_token = nn.Parameter(d2l.zeros(1, 1, num_hiddens))\n",
    "        num_steps = self.patch_embedding.num_patches + 1  # Add the cls token\n",
    "        # Positional embeddings are learnable, initialized to small noise\n",
    "        self.pos_embedding = nn.Parameter(\n",
    "            torch.randn(1, num_steps, num_hiddens) * 0.02)\n",
    "        self.dropout = nn.Dropout(emb_dropout)\n",
    "        self.blks = nn.Sequential()\n",
    "        for i in range(num_blks):\n",
    "            self.blks.add_module(f\"{i}\", ViTBlock(\n",
    "                num_hiddens, mlp_num_hiddens, num_heads, blk_dropout,\n",
    "                use_bias))\n",
    "        self.head = nn.Sequential(nn.LayerNorm(num_hiddens),\n",
    "                                  nn.Linear(num_hiddens, num_classes))\n",
    "\n",
    "    def forward(self, X):\n",
    "        X = self.patch_embedding(X)\n",
    "        X = d2l.concat((self.cls_token.expand(X.shape[0], -1, -1), X), 1)\n",
    "        X = self.dropout(X + self.pos_embedding)\n",
    "        for blk in self.blks:\n",
    "            X = blk(X)\n",
    "        return self.head(X[:, 0])"
   ],
   "execution_count": 6,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=vision-transformer-md-7 fw=all -->\n",
    "\n",
    "Why read the class from the “&lt;cls&gt;” token rather than, say, averaging\n",
    "all the patch representations? Both work. The extra token is a learned\n",
    "querying device (through two rounds of attention it can *ask* for whatever\n",
    "evidence the classification needs), and it matches the original recipe; the\n",
    "exercises invite you to try the averaging variant.\n",
    "\n",
    "## Training and What the Model Learns\n",
    "\n",
    "### A First Run\n",
    "\n",
    "We train a small ViT on Fashion-MNIST, resized to $96 \\times 96$ so that a\n",
    "$16 \\times 16$ patch size yields $6 \\times 6 = 36$ patch tokens. The recipe\n",
    "is deliberately the one used for CNNs in that section: 10\n",
    "epochs, batch size 128, plain SGD."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "vision-transformer-a-first-run-1"
   },
   "source": [
    "img_size, patch_size = 96, 16\n",
    "num_hiddens, mlp_num_hiddens, num_heads, num_blks = 512, 2048, 8, 2\n",
    "emb_dropout, blk_dropout, lr = 0.1, 0.1, 0.1\n",
    "model = ViT(img_size, patch_size, num_hiddens, mlp_num_hiddens, num_heads,\n",
    "            num_blks, emb_dropout, blk_dropout, lr)\n",
    "trainer = d2l.Trainer(max_epochs=10, num_gpus=1)\n",
    "data = d2l.FashionMNIST(batch_size=128, resize=(img_size, img_size))\n",
    "trainer.fit(model, data)"
   ],
   "execution_count": 7,
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n <metadata>\n  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n   <cc:Work>\n    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n    <dc:date>2026-07-19T14:07:21.724660</dc:date>\n    <dc:format>image/svg+xml</dc:format>\n    <dc:creator>\n     <cc:Agent>\n      <dc:title>Matplotlib v3.10.8, https://matplotlib.org/</dc:title>\n     </cc:Agent>\n    </dc:creator>\n   </cc:Work>\n  </rdf:RDF>\n </metadata>\n <defs>\n  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n </defs>\n <g id=\"figure_1\">\n  <g id=\"patch_1\">\n   <path d=\"M 0 183.35625 \nL 238.965625 183.35625 \nL 238.965625 0 \nL 0 0 \nz\n\" style=\"fill: #ffffff\"/>\n  </g>\n  <g id=\"axes_1\">\n   <g id=\"patch_2\">\n    <path d=\"M 30.103125 145.8 \nL 225.403125 145.8 \nL 225.403125 7.2 \nL 30.103125 7.2 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g id=\"matplotlib.axis_1\">\n    <g id=\"xtick_1\">\n     <g id=\"line2d_1\">\n      <defs>\n       <path id=\"mec588afeff\" d=\"M 0 0 \nL 0 3.5 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#mec588afeff\" x=\"30.103125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_1\">\n      <!-- 0 -->\n      <g transform=\"translate(26.921875 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \nQ 1547 4250 1301 3770 \nQ 1056 3291 1056 2328 \nQ 1056 1369 1301 889 \nQ 1547 409 2034 409 \nQ 2525 409 2770 889 \nQ 3016 1369 3016 2328 \nQ 3016 3291 2770 3770 \nQ 2525 4250 2034 4250 \nz\nM 2034 4750 \nQ 2819 4750 3233 4129 \nQ 3647 3509 3647 2328 \nQ 3647 1150 3233 529 \nQ 2819 -91 2034 -91 \nQ 1250 -91 836 529 \nQ 422 1150 422 2328 \nQ 422 3509 836 4129 \nQ 1250 4750 2034 4750 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_2\">\n     <g id=\"line2d_2\">\n      <g>\n       <use xlink:href=\"#mec588afeff\" x=\"69.163125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_2\">\n      <!-- 2 -->\n      <g transform=\"translate(65.981875 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \nL 3431 531 \nL 3431 0 \nL 469 0 \nL 469 531 \nQ 828 903 1448 1529 \nQ 2069 2156 2228 2338 \nQ 2531 2678 2651 2914 \nQ 2772 3150 2772 3378 \nQ 2772 3750 2511 3984 \nQ 2250 4219 1831 4219 \nQ 1534 4219 1204 4116 \nQ 875 4013 500 3803 \nL 500 4441 \nQ 881 4594 1212 4672 \nQ 1544 4750 1819 4750 \nQ 2544 4750 2975 4387 \nQ 3406 4025 3406 3419 \nQ 3406 3131 3298 2873 \nQ 3191 2616 2906 2266 \nQ 2828 2175 2409 1742 \nQ 1991 1309 1228 531 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-32\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_3\">\n     <g id=\"line2d_3\">\n      <g>\n       <use xlink:href=\"#mec588afeff\" x=\"108.223125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_3\">\n      <!-- 4 -->\n      <g transform=\"translate(105.041875 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \nL 825 1625 \nL 2419 1625 \nL 2419 4116 \nz\nM 2253 4666 \nL 3047 4666 \nL 3047 1625 \nL 3713 1625 \nL 3713 1100 \nL 3047 1100 \nL 3047 0 \nL 2419 0 \nL 2419 1100 \nL 313 1100 \nL 313 1709 \nL 2253 4666 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-34\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_4\">\n     <g id=\"line2d_4\">\n      <g>\n       <use xlink:href=\"#mec588afeff\" x=\"147.283125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_4\">\n      <!-- 6 -->\n      <g transform=\"translate(144.101875 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \nQ 1688 2584 1439 2293 \nQ 1191 2003 1191 1497 \nQ 1191 994 1439 701 \nQ 1688 409 2113 409 \nQ 2538 409 2786 701 \nQ 3034 994 3034 1497 \nQ 3034 2003 2786 2293 \nQ 2538 2584 2113 2584 \nz\nM 3366 4563 \nL 3366 3988 \nQ 3128 4100 2886 4159 \nQ 2644 4219 2406 4219 \nQ 1781 4219 1451 3797 \nQ 1122 3375 1075 2522 \nQ 1259 2794 1537 2939 \nQ 1816 3084 2150 3084 \nQ 2853 3084 3261 2657 \nQ 3669 2231 3669 1497 \nQ 3669 778 3244 343 \nQ 2819 -91 2113 -91 \nQ 1303 -91 875 529 \nQ 447 1150 447 2328 \nQ 447 3434 972 4092 \nQ 1497 4750 2381 4750 \nQ 2619 4750 2861 4703 \nQ 3103 4656 3366 4563 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-36\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_5\">\n     <g id=\"line2d_5\">\n      <g>\n       <use xlink:href=\"#mec588afeff\" x=\"186.343125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_5\">\n      <!-- 8 -->\n      <g transform=\"translate(183.161875 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \nQ 1584 2216 1326 1975 \nQ 1069 1734 1069 1313 \nQ 1069 891 1326 650 \nQ 1584 409 2034 409 \nQ 2484 409 2743 651 \nQ 3003 894 3003 1313 \nQ 3003 1734 2745 1975 \nQ 2488 2216 2034 2216 \nz\nM 1403 2484 \nQ 997 2584 770 2862 \nQ 544 3141 544 3541 \nQ 544 4100 942 4425 \nQ 1341 4750 2034 4750 \nQ 2731 4750 3128 4425 \nQ 3525 4100 3525 3541 \nQ 3525 3141 3298 2862 \nQ 3072 2584 2669 2484 \nQ 3125 2378 3379 2068 \nQ 3634 1759 3634 1313 \nQ 3634 634 3220 271 \nQ 2806 -91 2034 -91 \nQ 1263 -91 848 271 \nQ 434 634 434 1313 \nQ 434 1759 690 2068 \nQ 947 2378 1403 2484 \nz\nM 1172 3481 \nQ 1172 3119 1398 2916 \nQ 1625 2713 2034 2713 \nQ 2441 2713 2670 2916 \nQ 2900 3119 2900 3481 \nQ 2900 3844 2670 4047 \nQ 2441 4250 2034 4250 \nQ 1625 4250 1398 4047 \nQ 1172 3844 1172 3481 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-38\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_6\">\n     <g id=\"line2d_6\">\n      <g>\n       <use xlink:href=\"#mec588afeff\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_6\">\n      <!-- 10 -->\n      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-31\" d=\"M 794 531 \nL 1825 531 \nL 1825 4091 \nL 703 3866 \nL 703 4441 \nL 1819 4666 \nL 2450 4666 \nL 2450 531 \nL 3481 531 \nL 3481 0 \nL 794 0 \nL 794 531 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(63.623047 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"text_7\">\n     <!-- epoch -->\n     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \nL 3597 1613 \nL 953 1613 \nQ 991 1019 1311 708 \nQ 1631 397 2203 397 \nQ 2534 397 2845 478 \nQ 3156 559 3463 722 \nL 3463 178 \nQ 3153 47 2828 -22 \nQ 2503 -91 2169 -91 \nQ 1331 -91 842 396 \nQ 353 884 353 1716 \nQ 353 2575 817 3079 \nQ 1281 3584 2069 3584 \nQ 2775 3584 3186 3129 \nQ 3597 2675 3597 1894 \nz\nM 3022 2063 \nQ 3016 2534 2758 2815 \nQ 2500 3097 2075 3097 \nQ 1594 3097 1305 2825 \nQ 1016 2553 972 2059 \nL 3022 2063 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \nL 1159 -1331 \nL 581 -1331 \nL 581 3500 \nL 1159 3500 \nL 1159 2969 \nQ 1341 3281 1617 3432 \nQ 1894 3584 2278 3584 \nQ 2916 3584 3314 3078 \nQ 3713 2572 3713 1747 \nQ 3713 922 3314 415 \nQ 2916 -91 2278 -91 \nQ 1894 -91 1617 61 \nQ 1341 213 1159 525 \nz\nM 3116 1747 \nQ 3116 2381 2855 2742 \nQ 2594 3103 2138 3103 \nQ 1681 3103 1420 2742 \nQ 1159 2381 1159 1747 \nQ 1159 1113 1420 752 \nQ 1681 391 2138 391 \nQ 2594 391 2855 752 \nQ 3116 1113 3116 1747 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \nQ 1497 3097 1228 2736 \nQ 959 2375 959 1747 \nQ 959 1119 1226 758 \nQ 1494 397 1959 397 \nQ 2419 397 2687 759 \nQ 2956 1122 2956 1747 \nQ 2956 2369 2687 2733 \nQ 2419 3097 1959 3097 \nz\nM 1959 3584 \nQ 2709 3584 3137 3096 \nQ 3566 2609 3566 1747 \nQ 3566 888 3137 398 \nQ 2709 -91 1959 -91 \nQ 1206 -91 779 398 \nQ 353 888 353 1747 \nQ 353 2609 779 3096 \nQ 1206 3584 1959 3584 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \nL 3122 2828 \nQ 2878 2963 2633 3030 \nQ 2388 3097 2138 3097 \nQ 1578 3097 1268 2742 \nQ 959 2388 959 1747 \nQ 959 1106 1268 751 \nQ 1578 397 2138 397 \nQ 2388 397 2633 464 \nQ 2878 531 3122 666 \nL 3122 134 \nQ 2881 22 2623 -34 \nQ 2366 -91 2075 -91 \nQ 1284 -91 818 406 \nQ 353 903 353 1747 \nQ 353 2603 823 3093 \nQ 1294 3584 2113 3584 \nQ 2378 3584 2631 3529 \nQ 2884 3475 3122 3366 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \nL 3513 0 \nL 2938 0 \nL 2938 2094 \nQ 2938 2591 2744 2837 \nQ 2550 3084 2163 3084 \nQ 1697 3084 1428 2787 \nQ 1159 2491 1159 1978 \nL 1159 0 \nL 581 0 \nL 581 4863 \nL 1159 4863 \nL 1159 2956 \nQ 1366 3272 1645 3428 \nQ 1925 3584 2291 3584 \nQ 2894 3584 3203 3211 \nQ 3513 2838 3513 2113 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-65\"/>\n      <use xlink:href=\"#DejaVuSans-70\" transform=\"translate(61.523438 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" transform=\"translate(125 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(186.181641 0)\"/>\n      <use xlink:href=\"#DejaVuSans-68\" transform=\"translate(241.162109 0)\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_2\">\n    <g id=\"ytick_1\">\n     <g id=\"line2d_7\">\n      <defs>\n       <path id=\"mdd4da1611b\" d=\"M 0 0 \nL -3.5 0 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#mdd4da1611b\" x=\"30.103125\" y=\"132.08803\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_8\">\n      <!-- 0.5 -->\n      <g transform=\"translate(7.2 135.887249) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \nL 1344 794 \nL 1344 0 \nL 684 0 \nL 684 794 \nz\n\" transform=\"scale(0.015625)\"/>\n        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \nL 3169 4666 \nL 3169 4134 \nL 1269 4134 \nL 1269 2991 \nQ 1406 3038 1543 3061 \nQ 1681 3084 1819 3084 \nQ 2600 3084 3056 2656 \nQ 3513 2228 3513 1497 \nQ 3513 744 3044 326 \nQ 2575 -91 1722 -91 \nQ 1428 -91 1123 -41 \nQ 819 9 494 109 \nL 494 744 \nQ 775 591 1075 516 \nQ 1375 441 1709 441 \nQ 2250 441 2565 725 \nQ 2881 1009 2881 1497 \nQ 2881 1984 2565 2268 \nQ 2250 2553 1709 2553 \nQ 1456 2553 1204 2497 \nQ 953 2441 691 2322 \nL 691 4666 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-30\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-35\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_2\">\n     <g id=\"line2d_8\">\n      <g>\n       <use xlink:href=\"#mdd4da1611b\" x=\"30.103125\" y=\"108.353407\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_9\">\n      <!-- 1.0 -->\n      <g transform=\"translate(7.2 112.152626) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_3\">\n     <g id=\"line2d_9\">\n      <g>\n       <use xlink:href=\"#mdd4da1611b\" x=\"30.103125\" y=\"84.618785\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_10\">\n      <!-- 1.5 -->\n      <g transform=\"translate(7.2 88.418003) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-35\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_4\">\n     <g id=\"line2d_10\">\n      <g>\n       <use xlink:href=\"#mdd4da1611b\" x=\"30.103125\" y=\"60.884162\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_11\">\n      <!-- 2.0 -->\n      <g transform=\"translate(7.2 64.683381) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-32\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_5\">\n     <g id=\"line2d_11\">\n      <g>\n       <use xlink:href=\"#mdd4da1611b\" x=\"30.103125\" y=\"37.149539\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_12\">\n      <!-- 2.5 -->\n      <g transform=\"translate(7.2 40.948758) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-32\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-35\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_6\">\n     <g id=\"line2d_12\">\n      <g>\n       <use xlink:href=\"#mdd4da1611b\" x=\"30.103125\" y=\"13.414917\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_13\">\n      <!-- 3.0 -->\n      <g transform=\"translate(7.2 17.214135) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \nQ 3050 2419 3304 2112 \nQ 3559 1806 3559 1356 \nQ 3559 666 3084 287 \nQ 2609 -91 1734 -91 \nQ 1441 -91 1130 -33 \nQ 819 25 488 141 \nL 488 750 \nQ 750 597 1062 519 \nQ 1375 441 1716 441 \nQ 2309 441 2620 675 \nQ 2931 909 2931 1356 \nQ 2931 1769 2642 2001 \nQ 2353 2234 1838 2234 \nL 1294 2234 \nL 1294 2753 \nL 1863 2753 \nQ 2328 2753 2575 2939 \nQ 2822 3125 2822 3475 \nQ 2822 3834 2567 4026 \nQ 2313 4219 1838 4219 \nQ 1578 4219 1281 4162 \nQ 984 4106 628 3988 \nL 628 4550 \nQ 988 4650 1302 4700 \nQ 1616 4750 1894 4750 \nQ 2613 4750 3031 4423 \nQ 3450 4097 3450 3541 \nQ 3450 3153 3228 2886 \nQ 3006 2619 2597 2516 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-33\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"line2d_13\">\n    <path d=\"M 34.954394 13.5 \nL 44.698573 109.120079 \nL 54.442752 121.156613 \nL 64.186931 126.651918 \nL 73.93111 129.430704 \nL 83.675289 131.612882 \nL 93.419468 132.514589 \nL 103.163647 134.12137 \nL 112.907826 135.161482 \nL 122.652006 134.878003 \nL 132.396185 136.213307 \nL 142.140364 136.542262 \nL 151.884543 137.183005 \nL 161.628722 137.461963 \nL 171.372901 138.015336 \nL 181.11708 138.167815 \nL 190.861259 138.684776 \nL 200.605438 138.839354 \nL 210.349618 139.234671 \nL 220.093797 139.5 \n\" clip-path=\"url(#pd8a582ef65)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n   </g>\n   <g id=\"line2d_14\">\n    <path d=\"M 49.633125 95.245731 \nL 69.163125 122.939546 \nL 88.693125 132.86603 \nL 108.223125 134.459767 \nL 127.753125 135.243546 \nL 147.283125 135.873437 \nL 166.813125 135.743397 \nL 186.343125 136.431312 \nL 205.873125 137.75006 \nL 225.403125 137.134902 \n\" clip-path=\"url(#pd8a582ef65)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #ff7f0e; stroke-width: 1.5\"/>\n   </g>\n   <g id=\"line2d_15\">\n    <path d=\"M 49.633125 132.623186 \nL 69.163125 119.882726 \nL 88.693125 116.709346 \nL 108.223125 116.075609 \nL 127.753125 115.92539 \nL 147.283125 115.859669 \nL 166.813125 115.573314 \nL 186.343125 115.291653 \nL 205.873125 115.038159 \nL 225.403125 115.141434 \n\" clip-path=\"url(#pd8a582ef65)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #2ca02c; stroke-width: 1.5\"/>\n   </g>\n   <g id=\"patch_3\">\n    <path d=\"M 30.103125 145.8 \nL 30.103125 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_4\">\n    <path d=\"M 225.403125 145.8 \nL 225.403125 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_5\">\n    <path d=\"M 30.103125 145.8 \nL 225.403125 145.8 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_6\">\n    <path d=\"M 30.103125 7.2 \nL 225.403125 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"legend_1\">\n    <g id=\"patch_7\">\n     <path d=\"M 138.8125 60.06875 \nL 218.403125 60.06875 \nQ 220.403125 60.06875 220.403125 58.06875 \nL 220.403125 14.2 \nQ 220.403125 12.2 218.403125 12.2 \nL 138.8125 12.2 \nQ 136.8125 12.2 136.8125 14.2 \nL 136.8125 58.06875 \nQ 136.8125 60.06875 138.8125 60.06875 \nz\n\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n    </g>\n    <g id=\"line2d_16\">\n     <path d=\"M 140.8125 20.298438 \nL 150.8125 20.298438 \nL 160.8125 20.298438 \n\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n    </g>\n    <g id=\"text_14\">\n     <!-- train_loss -->\n     <g transform=\"translate(168.8125 23.798438) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \nL 1172 3500 \nL 2356 3500 \nL 2356 3053 \nL 1172 3053 \nL 1172 1153 \nQ 1172 725 1289 603 \nQ 1406 481 1766 481 \nL 2356 481 \nL 2356 0 \nL 1766 0 \nQ 1100 0 847 248 \nQ 594 497 594 1153 \nL 594 3053 \nL 172 3053 \nL 172 3500 \nL 594 3500 \nL 594 4494 \nL 1172 4494 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \nQ 2534 3019 2420 3045 \nQ 2306 3072 2169 3072 \nQ 1681 3072 1420 2755 \nQ 1159 2438 1159 1844 \nL 1159 0 \nL 581 0 \nL 581 3500 \nL 1159 3500 \nL 1159 2956 \nQ 1341 3275 1631 3429 \nQ 1922 3584 2338 3584 \nQ 2397 3584 2469 3576 \nQ 2541 3569 2628 3553 \nL 2631 2963 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \nQ 1497 1759 1228 1600 \nQ 959 1441 959 1056 \nQ 959 750 1161 570 \nQ 1363 391 1709 391 \nQ 2188 391 2477 730 \nQ 2766 1069 2766 1631 \nL 2766 1759 \nL 2194 1759 \nz\nM 3341 1997 \nL 3341 0 \nL 2766 0 \nL 2766 531 \nQ 2569 213 2275 61 \nQ 1981 -91 1556 -91 \nQ 1019 -91 701 211 \nQ 384 513 384 1019 \nQ 384 1609 779 1909 \nQ 1175 2209 1959 2209 \nL 2766 2209 \nL 2766 2266 \nQ 2766 2663 2505 2880 \nQ 2244 3097 1772 3097 \nQ 1472 3097 1187 3025 \nQ 903 2953 641 2809 \nL 641 3341 \nQ 956 3463 1253 3523 \nQ 1550 3584 1831 3584 \nQ 2591 3584 2966 3190 \nQ 3341 2797 3341 1997 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \nL 1178 3500 \nL 1178 0 \nL 603 0 \nL 603 3500 \nz\nM 603 4863 \nL 1178 4863 \nL 1178 4134 \nL 603 4134 \nL 603 4863 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \nL 3513 0 \nL 2938 0 \nL 2938 2094 \nQ 2938 2591 2744 2837 \nQ 2550 3084 2163 3084 \nQ 1697 3084 1428 2787 \nQ 1159 2491 1159 1978 \nL 1159 0 \nL 581 0 \nL 581 3500 \nL 1159 3500 \nL 1159 2956 \nQ 1366 3272 1645 3428 \nQ 1925 3584 2291 3584 \nQ 2894 3584 3203 3211 \nQ 3513 2838 3513 2113 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-5f\" d=\"M 3263 -1063 \nL 3263 -1509 \nL -63 -1509 \nL -63 -1063 \nL 3263 -1063 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \nL 1178 4863 \nL 1178 0 \nL 603 0 \nL 603 4863 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \nL 2834 2853 \nQ 2591 2978 2328 3040 \nQ 2066 3103 1784 3103 \nQ 1356 3103 1142 2972 \nQ 928 2841 928 2578 \nQ 928 2378 1081 2264 \nQ 1234 2150 1697 2047 \nL 1894 2003 \nQ 2506 1872 2764 1633 \nQ 3022 1394 3022 966 \nQ 3022 478 2636 193 \nQ 2250 -91 1575 -91 \nQ 1294 -91 989 -36 \nQ 684 19 347 128 \nL 347 722 \nQ 666 556 975 473 \nQ 1284 391 1588 391 \nQ 1994 391 2212 530 \nQ 2431 669 2431 922 \nQ 2431 1156 2273 1281 \nQ 2116 1406 1581 1522 \nL 1381 1569 \nQ 847 1681 609 1914 \nQ 372 2147 372 2553 \nQ 372 3047 722 3315 \nQ 1072 3584 1716 3584 \nQ 2034 3584 2315 3537 \nQ 2597 3491 2834 3397 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-74\"/>\n      <use xlink:href=\"#DejaVuSans-72\" transform=\"translate(39.208984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(80.322266 0)\"/>\n      <use xlink:href=\"#DejaVuSans-69\" transform=\"translate(141.601562 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6e\" transform=\"translate(169.384766 0)\"/>\n      <use xlink:href=\"#DejaVuSans-5f\" transform=\"translate(232.763672 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" transform=\"translate(282.763672 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" transform=\"translate(310.546875 0)\"/>\n      <use xlink:href=\"#DejaVuSans-73\" transform=\"translate(371.728516 0)\"/>\n      <use xlink:href=\"#DejaVuSans-73\" transform=\"translate(423.828125 0)\"/>\n     </g>\n    </g>\n    <g id=\"line2d_17\">\n     <path d=\"M 140.8125 35.254688 \nL 150.8125 35.254688 \nL 160.8125 35.254688 \n\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #ff7f0e; stroke-width: 1.5\"/>\n    </g>\n    <g id=\"text_15\">\n     <!-- val_loss -->\n     <g transform=\"translate(168.8125 38.754688) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-76\" d=\"M 191 3500 \nL 800 3500 \nL 1894 563 \nL 2988 3500 \nL 3597 3500 \nL 2284 0 \nL 1503 0 \nL 191 3500 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-76\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(59.179688 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" transform=\"translate(120.458984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-5f\" transform=\"translate(148.242188 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" transform=\"translate(198.242188 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" transform=\"translate(226.025391 0)\"/>\n      <use xlink:href=\"#DejaVuSans-73\" transform=\"translate(287.207031 0)\"/>\n      <use xlink:href=\"#DejaVuSans-73\" transform=\"translate(339.306641 0)\"/>\n     </g>\n    </g>\n    <g id=\"line2d_18\">\n     <path d=\"M 140.8125 50.210938 \nL 150.8125 50.210938 \nL 160.8125 50.210938 \n\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #2ca02c; stroke-width: 1.5\"/>\n    </g>\n    <g id=\"text_16\">\n     <!-- val_acc -->\n     <g transform=\"translate(168.8125 53.710938) scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-76\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(59.179688 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" transform=\"translate(120.458984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-5f\" transform=\"translate(148.242188 0)\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(198.242188 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(259.521484 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(314.501953 0)\"/>\n     </g>\n    </g>\n   </g>\n  </g>\n </g>\n <defs>\n  <clipPath id=\"pd8a582ef65\">\n   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n  </clipPath>\n </defs>\n</svg>\n"
     },
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "vision-transformer-a-first-run-2"
   },
   "source": [
    "vit_params = sum(p.numel() for p in model.parameters()\n",
    "                 if p.requires_grad)\n",
    "vit_acc = float(model.board.data['val_acc'][-1].y)\n",
    "print(f'ViT: {vit_params/1e6:.1f}M parameters, '\n",
    "      f'validation accuracy {vit_acc:.2f}')"
   ],
   "execution_count": 8,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "ViT: 6.5M parameters, validation accuracy 0.86\n"
     ]
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=vision-transformer-md-8 fw=all -->\n",
    "\n",
    "The model trains without drama to a validation accuracy of about 86–87%.\n",
    "Keep that number in mind; we will shortly see what a CNN of the same size\n",
    "does with the same ten epochs.\n",
    "\n",
    "### Do the Position Embeddings Discover the Grid?\n",
    "\n",
    "First, though, let us look inside. The position embeddings started as noise,\n",
    "and nothing in the training signal ever named a row or a column. In fully\n",
    "trained ViTs the embeddings famously end up reflecting the grid anyway: in\n",
    "the original paper's analysis, each position's embedding is most similar to\n",
    "the embeddings of its spatial neighbors, with clear row and column bands —\n",
    "two-dimensional structure induced by the data alone\n",
    "[@Dosovitskiy.Beyer.Kolesnikov.ea.2021]. Does our small model already\n",
    "show this? For each of the 36 patch positions we take its embedding vector,\n",
    "compute its cosine similarity with the embeddings of all 36 positions, and\n",
    "reshape those similarities into the $6 \\times 6$ patch grid. Laying out the\n",
    "36 resulting maps in the grid itself gives a picture in which position\n",
    "$(i, j)$ of the big grid shows how similar position $(i, j)$'s embedding is\n",
    "to everywhere else's."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "vision-transformer-do-the-position-embeddings-discover-the-grid"
   },
   "source": [
    "P = model.pos_embedding[0, 1:].detach().cpu()  # (36, 512): drop the cls slot\n",
    "P = P / P.norm(dim=1, keepdim=True)\n",
    "sim = P @ P.T                                  # Pairwise cosine similarities\n",
    "coords = torch.stack(torch.meshgrid(\n",
    "    torch.arange(6), torch.arange(6), indexing='ij'), -1).reshape(36, 2)\n",
    "dist = (coords[:, None] - coords[None]).abs().amax(-1)  # Chebyshev distance\n",
    "print(f'mean cosine similarity: grid neighbors {sim[dist == 1].mean():.2f}, '\n",
    "      f'distant positions {sim[dist >= 3].mean():.2f}')\n",
    "d2l.show_heatmaps(sim.reshape(6, 6, 6, 6), xlabel='', ylabel='',\n",
    "                  figsize=(5, 5), cmap='Reds')"
   ],
   "execution_count": 9,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "mean cosine similarity: grid neighbors 0.05, distant positions -0.03\n"
     ]
    },
    {
     "output_type": "display_data",
     "data": {
      "image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"304.511312pt\" height=\"301.705915pt\" viewBox=\"0 0 304.511312 301.705915\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n <metadata>\n  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n   <cc:Work>\n    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n    <dc:date>2026-07-19T14:07:23.381300</dc:date>\n    <dc:format>image/svg+xml</dc:format>\n    <dc:creator>\n     <cc:Agent>\n      <dc:title>Matplotlib v3.10.8, https://matplotlib.org/</dc:title>\n     </cc:Agent>\n    </dc:creator>\n   </cc:Work>\n  </rdf:RDF>\n </metadata>\n <defs>\n  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n </defs>\n <g id=\"figure_1\">\n  <g id=\"patch_1\">\n   <path d=\"M -0 301.705915 \nL 304.511312 301.705915 \nL 304.511312 0 \nL -0 0 \nz\n\" style=\"fill: #ffffff\"/>\n  </g>\n  <g id=\"axes_1\">\n   <g id=\"patch_2\">\n    <path d=\"M 20.5625 40.22779 \nL 52.448214 40.22779 \nL 52.448214 8.342076 \nL 20.5625 8.342076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p575ba2994c)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABGElEQVR4nO2WsUrDUBSGb9KLYGsCWdzEueKmoH2Ejh269AU6+QKOvoRP4lgKEocONeAoKliIllJCaASFklyf4P/HQy6cb/34k48z3WB/O3EGkSRQGWOMqWvsypJOg+EIOvc0o9uQ2pai0VJotBQaLYU1cQxlZzyl4yZ9wDKK6DY87ePvfn3yLbUtRaOl0GgpvIy21B7xp6l7f8WyKOi2uVxhaXmWl5fWaCk0WgqNlsL+Pi6hjG748zK4uIbOpXO+PT7B25cF3Xp5aY2WQqOl8DLaVt87KHvrDzoOzq6gC88HdNvkb9C57YZuvby0Rkuh0VJotBT2LsuhvLcHdNw84+enO+zyP/9UUP2lGZ16eWmNlkKjpfAy+h+5hTY5+0pYtgAAAABJRU5ErkJggg==\" id=\"image8be99aa47d\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"20.5625\" y=\"-7.82779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_1\">\n    <g id=\"xtick_1\">\n     <g id=\"line2d_1\">\n      <defs>\n       <path id=\"mb9a5b3b75b\" d=\"M 0 0 \nL 0 3.5 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"23.219643\" y=\"40.22779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_2\">\n     <g id=\"line2d_2\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"49.791071\" y=\"40.22779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_2\">\n    <g id=\"ytick_1\">\n     <g id=\"line2d_3\">\n      <defs>\n       <path id=\"m4c6e37c3be\" d=\"M 0 0 \nL -3.5 0 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"20.5625\" y=\"10.999219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_1\">\n      <!-- 0 -->\n      <g transform=\"translate(7.2 14.798438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \nQ 1547 4250 1301 3770 \nQ 1056 3291 1056 2328 \nQ 1056 1369 1301 889 \nQ 1547 409 2034 409 \nQ 2525 409 2770 889 \nQ 3016 1369 3016 2328 \nQ 3016 3291 2770 3770 \nQ 2525 4250 2034 4250 \nz\nM 2034 4750 \nQ 2819 4750 3233 4129 \nQ 3647 3509 3647 2328 \nQ 3647 1150 3233 529 \nQ 2819 -91 2034 -91 \nQ 1250 -91 836 529 \nQ 422 1150 422 2328 \nQ 422 3509 836 4129 \nQ 1250 4750 2034 4750 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_2\">\n     <g id=\"line2d_4\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"20.5625\" y=\"37.570647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_2\">\n      <!-- 5 -->\n      <g transform=\"translate(7.2 41.369866) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \nL 3169 4666 \nL 3169 4134 \nL 1269 4134 \nL 1269 2991 \nQ 1406 3038 1543 3061 \nQ 1681 3084 1819 3084 \nQ 2600 3084 3056 2656 \nQ 3513 2228 3513 1497 \nQ 3513 744 3044 326 \nQ 2575 -91 1722 -91 \nQ 1428 -91 1123 -41 \nQ 819 9 494 109 \nL 494 744 \nQ 775 591 1075 516 \nQ 1375 441 1709 441 \nQ 2250 441 2565 725 \nQ 2881 1009 2881 1497 \nQ 2881 1984 2565 2268 \nQ 2250 2553 1709 2553 \nQ 1456 2553 1204 2497 \nQ 953 2441 691 2322 \nL 691 4666 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_3\">\n    <path d=\"M 20.5625 40.22779 \nL 20.5625 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_4\">\n    <path d=\"M 52.448214 40.22779 \nL 52.448214 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_5\">\n    <path d=\"M 20.5625 40.22779 \nL 52.448214 40.22779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_6\">\n    <path d=\"M 20.5625 8.342076 \nL 52.448214 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_2\">\n   <g id=\"patch_7\">\n    <path d=\"M 58.825357 40.22779 \nL 90.711071 40.22779 \nL 90.711071 8.342076 \nL 58.825357 8.342076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p957ad9238d)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABEklEQVR4nO2ZvWoCQRRG76yjpRZaSMqAvQhWQor0VnkKn8CnsEztK6Szs0gRQRBFDAspRLEJi4IL/kLi9Qm+W1hcduCe9uMshymmmHX/8YgJ4RyciIh4ucBjuSq69DOHU/T6JqqR/OVsYtFaWLQWFq2FJ+/hyKtYtk8HvP1tHkwi4nQr7kGetEVrYdFaBBmN7zsi4sGHKOe6Pewma9Hl3S/e7MrLCBathUVrEWS0j55qcOR6Q7YvRzi5YkVUeTzE7ktbdIM8aYvWwqK1CDLa83EPR46/RfmWL+CxIr+a8mwKN2evphnBorWwaC2CjHa3NIF/tzqlZ1F+/+zj8XoWXZ584ahmS3SDPGmL1sKitQgy+g7IxTp+xjrCkgAAAABJRU5ErkJggg==\" id=\"image2abd1e8e4e\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"58.825357\" y=\"-7.82779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_3\">\n    <g id=\"xtick_3\">\n     <g id=\"line2d_5\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"61.4825\" y=\"40.22779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_4\">\n     <g id=\"line2d_6\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"88.053929\" y=\"40.22779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_4\">\n    <g id=\"ytick_3\">\n     <g id=\"line2d_7\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"58.825357\" y=\"10.999219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_4\">\n     <g id=\"line2d_8\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"58.825357\" y=\"37.570647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_8\">\n    <path d=\"M 58.825357 40.22779 \nL 58.825357 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_9\">\n    <path d=\"M 90.711071 40.22779 \nL 90.711071 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_10\">\n    <path d=\"M 58.825357 40.22779 \nL 90.711071 40.22779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_11\">\n    <path d=\"M 58.825357 8.342076 \nL 90.711071 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_3\">\n   <g id=\"patch_12\">\n    <path d=\"M 97.088214 40.22779 \nL 128.973929 40.22779 \nL 128.973929 8.342076 \nL 97.088214 8.342076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p13684a52de)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABFElEQVR4nO2YsUoDQRRFZ9YhbmRFtrIINokgCClkwWrBToJtIPmI/Iv5BfELrLUwfbBNrY2dsMRNxMTJWKS+t3zswDvt4TKHKZZl7N/zQzAA2+0jtff5KXZpRrf+7QXLn5puE2obikZLodFSaLQUzhzn2P6u6Th8LLDsXfGT/Ra77IROo7xpjZZCo6WIMtqZVgplmM/o2JZ3WCYH/OTqC7uzczqN8qY1WgqNlkKjpXBmtcT2sqDj8PoEnb0d85PbR9htN3Qa5U1rtBQaLUWU0c58vmP7TT6HxtDPln+c0qm9GUCXdPTXtBlotBQaLUWU0c5e4NfNsPN0PCmG0N2P+Kvp4XWJz60ruo3ypjVaCo2WIsrof3EFK+VYYe29AAAAAElFTkSuQmCC\" id=\"image19f8ea2e05\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"97.088214\" y=\"-7.82779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_5\">\n    <g id=\"xtick_5\">\n     <g id=\"line2d_9\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"99.745357\" y=\"40.22779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_6\">\n     <g id=\"line2d_10\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"126.316786\" y=\"40.22779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_6\">\n    <g id=\"ytick_5\">\n     <g id=\"line2d_11\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"97.088214\" y=\"10.999219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_6\">\n     <g id=\"line2d_12\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"97.088214\" y=\"37.570647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_13\">\n    <path d=\"M 97.088214 40.22779 \nL 97.088214 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_14\">\n    <path d=\"M 128.973929 40.22779 \nL 128.973929 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_15\">\n    <path d=\"M 97.088214 40.22779 \nL 128.973929 40.22779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_16\">\n    <path d=\"M 97.088214 8.342076 \nL 128.973929 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_4\">\n   <g id=\"patch_17\">\n    <path d=\"M 135.351071 40.22779 \nL 167.236786 40.22779 \nL 167.236786 8.342076 \nL 135.351071 8.342076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p1abe17972a)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABGklEQVR4nO2YsUqCURiGv3M6hqhQk00hCOESDQ4G3YFT3oNr4NDU0B24Ojd1A27ODk0u4eDgojQoYhAhv5Z5uoL3HT888D3rw+v3cJYfdPvBSxREvgCViIi7qGC53dBt/JhimfGtp/ZIsWgtLFoLi9YiSDwQm6PjuJxB587L/PL+F7tCiU6TfGmL1sKitUgyOkjxDFt/Qsf+5g46d5qn27/3Nywvq/wutUeKRWth0VpYtBZBvtZQulqdjg+vXej8fZtfJn8T+Ct+N8mXtmgtLFqLJKODfK6gjPMJHccsg+7n+YFuc50n8rvfdJvkS1u0FhathUVrEdx1A8q4XtDxbjiC7rE/ptteC38D/G2TbpN8aYvWwqK1SDL6H3PWM5hf0pfwAAAAAElFTkSuQmCC\" id=\"image1c8ca0ca75\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"135.351071\" y=\"-7.82779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_7\">\n    <g id=\"xtick_7\">\n     <g id=\"line2d_13\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"138.008214\" y=\"40.22779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_8\">\n     <g id=\"line2d_14\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"164.579643\" y=\"40.22779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_8\">\n    <g id=\"ytick_7\">\n     <g id=\"line2d_15\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"135.351071\" y=\"10.999219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_8\">\n     <g id=\"line2d_16\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"135.351071\" y=\"37.570647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_18\">\n    <path d=\"M 135.351071 40.22779 \nL 135.351071 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_19\">\n    <path d=\"M 167.236786 40.22779 \nL 167.236786 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_20\">\n    <path d=\"M 135.351071 40.22779 \nL 167.236786 40.22779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_21\">\n    <path d=\"M 135.351071 8.342076 \nL 167.236786 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_5\">\n   <g id=\"patch_22\">\n    <path d=\"M 173.613929 40.22779 \nL 205.499643 40.22779 \nL 205.499643 8.342076 \nL 173.613929 8.342076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p99f9d13d26)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABF0lEQVR4nO2XoU7DUBSGz23vKKuYYK4hmZmrnmGpIMGieAH8XgaFxyNIJrdkvACCTOAnFiABlgXRbLSXJ/h/edKbnM9++Xu/XNG0rlk/B0H0TqESEXHDArt8QLft5g3LrE+3CbUdxaK1sGgtLFoL787wuzbsv+i4fVlAl1Y3/ORDDZUrxnQa5U1btBYWrUWU0V5SD2X4eafjpLzA2/qXn9w7wdvvLT+XP7mbWLQWFq2FRWvh5Yg/EZNyyte7D+yaPzoNqycsJ5d0G+VNW7QWFq1FlNE+7D6xJX/MIiLtwx2WeU637nyEJXkNi0R60xathUVrYdFaePEZlGH5SMfu6ppIR7ez6ha6+9c53UZ50xathUVrEWX0P08nMOZUSHvbAAAAAElFTkSuQmCC\" id=\"imagee11289f7b1\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"173.613929\" y=\"-7.82779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_9\">\n    <g id=\"xtick_9\">\n     <g id=\"line2d_17\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"176.271071\" y=\"40.22779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_10\">\n     <g id=\"line2d_18\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"202.8425\" y=\"40.22779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_10\">\n    <g id=\"ytick_9\">\n     <g id=\"line2d_19\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"173.613929\" y=\"10.999219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_10\">\n     <g id=\"line2d_20\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"173.613929\" y=\"37.570647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_23\">\n    <path d=\"M 173.613929 40.22779 \nL 173.613929 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_24\">\n    <path d=\"M 205.499643 40.22779 \nL 205.499643 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_25\">\n    <path d=\"M 173.613929 40.22779 \nL 205.499643 40.22779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_26\">\n    <path d=\"M 173.613929 8.342076 \nL 205.499643 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_6\">\n   <g id=\"patch_27\">\n    <path d=\"M 211.876786 40.22779 \nL 243.7625 40.22779 \nL 243.7625 8.342076 \nL 211.876786 8.342076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#pbbf4777ddc)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABFElEQVR4nO2WMWpCQRRFZ76D8IMWFkERgwgRtLQS02UJSesCUmRHbiArSGlrK0gCweajoCIBCSI/EfwzruDe8pGBd9rDZQ6vGnt5nwSDqN9BZYwxZrnA7u+XTu3jE3Qh+6DbhNp/ikZLodFSaLQUzra6UCb3Azoujgcs8xPd2loDu7RKt1FeWqOl0Ggpoox2IT9C6VeffH34xm675tuHArtzTqdRXlqjpdBoKTRaCmfmMyjt8wsdh04fy/SGv1xOofK7jE6jvLRGS6HRUkQZ7cJ2A6Vff9Fx0mhDZ3tDui2mb1h68m01kV5ao6XQaCk0Wgrnd3soS5czHYfTD5aVGn/5tgnV62hMp1FeWqOl0Ggpooy+AlP/Nq7AW8zFAAAAAElFTkSuQmCC\" id=\"imagee0f3023394\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"211.876786\" y=\"-7.82779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_11\">\n    <g id=\"xtick_11\">\n     <g id=\"line2d_21\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"214.533929\" y=\"40.22779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_12\">\n     <g id=\"line2d_22\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"241.105357\" y=\"40.22779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_12\">\n    <g id=\"ytick_11\">\n     <g id=\"line2d_23\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"211.876786\" y=\"10.999219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_12\">\n     <g id=\"line2d_24\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"211.876786\" y=\"37.570647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_28\">\n    <path d=\"M 211.876786 40.22779 \nL 211.876786 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_29\">\n    <path d=\"M 243.7625 40.22779 \nL 243.7625 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_30\">\n    <path d=\"M 211.876786 40.22779 \nL 243.7625 40.22779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_31\">\n    <path d=\"M 211.876786 8.342076 \nL 243.7625 8.342076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_7\">\n   <g id=\"patch_32\">\n    <path d=\"M 20.5625 87.74779 \nL 52.448214 87.74779 \nL 52.448214 55.862076 \nL 20.5625 55.862076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#pdd02c00f72)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABEUlEQVR4nO2Zu2oCURRFzx0GEzOKCEES4iA+mqCdkP/yD1KnS2ORz/ADbEWwiKQQBNHGByoitkavX7B3efDCWe1myeIUtxjd+bvjBeDyBTSJiIg/HeHmSq/UlcwD3g47qkb8l+8Ti9bCorWwaC1i+ha3PqjsxgM8pnXuFktw8/sVdYO8tEVrYdFaBBkdS5KHY1RtUvk6HeNxOaeuv/zjMZujbpCXtmgtLFoLi9bCzd4b8BNCbTSi8qX3g8cn/ta6tAE3P+xTN8hLW7QWFq1FkNHx12QDx+7ij9svZTi5wjNV/XqBt9WSukFe2qK1sGgtLFqL+LON31p5TKjsf4d4i/g93FsFu1v7d+s+sGgtLFqLG5Z9MLK2NFBVAAAAAElFTkSuQmCC\" id=\"image87e8f17408\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"20.5625\" y=\"-55.34779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_13\">\n    <g id=\"xtick_13\">\n     <g id=\"line2d_25\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"23.219643\" y=\"87.74779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_14\">\n     <g id=\"line2d_26\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"49.791071\" y=\"87.74779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_14\">\n    <g id=\"ytick_13\">\n     <g id=\"line2d_27\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"20.5625\" y=\"58.519219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_3\">\n      <!-- 0 -->\n      <g transform=\"translate(7.2 62.318437) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_14\">\n     <g id=\"line2d_28\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"20.5625\" y=\"85.090647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_4\">\n      <!-- 5 -->\n      <g transform=\"translate(7.2 88.889866) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_33\">\n    <path d=\"M 20.5625 87.74779 \nL 20.5625 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_34\">\n    <path d=\"M 52.448214 87.74779 \nL 52.448214 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_35\">\n    <path d=\"M 20.5625 87.74779 \nL 52.448214 87.74779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_36\">\n    <path d=\"M 20.5625 55.862076 \nL 52.448214 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_8\">\n   <g id=\"patch_37\">\n    <path d=\"M 58.825357 87.74779 \nL 90.711071 87.74779 \nL 90.711071 55.862076 \nL 58.825357 55.862076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p6af1523c07)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABGElEQVR4nO2YvUqCYRiG71c+CdSMSEKMiCKkQVzysNza2joARx07goamDsBRN6Gm9pIiQ/An0e/1CO57fOiF51pvLrx4hhf5Qv7xHsEIBToBQD4ZcvX0XLooHXK3WpOqrvqneLQVHm2FR1sR4uKXvtO7l0dtX9zwbTbV7mrBo67bUk3y0h5thUdbkWR0FmefdNw8PUv54P6Bj1ct6ca3Md1C/VK6SV7ao63waCs82ooMlWM6Fk8qUo6vIz429d9L/Hzx7W8l1SQv7dFWeLQVSUZn2PDnpTvgX0UBoH/b4eN2q3853/FNPMNAopf2aCs82gqPtiLk82/+1bR3p+XGGR9b4g0HEMpHdIvrpXSTvLRHW+HRViQZvQf0bzQ8VH+nTAAAAABJRU5ErkJggg==\" id=\"imagedd358d668c\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"58.825357\" y=\"-55.34779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_15\">\n    <g id=\"xtick_15\">\n     <g id=\"line2d_29\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"61.4825\" y=\"87.74779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_16\">\n     <g id=\"line2d_30\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"88.053929\" y=\"87.74779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_16\">\n    <g id=\"ytick_15\">\n     <g id=\"line2d_31\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"58.825357\" y=\"58.519219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_16\">\n     <g id=\"line2d_32\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"58.825357\" y=\"85.090647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_38\">\n    <path d=\"M 58.825357 87.74779 \nL 58.825357 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_39\">\n    <path d=\"M 90.711071 87.74779 \nL 90.711071 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_40\">\n    <path d=\"M 58.825357 87.74779 \nL 90.711071 87.74779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_41\">\n    <path d=\"M 58.825357 55.862076 \nL 90.711071 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_9\">\n   <g id=\"patch_42\">\n    <path d=\"M 97.088214 87.74779 \nL 128.973929 87.74779 \nL 128.973929 55.862076 \nL 97.088214 55.862076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#pae6153d92f)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABGklEQVR4nO2ZsUoDQRRFZ7MTgkTB7whWwUaSMo1lPsDOTu39BH/CLmVKSRdIEUKqbRaChbDBBVlBFhYlLprZsUh9b/lw4J32cDOHV2yTyKULbwC+fEfqQCuGKu6P6NQ9PWL5+8OfpfafotFSaLQUGi2FNd1TbJ8TOvavGXa9C/4y+xbv93Qa5KU1WgqNliLIaOuzFNvyg46j8yH55TZ/uXsMld++0GmQl9ZoKTRaCo2WwhrnoPTfO75OVtidDejUr5fQRZdjug3y0hothUZLEWS0NZ0jbOuajm/vJ9A9TOd0e3JzhWWR022Ql9ZoKTRaCo2WImryDfx3q5nh77AxxvivTyyrim6btwK6+PqOboO8tEZLodFSBBn9B7YkPcRohvoiAAAAAElFTkSuQmCC\" id=\"image3a7dc5adab\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"97.088214\" y=\"-55.34779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_17\">\n    <g id=\"xtick_17\">\n     <g id=\"line2d_33\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"99.745357\" y=\"87.74779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_18\">\n     <g id=\"line2d_34\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"126.316786\" y=\"87.74779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_18\">\n    <g id=\"ytick_17\">\n     <g id=\"line2d_35\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"97.088214\" y=\"58.519219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_18\">\n     <g id=\"line2d_36\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"97.088214\" y=\"85.090647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_43\">\n    <path d=\"M 97.088214 87.74779 \nL 97.088214 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_44\">\n    <path d=\"M 128.973929 87.74779 \nL 128.973929 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_45\">\n    <path d=\"M 97.088214 87.74779 \nL 128.973929 87.74779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_46\">\n    <path d=\"M 97.088214 55.862076 \nL 128.973929 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_10\">\n   <g id=\"patch_47\">\n    <path d=\"M 135.351071 87.74779 \nL 167.236786 87.74779 \nL 167.236786 55.862076 \nL 135.351071 55.862076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p2cd4f0bb44)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABHUlEQVR4nO2XvWrCUBiGvxPTOAhFp9riJoJLXbwEha69lG69hdLdSxA6ikOhmyB16ewPqIi6upUWUiXHofP7Fhy+euB71oc3eXKGkLjD4MULwJWukPoTd1OlPnvrYnmR0G10StB/Y9FaWLQWFq1FLEkeSr+e0bGfT6CLWvf8zl+f2EU5Og3ypC1aC4vWIsjoWPY/2Eb8mVz9Fk9rTbo99MmnqYdfy7/XpvZMsWgtLFoLi9YiFke61ws63r9/QOeqDb6dLqFL7tp0G+RJW7QWFq1FkNGxFC6xLVfo+Hv1Ct1Dnb+2Or0nLNOUboM8aYvWwqK1sGgtXLYZw//1bITfwyIifjSELt3u6DZ/XYQu9/hMt0GetEVrYdFaBBl9BPN7MYVMttTRAAAAAElFTkSuQmCC\" id=\"image26010d2c76\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"135.351071\" y=\"-55.34779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_19\">\n    <g id=\"xtick_19\">\n     <g id=\"line2d_37\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"138.008214\" y=\"87.74779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_20\">\n     <g id=\"line2d_38\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"164.579643\" y=\"87.74779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_20\">\n    <g id=\"ytick_19\">\n     <g id=\"line2d_39\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"135.351071\" y=\"58.519219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_20\">\n     <g id=\"line2d_40\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"135.351071\" y=\"85.090647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_48\">\n    <path d=\"M 135.351071 87.74779 \nL 135.351071 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_49\">\n    <path d=\"M 167.236786 87.74779 \nL 167.236786 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_50\">\n    <path d=\"M 135.351071 87.74779 \nL 167.236786 87.74779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_51\">\n    <path d=\"M 135.351071 55.862076 \nL 167.236786 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_11\">\n   <g id=\"patch_52\">\n    <path d=\"M 173.613929 87.74779 \nL 205.499643 87.74779 \nL 205.499643 55.862076 \nL 173.613929 55.862076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p0d754d2a73)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABGElEQVR4nO2YsUpCYRiG/Q6/lKSkEG0NegYVGs4UNARCN+PgpTTm4gV4AUEqjmexocmhhjZxOJtUiGl1/q7gfYOGz374nvXh5Tx82/nl63HsCwApHiL1K1I9od7vPrBcv9Ft9JegfWPRWli0FhathZODEpQ+W9Bx1L7A2/cV3fqHKZb1Jv8utf8Ui9bCorUIMtpFjQTK7/Ser5MOVH424ttGC7tylU6DvLRFa2HRWli0Fs6vMmxr/Bmg8LmFSuJzOs2HA7y9vKLbIC9t0VpYtBZBRrt8+YLtK/+jzucplpUa3fZuJtD1767pNshLW7QWFq2FRWvh5PQMyvz5iY4lxs8AcnRMt/3bLpbbDd0GeWmL1sKitQgy+gdVSjEuUxgl6wAAAABJRU5ErkJggg==\" id=\"imageff9040c2f3\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"173.613929\" y=\"-55.34779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_21\">\n    <g id=\"xtick_21\">\n     <g id=\"line2d_41\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"176.271071\" y=\"87.74779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_22\">\n     <g id=\"line2d_42\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"202.8425\" y=\"87.74779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_22\">\n    <g id=\"ytick_21\">\n     <g id=\"line2d_43\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"173.613929\" y=\"58.519219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_22\">\n     <g id=\"line2d_44\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"173.613929\" y=\"85.090647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_53\">\n    <path d=\"M 173.613929 87.74779 \nL 173.613929 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_54\">\n    <path d=\"M 205.499643 87.74779 \nL 205.499643 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_55\">\n    <path d=\"M 173.613929 87.74779 \nL 205.499643 87.74779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_56\">\n    <path d=\"M 173.613929 55.862076 \nL 205.499643 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_12\">\n   <g id=\"patch_57\">\n    <path d=\"M 211.876786 87.74779 \nL 243.7625 87.74779 \nL 243.7625 55.862076 \nL 211.876786 55.862076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#pd7f4098616)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABEUlEQVR4nO2YzwoBURSH545ZKs2ChayUkrJhz8YbyFt4Ch7BCyh72VuwVEZZURQrpcak/EuYuRbW57ewONw63/br13yd1W1UuJlri0CPB5T6UChjj9AR7Y4+nNrff/V3SDQXEs2FRHPhWKeAtq8nXm+XpFL5EpwqN0W6yI7BrZGXlmguJJoLI6NVuJ6ST1M7nYPjcDakZbCHW7taJ50OdngL7Z8i0VxINBcSzYWjvREpdSUBxyqZoV22iL/8uJMq6nXg1MhLSzQXEs2FkdGO5YMnZNyFY73yaHfAz0vrdiZVs9WHUyMvLdFcSDQXEs2FenXb9C+EWgOvQ/BXVeF7RIsJLa8XuDXy0hLNhURzYWT0G/a+OwAng6+yAAAAAElFTkSuQmCC\" id=\"image87aba363d4\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"211.876786\" y=\"-55.34779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_23\">\n    <g id=\"xtick_23\">\n     <g id=\"line2d_45\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"214.533929\" y=\"87.74779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_24\">\n     <g id=\"line2d_46\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"241.105357\" y=\"87.74779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_24\">\n    <g id=\"ytick_23\">\n     <g id=\"line2d_47\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"211.876786\" y=\"58.519219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_24\">\n     <g id=\"line2d_48\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"211.876786\" y=\"85.090647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_58\">\n    <path d=\"M 211.876786 87.74779 \nL 211.876786 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_59\">\n    <path d=\"M 243.7625 87.74779 \nL 243.7625 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_60\">\n    <path d=\"M 211.876786 87.74779 \nL 243.7625 87.74779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_61\">\n    <path d=\"M 211.876786 55.862076 \nL 243.7625 55.862076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_13\">\n   <g id=\"patch_62\">\n    <path d=\"M 20.5625 135.26779 \nL 52.448214 135.26779 \nL 52.448214 103.382076 \nL 20.5625 103.382076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p37a9cd4af8)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABD0lEQVR4nO2YPUrEUBRG75OARCb4B4GQZsiQQg2jNroke/vZxMAwm5h2NqALmNJCmxRWEWIhGcQXnyv4vvLig3vaj5McbhXi/PIxCOLqFk4iIrIf8OZ/uJuXeGtfqXrAn/w/sWgtLFoLi9YikZMzOLqypnJoX7A7veBvHke8NXdUjfLSFq2FRWsRZXQi3sPRFRWVw+4Jb8MXdV3V4DHNqBvlpS1aC4vWwqK1cA8ygb8Q1v0blX83KzzmBX9xfQ238LylbpSXtmgtLFqLKKOTxQ35e+m/uT2/h5PLTqnK9nCYUjfKS1u0FhathUVrkZxf4k9Id3TM7c8OTqF75245w27/QdUoL23RWli0FlFG/wEuRS0OqhkHRwAAAABJRU5ErkJggg==\" id=\"imaged0889b1ca4\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"20.5625\" y=\"-102.86779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_25\">\n    <g id=\"xtick_25\">\n     <g id=\"line2d_49\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"23.219643\" y=\"135.26779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_26\">\n     <g id=\"line2d_50\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"49.791071\" y=\"135.26779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_26\">\n    <g id=\"ytick_25\">\n     <g id=\"line2d_51\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"20.5625\" y=\"106.039219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_5\">\n      <!-- 0 -->\n      <g transform=\"translate(7.2 109.838438) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_26\">\n     <g id=\"line2d_52\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"20.5625\" y=\"132.610647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_6\">\n      <!-- 5 -->\n      <g transform=\"translate(7.2 136.409866) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_63\">\n    <path d=\"M 20.5625 135.26779 \nL 20.5625 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_64\">\n    <path d=\"M 52.448214 135.26779 \nL 52.448214 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_65\">\n    <path d=\"M 20.5625 135.26779 \nL 52.448214 135.26779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_66\">\n    <path d=\"M 20.5625 103.382076 \nL 52.448214 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_14\">\n   <g id=\"patch_67\">\n    <path d=\"M 58.825357 135.26779 \nL 90.711071 135.26779 \nL 90.711071 103.382076 \nL 58.825357 103.382076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p94312a9f28)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABGklEQVR4nO2YvWoCQRRG78iYQlb8IZCNBMFWG8XKwtLKNo9kbWedLlU6q/Q+gCJYCJoiLCsmQgpjpa5P8H3lxYF72sO3c5hiWdadP98yQTzWoBIRkWSLXanKt19r7JpdOs3xJ98nFq2FRWth0Vp491SH8jp9p2PX7WH5Td7hIpIt51juU7oN8qYtWguL1iLIaJ/9/UC5+5jR8XN/gGUh4icf9tidTnQa5E1btBYWrYVFa+FzjRaUo0VCx5PNCsuXBj85xZ+fbjCk0yBv2qK1sGgtgoz2Uo6hHL+26Tg7/ELn/o/85BifK1GFToO8aYvWwqK1sGgtvPN5KB96HTp2dfL5GeO/sSIirlDE8nKh2yBv2qK1sGgtgoy+Acz8K2oWUspBAAAAAElFTkSuQmCC\" id=\"imagebee3538138\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"58.825357\" y=\"-102.86779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_27\">\n    <g id=\"xtick_27\">\n     <g id=\"line2d_53\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"61.4825\" y=\"135.26779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_28\">\n     <g id=\"line2d_54\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"88.053929\" y=\"135.26779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_28\">\n    <g id=\"ytick_27\">\n     <g id=\"line2d_55\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"58.825357\" y=\"106.039219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_28\">\n     <g id=\"line2d_56\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"58.825357\" y=\"132.610647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_68\">\n    <path d=\"M 58.825357 135.26779 \nL 58.825357 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_69\">\n    <path d=\"M 90.711071 135.26779 \nL 90.711071 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_70\">\n    <path d=\"M 58.825357 135.26779 \nL 90.711071 135.26779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_71\">\n    <path d=\"M 58.825357 103.382076 \nL 90.711071 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_15\">\n   <g id=\"patch_72\">\n    <path d=\"M 97.088214 135.26779 \nL 128.973929 135.26779 \nL 128.973929 103.382076 \nL 97.088214 103.382076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p107ab9e58e)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABHElEQVR4nO2ZsWpCQRBFZ/UZiaBBQrAVUgQsrCW9jd+SLq1lulQWwfr9gZ1oaWMTrYRAmqCiIURQECEQs5svuLcc3sKc9nJnDlsMPJ47z0ZBEMc9jEREcrdNHF5c0q6cf3FWqvC9fHI2MWktTFoLk9bC+d0a3umweuPtchUPLhRp1e82uFu5pt0oX9qktTBpLaKUTvz7KwwdOWkiIj7twSz/+Mw3L/E5dbU6rUb50iathUlrYdJaJPL9CcOQL9Dyw9MAZi93Ddp1rTbee/ii3Shf2qS1MGktopR2f/Mx/hrffvC29zAK0wmtJt0+7trJywgmrYVJaxGltPPLBf67dXVDy36YksmOdnP3HZiFnxPv0jSjmLQWJq1FlNL/TQ06wxItu/oAAAAASUVORK5CYII=\" id=\"image08ce860583\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"97.088214\" y=\"-102.86779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_29\">\n    <g id=\"xtick_29\">\n     <g id=\"line2d_57\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"99.745357\" y=\"135.26779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_30\">\n     <g id=\"line2d_58\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"126.316786\" y=\"135.26779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_30\">\n    <g id=\"ytick_29\">\n     <g id=\"line2d_59\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"97.088214\" y=\"106.039219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_30\">\n     <g id=\"line2d_60\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"97.088214\" y=\"132.610647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_73\">\n    <path d=\"M 97.088214 135.26779 \nL 97.088214 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_74\">\n    <path d=\"M 128.973929 135.26779 \nL 128.973929 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_75\">\n    <path d=\"M 97.088214 135.26779 \nL 128.973929 135.26779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_76\">\n    <path d=\"M 97.088214 103.382076 \nL 128.973929 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_16\">\n   <g id=\"patch_77\">\n    <path d=\"M 135.351071 135.26779 \nL 167.236786 135.26779 \nL 167.236786 103.382076 \nL 135.351071 103.382076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#pff69955f02)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABHElEQVR4nO2YIWuCURSGv/v5CQbFyYZYDIrgGMzgwGSxCmaLYBP8FXaLzdUl/4FZFgYGi0bLBuJAcGARNubutay+r+3ghfPUh5f7cOI1v4OuCwDmoYLUv3+Czj4P6TY2GEPnvrZ0G1J7pWi0FBothUZLEQXJJJRhvUXH5iYLnS0W6dbtN8R90q2Xl9ZoKTRaCi+joyAeh9LtPuj42OtAt13v6bZ0/4jlzzfdenlpjZZCo6XQaCkiky9A6ewfHScaNehG0xe6HZer+N3lG916eWmNlkKjpfAy2pxmE/hreolYrQmdO+zo1m7WOOo2R7deXlqjpdBoKTRaisjNX6EM2306tu8r6EwqQ7cmfQedW8zo1stLa7QUGi2Fl9Fn/rs2ycVDuXcAAAAASUVORK5CYII=\" id=\"image38adb92148\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"135.351071\" y=\"-102.86779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_31\">\n    <g id=\"xtick_31\">\n     <g id=\"line2d_61\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"138.008214\" y=\"135.26779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_32\">\n     <g id=\"line2d_62\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"164.579643\" y=\"135.26779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_32\">\n    <g id=\"ytick_31\">\n     <g id=\"line2d_63\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"135.351071\" y=\"106.039219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_32\">\n     <g id=\"line2d_64\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"135.351071\" y=\"132.610647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_78\">\n    <path d=\"M 135.351071 135.26779 \nL 135.351071 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_79\">\n    <path d=\"M 167.236786 135.26779 \nL 167.236786 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_80\">\n    <path d=\"M 135.351071 135.26779 \nL 167.236786 135.26779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_81\">\n    <path d=\"M 135.351071 103.382076 \nL 167.236786 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_17\">\n   <g id=\"patch_82\">\n    <path d=\"M 173.613929 135.26779 \nL 205.499643 135.26779 \nL 205.499643 103.382076 \nL 173.613929 103.382076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p6bc373f8d4)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABFklEQVR4nO2ZsU7CUBSG74VWKjSakHRglQ4SQeMjuDnyImy8BYwkuPsAhsFJJxc2RuNsHBhK4mLUtPUyOP//eMJJzrf++eDLGZoGfLVeBYdoxnByzjnfOoZbI7+mbr15xuNuS90GXQ8Ui5bCoqWwaCkid5TAMTw9cHuIn8Xh7JKqP7M53JLxLXVVXtqipbBoKVRGR759Akf8zvqPz6/IyO8R6j889gfUVXlpi5bCoqWwaCmi8PWJ17Kkcnh/w2PN3enjK9yWdyPqqry0RUth0VKojOavpmlK5fCCf/msPu6pu5jc4M/9/aauyktbtBQWLYVFSxH5bg+OPub/brnzCzg1s4y7RYG/t3NKVZWXtmgpLFoKldF7PgQuG76LJwAAAAAASUVORK5CYII=\" id=\"image1c60275ec9\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"173.613929\" y=\"-102.86779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_33\">\n    <g id=\"xtick_33\">\n     <g id=\"line2d_65\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"176.271071\" y=\"135.26779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_34\">\n     <g id=\"line2d_66\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"202.8425\" y=\"135.26779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_34\">\n    <g id=\"ytick_33\">\n     <g id=\"line2d_67\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"173.613929\" y=\"106.039219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_34\">\n     <g id=\"line2d_68\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"173.613929\" y=\"132.610647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_83\">\n    <path d=\"M 173.613929 135.26779 \nL 173.613929 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_84\">\n    <path d=\"M 205.499643 135.26779 \nL 205.499643 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_85\">\n    <path d=\"M 173.613929 135.26779 \nL 205.499643 135.26779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_86\">\n    <path d=\"M 173.613929 103.382076 \nL 205.499643 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_18\">\n   <g id=\"patch_87\">\n    <path d=\"M 211.876786 135.26779 \nL 243.7625 135.26779 \nL 243.7625 103.382076 \nL 211.876786 103.382076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p79269eded6)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABGElEQVR4nO2YsUoDQRRFZ9ZZCWKxEBQkdlZpQkghVmksJD8g+A3+hd9gn1+wTW21QWwScOMWgghJY2GTwO4yY2F9bxP24cA77eUkh1dMsbZ5egwGkFzeoOmPAwcn//XO3T1IWvvlFtFoKTRaCo2Wwpn0EI5h88Ht4wxvdUXVpH8FN1++cJeu/xSNlkKjpYgy2tneBRztyTmVfTGHW1i+8n8mT54pF1SN8tIaLYVGS6HRUrhQ4PfUds+4nZ1id0jeYWOMdSnc7u8eqBvlpTVaCo2WIspo28ym+Kvp6Jrb2x84+c8Vd6sdnEL+TNUoL63RUmi0FBothTOdI7w2NZXD9xpuyWBMXf+WY3dyS90oL63RUmi0FFFG/wKlVzCtJWzaHgAAAABJRU5ErkJggg==\" id=\"image51b5f410e3\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"211.876786\" y=\"-102.86779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_35\">\n    <g id=\"xtick_35\">\n     <g id=\"line2d_69\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"214.533929\" y=\"135.26779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_36\">\n     <g id=\"line2d_70\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"241.105357\" y=\"135.26779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_36\">\n    <g id=\"ytick_35\">\n     <g id=\"line2d_71\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"211.876786\" y=\"106.039219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_36\">\n     <g id=\"line2d_72\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"211.876786\" y=\"132.610647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_88\">\n    <path d=\"M 211.876786 135.26779 \nL 211.876786 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_89\">\n    <path d=\"M 243.7625 135.26779 \nL 243.7625 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_90\">\n    <path d=\"M 211.876786 135.26779 \nL 243.7625 135.26779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_91\">\n    <path d=\"M 211.876786 103.382076 \nL 243.7625 103.382076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_19\">\n   <g id=\"patch_92\">\n    <path d=\"M 20.5625 182.78779 \nL 52.448214 182.78779 \nL 52.448214 150.902076 \nL 20.5625 150.902076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p33c571d1ce)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABF0lEQVR4nO2WsWoCURBFZ9ZnlCBaJKVIICimClZiH0Ig/oOlpY0fYZcqlvmNdJIfECsDqWzEZsFGEgyBfSms7y0HH8xpL2f3MMWy+vcyiQLQ3gBNJ/Id3lodqmr9Cm5x+UHdjK5nikdb4dFWeLQVQVThmN31qVwcF3iM8PN/4vcHb/ucqkle2qOt8GgrkozWsdTgt2m+fqdyMZ/hBz888TfXGnCKnyuqJnlpj7bCo63waCvC69sUr6UylfXxGY/bDXWz4QhuxUWFu3Q9UzzaCo+2IsnoIDdtOGr1ksqxeYvd9j119bqJt8Oeukle2qOt8GgrPNqKIJsvOMZWl9vHb+ySTUSkCPi3N+Zb6iZ5aY+2wqOtSDL6HxKWL8b1DLwdAAAAAElFTkSuQmCC\" id=\"image6f3c01e762\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"20.5625\" y=\"-150.38779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_37\">\n    <g id=\"xtick_37\">\n     <g id=\"line2d_73\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"23.219643\" y=\"182.78779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_38\">\n     <g id=\"line2d_74\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"49.791071\" y=\"182.78779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_38\">\n    <g id=\"ytick_37\">\n     <g id=\"line2d_75\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"20.5625\" y=\"153.559219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_7\">\n      <!-- 0 -->\n      <g transform=\"translate(7.2 157.358438) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_38\">\n     <g id=\"line2d_76\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"20.5625\" y=\"180.130647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_8\">\n      <!-- 5 -->\n      <g transform=\"translate(7.2 183.929866) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_93\">\n    <path d=\"M 20.5625 182.78779 \nL 20.5625 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_94\">\n    <path d=\"M 52.448214 182.78779 \nL 52.448214 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_95\">\n    <path d=\"M 20.5625 182.78779 \nL 52.448214 182.78779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_96\">\n    <path d=\"M 20.5625 150.902076 \nL 52.448214 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_20\">\n   <g id=\"patch_97\">\n    <path d=\"M 58.825357 182.78779 \nL 90.711071 182.78779 \nL 90.711071 150.902076 \nL 58.825357 150.902076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p57cb5492df)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABGUlEQVR4nO2YvWoCQRRG70wGXfKjCFERdlHstBRfIC9iaSob38DGFxAEX8UmTbpAIGUEK1MJIqIuRHHHJ/iu3cWBe9rDxz1Mtay5fM89IXIRVEREPj1AZysxu81+PvG288ZuLWvvFI2WQqOl0GgpHNkHKC+TMTu2vT502WrBbv3HHG/LNf4ua+8UjZZCo6UIMtr50z+Ugxn+fCQimr4PsSy+8pebTeyOe3Ya5EtrtBQaLYVGS+Fsow3lqJuwY/+3xPJ84rdpCp0pldltkC+t0VJotBRBRjvKP0L5VH3h15s1ds+FG5cdVDZpsdMgX1qjpdBoKTRaCmdcDsqozv8G8LstdObGYRPXoct+v9htkC+t0VJotBRBRl8BpkgtP+COP4AAAAAASUVORK5CYII=\" id=\"image1e24d23edc\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"58.825357\" y=\"-150.38779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_39\">\n    <g id=\"xtick_39\">\n     <g id=\"line2d_77\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"61.4825\" y=\"182.78779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_40\">\n     <g id=\"line2d_78\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"88.053929\" y=\"182.78779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_40\">\n    <g id=\"ytick_39\">\n     <g id=\"line2d_79\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"58.825357\" y=\"153.559219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_40\">\n     <g id=\"line2d_80\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"58.825357\" y=\"180.130647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_98\">\n    <path d=\"M 58.825357 182.78779 \nL 58.825357 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_99\">\n    <path d=\"M 90.711071 182.78779 \nL 90.711071 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_100\">\n    <path d=\"M 58.825357 182.78779 \nL 90.711071 182.78779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_101\">\n    <path d=\"M 58.825357 150.902076 \nL 90.711071 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_21\">\n   <g id=\"patch_102\">\n    <path d=\"M 97.088214 182.78779 \nL 128.973929 182.78779 \nL 128.973929 150.902076 \nL 97.088214 150.902076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#pa4ed43525a)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABG0lEQVR4nO2YsWoCQRRF3+5OsVtoRNgugTSaNsTORiSWfoEgxDYWWvsP6RbyGyFFurR2apvCNDGFixuCCUiQMJMvuC9g8XDgnfZwmcMrJ/h9vHeEKHKoiIioXMHu/IKdhukpdO7rg9+y9kjRaCk0WgqNlsJQnEAZdm/YsV0usPzc8C+nZ1C59Rs79fLSGi2FRkvhZbRxTw/Y1q/Y8bDZhy67G7BbG0VY7n/YrZeX1mgpNFoKjZbCBK3rg8fZuAPdfjpnt0lvBJ2dPbNbLy+t0VJotBReRhs6qULptv/8Xt5OoIt33/zLSQm7Mm4i8vTSGi2FRkuh0VIYen2BMrhss2NXvEMX1hrs1q7wu5Sv2K2Xl9ZoKTRaCi+j/wC4xDD6FEU/CgAAAABJRU5ErkJggg==\" id=\"image65fb963f14\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"97.088214\" y=\"-150.38779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_41\">\n    <g id=\"xtick_41\">\n     <g id=\"line2d_81\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"99.745357\" y=\"182.78779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_42\">\n     <g id=\"line2d_82\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"126.316786\" y=\"182.78779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_42\">\n    <g id=\"ytick_41\">\n     <g id=\"line2d_83\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"97.088214\" y=\"153.559219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_42\">\n     <g id=\"line2d_84\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"97.088214\" y=\"180.130647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_103\">\n    <path d=\"M 97.088214 182.78779 \nL 97.088214 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_104\">\n    <path d=\"M 128.973929 182.78779 \nL 128.973929 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_105\">\n    <path d=\"M 97.088214 182.78779 \nL 128.973929 182.78779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_106\">\n    <path d=\"M 97.088214 150.902076 \nL 128.973929 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_22\">\n   <g id=\"patch_107\">\n    <path d=\"M 135.351071 182.78779 \nL 167.236786 182.78779 \nL 167.236786 150.902076 \nL 135.351071 150.902076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p4f84ea0f34)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABJklEQVR4nO2YMUtCYRSGz7l+g2bNIYHh4CBSQ0419A9cpNl+QVNTU6tL4ebmIC5u0a8IWoJsjgQhKEKXwMqug/P7uh3uB+dZH977Pdzh43L19/o8FYAeHiG1JpeD6u2qS6f7/Rssp690m1CbUTzaCo+2wqOtCOyu1YNjOtbCDnS71SHdJuUadP/vE76lNqN4tBUebUWU0UFLe9guvul42b+F7vJ+TLe9HvwiFpl90W2Ub9qjrfBoKzzaiiCJQpnOPulYW23oLkYP/OR8EZ/78kynUb5pj7bCo62IMjrI1ja2fz90nNRPoKuPn+h2OehAp5UKP5fajOLRVni0FR5tRUgf8SekNs/oOJ1/YJkn97+ISOMUP/duwx9X/uRs4tFWeLQVUUavAIKTLxtO5UnHAAAAAElFTkSuQmCC\" id=\"image27eede1fa8\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"135.351071\" y=\"-150.38779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_43\">\n    <g id=\"xtick_43\">\n     <g id=\"line2d_85\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"138.008214\" y=\"182.78779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_44\">\n     <g id=\"line2d_86\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"164.579643\" y=\"182.78779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_44\">\n    <g id=\"ytick_43\">\n     <g id=\"line2d_87\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"135.351071\" y=\"153.559219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_44\">\n     <g id=\"line2d_88\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"135.351071\" y=\"180.130647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_108\">\n    <path d=\"M 135.351071 182.78779 \nL 135.351071 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_109\">\n    <path d=\"M 167.236786 182.78779 \nL 167.236786 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_110\">\n    <path d=\"M 135.351071 182.78779 \nL 167.236786 182.78779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_111\">\n    <path d=\"M 135.351071 150.902076 \nL 167.236786 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_23\">\n   <g id=\"patch_112\">\n    <path d=\"M 173.613929 182.78779 \nL 205.499643 182.78779 \nL 205.499643 150.902076 \nL 173.613929 150.902076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p97b08c5e75)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABDElEQVR4nO2YvwqBURiHz+EgMXz5ny99ZhlcgM0tuAwDch02VyCTURY3QJiIRZKFsimlhOMK3t/45tT7rL+evqd3+IajP4eVVQT2uKEmpZRSOh/QW1CB7nc2osdiGbohuP4pEs2FRHMh0VwYe9qRo13Psd3IkJN+v6D6HI7JLd5uQdfJS0s0FxLNhZPRRkWi5Kj9ErYfd9pNeFDtTffkNuhq6Dp5aYnmQqK5kGguDHoGsLcrtsF/+rOYQLXfrJFbqFqHrpOXlmguJJoLJ6ON0qDbGGynC/S2XUI11unSYxh/18lLSzQXEs2FRHNhdNIjR5vzsX0501sqi10NngnsF6pOXlqiuZBoLpyM/gF9jiqzM9/riQAAAABJRU5ErkJggg==\" id=\"image12ffd11c52\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"173.613929\" y=\"-150.38779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_45\">\n    <g id=\"xtick_45\">\n     <g id=\"line2d_89\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"176.271071\" y=\"182.78779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_46\">\n     <g id=\"line2d_90\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"202.8425\" y=\"182.78779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_46\">\n    <g id=\"ytick_45\">\n     <g id=\"line2d_91\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"173.613929\" y=\"153.559219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_46\">\n     <g id=\"line2d_92\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"173.613929\" y=\"180.130647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_113\">\n    <path d=\"M 173.613929 182.78779 \nL 173.613929 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_114\">\n    <path d=\"M 205.499643 182.78779 \nL 205.499643 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_115\">\n    <path d=\"M 173.613929 182.78779 \nL 205.499643 182.78779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_116\">\n    <path d=\"M 173.613929 150.902076 \nL 205.499643 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_24\">\n   <g id=\"patch_117\">\n    <path d=\"M 211.876786 182.78779 \nL 243.7625 182.78779 \nL 243.7625 150.902076 \nL 211.876786 150.902076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#pb1577dfa55)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABDklEQVR4nO2YvQ4BQRRGd8awKDQkolBo0ChEoVPoPYrCM3gLD6AVvUKBRHgCEQWR+CmIkjWjUN+vvNlJ7mlPvszJFLvJqGg1cQGBrrcp9efzpl0iAad2tyWdLtfgVkMbUySaC4nmQqK5MG49p22jA8fudiSdyuXhFvloOIBbL29aormQaC68jDaqUCSluxzg2N1OtHtc8cmpNKn6oyWcennTEs2FRHMh0Vwoez+STwh2MYVj3eqSzp73eFuq0NvNDG+hjSkSzYVEc+FlNPzkqWwOjr+zMS1fT7jVnR4tTYi30MYUieZCormQaC6Mu4NngDADx6rapKWN8MmO/D0EgUnCqZc3LdFcSDQXXkb/AAhkOJkB+DrFAAAAAElFTkSuQmCC\" id=\"image9e4beb9287\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"211.876786\" y=\"-150.38779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_47\">\n    <g id=\"xtick_47\">\n     <g id=\"line2d_93\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"214.533929\" y=\"182.78779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_48\">\n     <g id=\"line2d_94\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"241.105357\" y=\"182.78779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_48\">\n    <g id=\"ytick_47\">\n     <g id=\"line2d_95\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"211.876786\" y=\"153.559219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_48\">\n     <g id=\"line2d_96\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"211.876786\" y=\"180.130647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_118\">\n    <path d=\"M 211.876786 182.78779 \nL 211.876786 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_119\">\n    <path d=\"M 243.7625 182.78779 \nL 243.7625 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_120\">\n    <path d=\"M 211.876786 182.78779 \nL 243.7625 182.78779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_121\">\n    <path d=\"M 211.876786 150.902076 \nL 243.7625 150.902076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_25\">\n   <g id=\"patch_122\">\n    <path d=\"M 20.5625 230.30779 \nL 52.448214 230.30779 \nL 52.448214 198.422076 \nL 20.5625 198.422076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p614d8b9236)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABF0lEQVR4nO2YsWoCQRRF3yyDRKOYwsImgQQFg9r6AdbmGyzFfEp6/Zl06WOliAkBRYugiWBj1MBOitT3lg8H3mkPd+cwxcKu+316DAJw7Qek/n2xBF0YvdCtXN9hd/yh04Q/+TyxaC0sWguL1sL1JA/f08PpM18f91CFzznfZi6wG7/SaZQ3bdFaWLQWUUb7waAPpcsV6DhdzrBcfPCTq3Wowm5Hp1HetEVrYdFaWLQWXkplbLN5Ok7uW9Cl7xO6dbcNLLcbfi61Z4pFa2HRWkQZ7aVQhNJdXtFx8Bnokk6Xb8mXvKs06TbKm7ZoLSxaC4vWwsv3GsrwtaLjdPWG5elAt+6mhmWAP3JFJNKbtmgtLFqLKKP/AOeAMlK0T4pCAAAAAElFTkSuQmCC\" id=\"image61e2ad8fd5\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"20.5625\" y=\"-197.90779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_49\">\n    <g id=\"xtick_49\">\n     <g id=\"line2d_97\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"23.219643\" y=\"230.30779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_50\">\n     <g id=\"line2d_98\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"49.791071\" y=\"230.30779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_50\">\n    <g id=\"ytick_49\">\n     <g id=\"line2d_99\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"20.5625\" y=\"201.079219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_9\">\n      <!-- 0 -->\n      <g transform=\"translate(7.2 204.878438) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_50\">\n     <g id=\"line2d_100\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"20.5625\" y=\"227.650647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_10\">\n      <!-- 5 -->\n      <g transform=\"translate(7.2 231.449866) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_123\">\n    <path d=\"M 20.5625 230.30779 \nL 20.5625 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_124\">\n    <path d=\"M 52.448214 230.30779 \nL 52.448214 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_125\">\n    <path d=\"M 20.5625 230.30779 \nL 52.448214 230.30779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_126\">\n    <path d=\"M 20.5625 198.422076 \nL 52.448214 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_26\">\n   <g id=\"patch_127\">\n    <path d=\"M 58.825357 230.30779 \nL 90.711071 230.30779 \nL 90.711071 198.422076 \nL 58.825357 198.422076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p4402941511)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABH0lEQVR4nO2ZMWoCQRSG36yTTnCrbSKWgitJQKutUqbxAt4hXYqQa4R4BO8gdikCaRar7Sz0AkkgEQTdmZzg/8uHA+9rf/43H6+a2XXx8BMFcH59RpGIiGSzOQ47V7Qb63ccDm/5uTS9UExaC5PWwqS18GHfwPDxZUnLi9ENDotr2o2fHzBz5ZR2k9y0SWth0lokKe1dMYDh29MDLWfVjEz2tNv2V3huf8jPpemFYtJamLQWJq2Fj79fMHTlmJZDg6+X2d097bpejuduN7Sb5KZNWguT1iJJae88/roZuz3e7uDrZ9jhV76IiIwnOGtqWk1y0yathUlrYdJauPD3Df9uhXrNy3kBs3g68pPbFkZZWdFqkps2aS1MWoskpf8BAeIxBc24b9UAAAAASUVORK5CYII=\" id=\"imagee122e4e95d\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"58.825357\" y=\"-197.90779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_51\">\n    <g id=\"xtick_51\">\n     <g id=\"line2d_101\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"61.4825\" y=\"230.30779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_52\">\n     <g id=\"line2d_102\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"88.053929\" y=\"230.30779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_52\">\n    <g id=\"ytick_51\">\n     <g id=\"line2d_103\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"58.825357\" y=\"201.079219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_52\">\n     <g id=\"line2d_104\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"58.825357\" y=\"227.650647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_128\">\n    <path d=\"M 58.825357 230.30779 \nL 58.825357 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_129\">\n    <path d=\"M 90.711071 230.30779 \nL 90.711071 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_130\">\n    <path d=\"M 58.825357 230.30779 \nL 90.711071 230.30779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_131\">\n    <path d=\"M 58.825357 198.422076 \nL 90.711071 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_27\">\n   <g id=\"patch_132\">\n    <path d=\"M 97.088214 230.30779 \nL 128.973929 230.30779 \nL 128.973929 198.422076 \nL 97.088214 198.422076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p829ab20925)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABFElEQVR4nO2YwUoCURSGzx1miiRRVwkJ+g61dBk+QfQYte1FwpX4AK50JSjiUgQXCUJQCLYLgtJdMXPHJ/j/oMVhLpxv+/EPH3cxXK7L3la5APLlFCkREUknE+iS+0e69bMhdK7RotuI2oJi0VpYtBYWrUWcf+z+PX7oL6B7ut3SrbtuY1eu0W2QJ23RWli0FkFGu3TUhVdTqTf5ev8JVXR1Q6f+ec6/TQjypC1aC4vWwqK1iCVOsM09XycnULnTEt/+/mB3XqHTIE/aorWwaC2CjI7l8IXt2R+/rZc1VNnrhk6jzh10fjrgW15VTCxaC4vWwqK1cOm4h58Q9t98fXGJnc/49p28qlbt1bQYWLQWFq3FEW/yMqP4ytEbAAAAAElFTkSuQmCC\" id=\"imagef23201a9ff\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"97.088214\" y=\"-197.90779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_53\">\n    <g id=\"xtick_53\">\n     <g id=\"line2d_105\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"99.745357\" y=\"230.30779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_54\">\n     <g id=\"line2d_106\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"126.316786\" y=\"230.30779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_54\">\n    <g id=\"ytick_53\">\n     <g id=\"line2d_107\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"97.088214\" y=\"201.079219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_54\">\n     <g id=\"line2d_108\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"97.088214\" y=\"227.650647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_133\">\n    <path d=\"M 97.088214 230.30779 \nL 97.088214 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_134\">\n    <path d=\"M 128.973929 230.30779 \nL 128.973929 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_135\">\n    <path d=\"M 97.088214 230.30779 \nL 128.973929 230.30779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_136\">\n    <path d=\"M 97.088214 198.422076 \nL 128.973929 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_28\">\n   <g id=\"patch_137\">\n    <path d=\"M 135.351071 230.30779 \nL 167.236786 230.30779 \nL 167.236786 198.422076 \nL 135.351071 198.422076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p4817fb5466)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABFklEQVR4nO2Qv0rCYRSGz/fzM5IkMnRQQlycrMVoEaQLaHZw6VYcvAA3cQkvodWpoS3KrZY2HaRBwr/VIH5dwfuOBz84z/rycB6O240egiD+fuEkIhLGL3B7Hz5T96pzj8dqjboJXQ8Ui9bCorWwaC28LL7xmstT2d004DboPlK337zD48+KulF+2qK1sGgtooz2UizjNX3E7c0UTr32NVXdySnc9tsldaP8tEVrYdFaWLQWXrJneP2acLtQglO6fklVd45dGT9RN8pPW7QWFq1FlNHeJSk4hs2aysltC7sXVeruP9/wmCvwu3Q9UCxaC4vWwqK18OHjFa/HGW6v5nAK8xlVk2IFu6xJIv20RWth0VpEGf0PsdYvL0pIN74AAAAASUVORK5CYII=\" id=\"image52f3f63d89\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"135.351071\" y=\"-197.90779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_55\">\n    <g id=\"xtick_55\">\n     <g id=\"line2d_109\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"138.008214\" y=\"230.30779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_56\">\n     <g id=\"line2d_110\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"164.579643\" y=\"230.30779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_56\">\n    <g id=\"ytick_55\">\n     <g id=\"line2d_111\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"135.351071\" y=\"201.079219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_56\">\n     <g id=\"line2d_112\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"135.351071\" y=\"227.650647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_138\">\n    <path d=\"M 135.351071 230.30779 \nL 135.351071 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_139\">\n    <path d=\"M 167.236786 230.30779 \nL 167.236786 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_140\">\n    <path d=\"M 135.351071 230.30779 \nL 167.236786 230.30779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_141\">\n    <path d=\"M 135.351071 198.422076 \nL 167.236786 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_29\">\n   <g id=\"patch_142\">\n    <path d=\"M 173.613929 230.30779 \nL 205.499643 230.30779 \nL 205.499643 198.422076 \nL 173.613929 198.422076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p969553b572)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABD0lEQVR4nO2YMQtBURiG7zmOK+Wmm3TFgEGymJjMBhb/wKzs/oSVPyGD2aRsSlksFj/AIGUi9xjM36sMX059z/r03vt0htPtqtdpZz0Cez5S6kMYkUrXW3Aab5akU80O3Gpc9Z9INBcSzYVEc2HQXRyvFnCshyNa3i5wOx5MSDdfz/B7of1TJJoLiebCyWijilVSqjCEYxWVaZcrwe2036BlKg23Tp60RHMh0VxINBfG0wnaBgEc29OedLGN8YsD+i7W9TbcOnnSEs2FRHPhZLRRYYGUqgY+H79gD1vo/V6X3t6vcOvkSUs0FxLNhURzYbwM+E3wfPz+5KSPfaVGKpXNw6mTJy3RXEg0F05GvwHexSXlg5l5ygAAAABJRU5ErkJggg==\" id=\"image4a11b6a921\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"173.613929\" y=\"-197.90779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_57\">\n    <g id=\"xtick_57\">\n     <g id=\"line2d_113\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"176.271071\" y=\"230.30779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_58\">\n     <g id=\"line2d_114\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"202.8425\" y=\"230.30779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_58\">\n    <g id=\"ytick_57\">\n     <g id=\"line2d_115\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"173.613929\" y=\"201.079219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_58\">\n     <g id=\"line2d_116\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"173.613929\" y=\"227.650647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_143\">\n    <path d=\"M 173.613929 230.30779 \nL 173.613929 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_144\">\n    <path d=\"M 205.499643 230.30779 \nL 205.499643 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_145\">\n    <path d=\"M 173.613929 230.30779 \nL 205.499643 230.30779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_146\">\n    <path d=\"M 173.613929 198.422076 \nL 205.499643 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_30\">\n   <g id=\"patch_147\">\n    <path d=\"M 211.876786 230.30779 \nL 243.7625 230.30779 \nL 243.7625 198.422076 \nL 211.876786 198.422076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#pdbe48585c4)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABEElEQVR4nO2YOwrCQBRF38TxUwhBCIiKhVhoIVjYCS7FBYjLcQ+uQnAFFoJgo0ERP52NhYjOWFjfJ1g8HHinPdzkMEUYYp7zqSdA1Owi9aFUwe75YKcmV4DutZyz24i1f4pGS6HRUmi0FJYOKbaNDr++XbGLMuzU7VbQjftDdhvkSWu0FBotRZDRlnJ5bLP4+khE5E4b6Eyc/NpEk9GA9UGetEZLodFSaLQUxl1S+AvBLWb8uNXDMmPZrd+v8XOTKrsN8qQ1WgqNliLIaOPOW/jJ+/bn0x3x1ZS8419cKmNps+w2yJPWaCk0WgqNlsL66wVKk9TYsSnGWD7u7Daqt6Hz7sVvWfunaLQUGi1FkNFvMjkxO7ULEscAAAAASUVORK5CYII=\" id=\"image2f18dd9149\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"211.876786\" y=\"-197.90779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_59\">\n    <g id=\"xtick_59\">\n     <g id=\"line2d_117\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"214.533929\" y=\"230.30779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_60\">\n     <g id=\"line2d_118\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"241.105357\" y=\"230.30779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_60\">\n    <g id=\"ytick_59\">\n     <g id=\"line2d_119\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"211.876786\" y=\"201.079219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_60\">\n     <g id=\"line2d_120\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"211.876786\" y=\"227.650647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_148\">\n    <path d=\"M 211.876786 230.30779 \nL 211.876786 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_149\">\n    <path d=\"M 243.7625 230.30779 \nL 243.7625 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_150\">\n    <path d=\"M 211.876786 230.30779 \nL 243.7625 230.30779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_151\">\n    <path d=\"M 211.876786 198.422076 \nL 243.7625 198.422076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_31\">\n   <g id=\"patch_152\">\n    <path d=\"M 20.5625 277.82779 \nL 52.448214 277.82779 \nL 52.448214 245.942076 \nL 20.5625 245.942076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p751b9ae30f)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABEklEQVR4nO2YMWoCURRF3x9+JDiTyBTBSklAUk4r2UBwEemzFRcgWUrcwTSWCtOEoBhQBBmIg4KCPyu4t3zkwzvt4fIPrxAZ9y5ZEMDH4hMpERFxdzl013LKt8ULdOHni24Tav8pFq2FRWth0Vr4yfgNyuSpoONQb7HsD/jLtyl2zS+dRnlpi9bCorWIMtq7LIPyulvRcZiXWK6/+cujR+xS3CQS6aUtWguL1sKitfBy08L21PB1dg9VOBzo1F3OWNZ7uo3y0hathUVrEWW0lw7+8uk6D3Ts8i50offMXz7in1M3fKXTKC9t0VpYtBYWrYWX3QbKcD7RcahmWLb5ZwD6l3hZ0WmUl7ZoLSxaiyij/wAYDC5kKMvlDgAAAABJRU5ErkJggg==\" id=\"imagee369fc8806\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"20.5625\" y=\"-245.42779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_61\">\n    <g id=\"xtick_61\">\n     <g id=\"line2d_121\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"23.219643\" y=\"277.82779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_11\">\n      <!-- 0 -->\n      <g transform=\"translate(20.038393 292.426228) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_62\">\n     <g id=\"line2d_122\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"49.791071\" y=\"277.82779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_12\">\n      <!-- 5 -->\n      <g transform=\"translate(46.609821 292.426228) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_62\">\n    <g id=\"ytick_61\">\n     <g id=\"line2d_123\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"20.5625\" y=\"248.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_13\">\n      <!-- 0 -->\n      <g transform=\"translate(7.2 252.398438) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_62\">\n     <g id=\"line2d_124\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"20.5625\" y=\"275.170647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_14\">\n      <!-- 5 -->\n      <g transform=\"translate(7.2 278.969866) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_153\">\n    <path d=\"M 20.5625 277.82779 \nL 20.5625 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_154\">\n    <path d=\"M 52.448214 277.82779 \nL 52.448214 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_155\">\n    <path d=\"M 20.5625 277.82779 \nL 52.448214 277.82779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_156\">\n    <path d=\"M 20.5625 245.942076 \nL 52.448214 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_32\">\n   <g id=\"patch_157\">\n    <path d=\"M 58.825357 277.82779 \nL 90.711071 277.82779 \nL 90.711071 245.942076 \nL 58.825357 245.942076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#pf600e04a76)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABFElEQVR4nO2WPWpCQRRG74wv2Cn+BAymEIJFkBQSbbKSbCGdG7HMHoSUSWNnZZPGzkAglaRIRERSaIIzWcH32V1m4J7248w7vGIYF1ZvUQAPlwM0iYjI48sYbu56SN24fMVj7Zy6nq6JYtFaWLQWFq2FO34s4D0dpxNu927xwZU6d8MRu802VbP80xathUVrkWV0Ib97OPr7EZXD7AmPFx3qxt0Gbq5qT9M0sGgtLFqLLKML9kSUszK3Gy04xe8VVf1VH49/B+7SNVEsWguL1iLL6II+EU/IvouvLVdpUjfu1nAL82f+XZ6VJhathUVrYdFaFLL9wuv+54SOb/Lw+c5VX8Kn3txxlZ+cJhathUVrkWX0Pw2UMdEsmJOjAAAAAElFTkSuQmCC\" id=\"image4d953bc800\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"58.825357\" y=\"-245.42779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_63\">\n    <g id=\"xtick_63\">\n     <g id=\"line2d_125\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"61.4825\" y=\"277.82779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_15\">\n      <!-- 0 -->\n      <g transform=\"translate(58.30125 292.426228) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_64\">\n     <g id=\"line2d_126\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"88.053929\" y=\"277.82779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_16\">\n      <!-- 5 -->\n      <g transform=\"translate(84.872679 292.426228) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_64\">\n    <g id=\"ytick_63\">\n     <g id=\"line2d_127\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"58.825357\" y=\"248.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_64\">\n     <g id=\"line2d_128\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"58.825357\" y=\"275.170647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_158\">\n    <path d=\"M 58.825357 277.82779 \nL 58.825357 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_159\">\n    <path d=\"M 90.711071 277.82779 \nL 90.711071 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_160\">\n    <path d=\"M 58.825357 277.82779 \nL 90.711071 277.82779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_161\">\n    <path d=\"M 58.825357 245.942076 \nL 90.711071 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_33\">\n   <g id=\"patch_162\">\n    <path d=\"M 97.088214 277.82779 \nL 128.973929 277.82779 \nL 128.973929 245.942076 \nL 97.088214 245.942076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#pf032d206ec)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABFElEQVR4nO2WPWpCQRRG77w3EU0gFpIm4k9lQgohkHQpUkkKC/dha+FC1JVYW6ZzA1bpAkFRTAISzYwr+D67iwP3tIdv5jDF47nw9REFEN6nSImISL83hG486dOt3NagcvUWnWb85PPEorWwaC0sWgsvhz2U2eMrHY8Gb1iGf7p17RfsKlW6TfKlLVoLi9YiyWgfFnMos7snOnadLnbNB77Ncyx3v3Sb5EtbtBYWrYVFa+GleAVlPPW9vH/GslDiN+ceO+f4vfzk88SitbBoLZKM9rL6hDLGQMeuQX4/L6/5zdslvvd7TadJvrRFa2HRWli0Fl72f9heFOjYlW+giz8bfjP7JZ7P6DTJl7ZoLSxaiySjj507KxfX6E1eAAAAAElFTkSuQmCC\" id=\"image8f676f9474\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"97.088214\" y=\"-245.42779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_65\">\n    <g id=\"xtick_65\">\n     <g id=\"line2d_129\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"99.745357\" y=\"277.82779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_17\">\n      <!-- 0 -->\n      <g transform=\"translate(96.564107 292.426228) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_66\">\n     <g id=\"line2d_130\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"126.316786\" y=\"277.82779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_18\">\n      <!-- 5 -->\n      <g transform=\"translate(123.135536 292.426228) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_66\">\n    <g id=\"ytick_65\">\n     <g id=\"line2d_131\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"97.088214\" y=\"248.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_66\">\n     <g id=\"line2d_132\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"97.088214\" y=\"275.170647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_163\">\n    <path d=\"M 97.088214 277.82779 \nL 97.088214 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_164\">\n    <path d=\"M 128.973929 277.82779 \nL 128.973929 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_165\">\n    <path d=\"M 97.088214 277.82779 \nL 128.973929 277.82779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_166\">\n    <path d=\"M 97.088214 245.942076 \nL 128.973929 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_34\">\n   <g id=\"patch_167\">\n    <path d=\"M 135.351071 277.82779 \nL 167.236786 277.82779 \nL 167.236786 245.942076 \nL 135.351071 245.942076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p8da7c071d6)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABIUlEQVR4nO2WsUqCURiGv/N3CofAEilQaFCkod3JJjehIbqGtiZvoLXFJfAC3LwAV3GpqSESQhByEEGICBot7fMK3nf88MD3rA/v/z+c4XDC+rGtgqicQyUiok8j6O4eBnTb7d9j+bui24zaHcWjrfBoKzzaiigZ6T7M03Fo3UB323um26xxBZ1+zvmW2h3Fo63waCuSjI5Su4AylCp0rNNX6ArHOf7nzR/+ruLXskiiJ+3RVni0FR5tRZTlAkrNF/m6cAJVqX7Gt0en2L0M6TTJk/ZoKzzaiiSjo+TIE/Lni473GtfQZZ063epsjOX+Ad0medIebYVHW+HRVkR9x/dluGzSsX4vofv/eKPbUK7i707IHS6JnrRHW+HRViQZvQVU5DB0GAU55wAAAABJRU5ErkJggg==\" id=\"image49a07783af\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"135.351071\" y=\"-245.42779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_67\">\n    <g id=\"xtick_67\">\n     <g id=\"line2d_133\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"138.008214\" y=\"277.82779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_19\">\n      <!-- 0 -->\n      <g transform=\"translate(134.826964 292.426228) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_68\">\n     <g id=\"line2d_134\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"164.579643\" y=\"277.82779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_20\">\n      <!-- 5 -->\n      <g transform=\"translate(161.398393 292.426228) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_68\">\n    <g id=\"ytick_67\">\n     <g id=\"line2d_135\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"135.351071\" y=\"248.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_68\">\n     <g id=\"line2d_136\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"135.351071\" y=\"275.170647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_168\">\n    <path d=\"M 135.351071 277.82779 \nL 135.351071 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_169\">\n    <path d=\"M 167.236786 277.82779 \nL 167.236786 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_170\">\n    <path d=\"M 135.351071 277.82779 \nL 167.236786 277.82779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_171\">\n    <path d=\"M 135.351071 245.942076 \nL 167.236786 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_35\">\n   <g id=\"patch_172\">\n    <path d=\"M 173.613929 277.82779 \nL 205.499643 277.82779 \nL 205.499643 245.942076 \nL 173.613929 245.942076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#pded4c849c7)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABGUlEQVR4nO2YvUrDUBiGzwlZWzXUH2oFK0IHnVw6OfVG3HstHXsn2UrpLNWxZFAIKIVA62ALWmqS4xW8b6HD1xz4nvXhPXk4QwixxdvUGYB7mSC1m9YN1f3HJ+iG8YBug316Do1GS6HRUmi0FDZ/juF72qQJX69X2EUNvq0dQRU89OjUy5vWaCk0Wgovo235MYOvvHL+TsfBVYecbOm2nI6wrJ/w51JbUTRaCo2WQqOlCN33Atv8j47ddgNdcH5Nt/aui8/9yujWy5vWaCk0Wgovo0P3+wOlvbylY5el0BW7Pmvb9/jccMm31FYUjZZCo6XQaCls+ZngXwivY74+bWJHPluNMcYen2EXXdCtlzet0VJotBReRv8DM0M5oAR/FpMAAAAASUVORK5CYII=\" id=\"image32f6ccd98f\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"173.613929\" y=\"-245.42779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_69\">\n    <g id=\"xtick_69\">\n     <g id=\"line2d_137\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"176.271071\" y=\"277.82779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_21\">\n      <!-- 0 -->\n      <g transform=\"translate(173.089821 292.426228) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_70\">\n     <g id=\"line2d_138\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"202.8425\" y=\"277.82779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_22\">\n      <!-- 5 -->\n      <g transform=\"translate(199.66125 292.426228) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_70\">\n    <g id=\"ytick_69\">\n     <g id=\"line2d_139\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"173.613929\" y=\"248.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_70\">\n     <g id=\"line2d_140\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"173.613929\" y=\"275.170647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_173\">\n    <path d=\"M 173.613929 277.82779 \nL 173.613929 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_174\">\n    <path d=\"M 205.499643 277.82779 \nL 205.499643 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_175\">\n    <path d=\"M 173.613929 277.82779 \nL 205.499643 277.82779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_176\">\n    <path d=\"M 173.613929 245.942076 \nL 205.499643 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_36\">\n   <g id=\"patch_177\">\n    <path d=\"M 211.876786 277.82779 \nL 243.7625 277.82779 \nL 243.7625 245.942076 \nL 211.876786 245.942076 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g clip-path=\"url(#p6cd1d0f08d)\">\n    <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAC0AAAAtCAYAAAA6GuKaAAABDklEQVR4nO2YsUoDQRRF34yDLIZlFYKBFAHBUvILtvkXC78rpZ+QSu0NsQliiqwSVgQjmmWZ8QvutXvswDvt4c6enWKLdd1ingTRHqASEZHyDLvfb74l/mZ2S6een9xPLFoLi9bCorVw3f0d/E479h0WkfT+ig8eX/IHn55DF1cPdJvlTVu0FhatRZbRQZo3KP30mo5jMcBy/0m3qV5D5y6u6DbLm7ZoLSxaC4vWIkgIUKafLzp21RDLQUW3cfOMZbOl2yxv2qK1sGgtsowO4o+wZU5E4vIRy5OSbv1ogs99eeJbanuKRWth0VpYtBZBPnbYun/eqWuxa2o6TeyP7HFBt1netEVrYdFaZBn9B+/2MKv2WalmAAAAAElFTkSuQmCC\" id=\"image21a78426ac\" transform=\"scale(1 -1) translate(0 -32.4)\" x=\"211.876786\" y=\"-245.42779\" width=\"32.4\" height=\"32.4\"/>\n   </g>\n   <g id=\"matplotlib.axis_71\">\n    <g id=\"xtick_71\">\n     <g id=\"line2d_141\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"214.533929\" y=\"277.82779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_23\">\n      <!-- 0 -->\n      <g transform=\"translate(211.352679 292.426228) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_72\">\n     <g id=\"line2d_142\">\n      <g>\n       <use xlink:href=\"#mb9a5b3b75b\" x=\"241.105357\" y=\"277.82779\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_24\">\n      <!-- 5 -->\n      <g transform=\"translate(237.924107 292.426228) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_72\">\n    <g id=\"ytick_71\">\n     <g id=\"line2d_143\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"211.876786\" y=\"248.599219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_72\">\n     <g id=\"line2d_144\">\n      <g>\n       <use xlink:href=\"#m4c6e37c3be\" x=\"211.876786\" y=\"275.170647\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"patch_178\">\n    <path d=\"M 211.876786 277.82779 \nL 211.876786 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_179\">\n    <path d=\"M 243.7625 277.82779 \nL 243.7625 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_180\">\n    <path d=\"M 211.876786 277.82779 \nL 243.7625 277.82779 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_181\">\n    <path d=\"M 211.876786 245.942076 \nL 243.7625 245.942076 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n  <g id=\"axes_37\">\n   <g id=\"patch_182\">\n    <path d=\"M 257.7125 226.244933 \nL 266.0285 226.244933 \nL 266.0285 59.924933 \nL 257.7125 59.924933 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <image xlink:href=\"data:image/png;base64,\niVBORw0KGgoAAAANSUhEUgAAAAsAAADnCAYAAADIKqzfAAABNElEQVR4nO2ZMQ7DIBAEL5L//9Y06eBInY4pVhocu16thl2OEPu1Pu9Vm89Va1tbVxURd6ecEXNugfvaLAYSk1KI87Is0IFhmZRtrSc6CUZPIk6NVW5gkTObbpZGbIGoQYTR5zUYdEZp9NgXs113YhoS5okmxbFAyZGbq5v9AElKcWBIGnzG6kfsKAUtMJhG7uLqYJbkLEkD/ZWWMAePL8euuz0zm8HY2x52mKNJyZWSSmNNxe3r9g2CS0FdRUr5g+gcY+U4vtBGYs7k9azEebKcCUbO2ZFzOxpsNoMpjBOjS5ZCnEmDQEvT2Dem4hgGiJnuOuLMmFGD5IsYLAVhAHETZpgGEkswkDh25DIM4mwZK8khg5xTW3Q6ME6MLsc8LMwEgzkTMXFmty/iPGIXV0l0MeYvxtbRHLYHoMMAAAAASUVORK5CYII=\" id=\"imageecaf1f03be\" transform=\"scale(1 -1) translate(0 -166.32)\" x=\"257.76\" y=\"-59.76\" width=\"7.92\" height=\"166.32\"/>\n   <g id=\"matplotlib.axis_73\"/>\n   <g id=\"matplotlib.axis_74\">\n    <g id=\"ytick_73\">\n     <g id=\"line2d_145\">\n      <defs>\n       <path id=\"m0df215f179\" d=\"M 0 0 \nL 3.5 0 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#m0df215f179\" x=\"266.0285\" y=\"219.909852\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_25\">\n      <!-- −0.2 -->\n      <g transform=\"translate(273.0285 223.709071) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-2212\" d=\"M 678 2272 \nL 4684 2272 \nL 4684 1741 \nL 678 1741 \nL 678 2272 \nz\n\" transform=\"scale(0.015625)\"/>\n        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \nL 1344 794 \nL 1344 0 \nL 684 0 \nL 684 794 \nz\n\" transform=\"scale(0.015625)\"/>\n        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \nL 3431 531 \nL 3431 0 \nL 469 0 \nL 469 531 \nQ 828 903 1448 1529 \nQ 2069 2156 2228 2338 \nQ 2531 2678 2651 2914 \nQ 2772 3150 2772 3378 \nQ 2772 3750 2511 3984 \nQ 2250 4219 1831 4219 \nQ 1534 4219 1204 4116 \nQ 875 4013 500 3803 \nL 500 4441 \nQ 881 4594 1212 4672 \nQ 1544 4750 1819 4750 \nQ 2544 4750 2975 4387 \nQ 3406 4025 3406 3419 \nQ 3406 3131 3298 2873 \nQ 3191 2616 2906 2266 \nQ 2828 2175 2409 1742 \nQ 1991 1309 1228 531 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-2212\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(83.789062 0)\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(147.412109 0)\"/>\n       <use xlink:href=\"#DejaVuSans-32\" transform=\"translate(179.199219 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_74\">\n     <g id=\"line2d_146\">\n      <g>\n       <use xlink:href=\"#m0df215f179\" x=\"266.0285\" y=\"193.245699\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_26\">\n      <!-- 0.0 -->\n      <g transform=\"translate(273.0285 197.044918) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_75\">\n     <g id=\"line2d_147\">\n      <g>\n       <use xlink:href=\"#m0df215f179\" x=\"266.0285\" y=\"166.581546\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_27\">\n      <!-- 0.2 -->\n      <g transform=\"translate(273.0285 170.380765) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-32\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_76\">\n     <g id=\"line2d_148\">\n      <g>\n       <use xlink:href=\"#m0df215f179\" x=\"266.0285\" y=\"139.917393\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_28\">\n      <!-- 0.4 -->\n      <g transform=\"translate(273.0285 143.716611) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \nL 825 1625 \nL 2419 1625 \nL 2419 4116 \nz\nM 2253 4666 \nL 3047 4666 \nL 3047 1625 \nL 3713 1625 \nL 3713 1100 \nL 3047 1100 \nL 3047 0 \nL 2419 0 \nL 2419 1100 \nL 313 1100 \nL 313 1709 \nL 2253 4666 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-30\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-34\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_77\">\n     <g id=\"line2d_149\">\n      <g>\n       <use xlink:href=\"#m0df215f179\" x=\"266.0285\" y=\"113.253239\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_29\">\n      <!-- 0.6 -->\n      <g transform=\"translate(273.0285 117.052458) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \nQ 1688 2584 1439 2293 \nQ 1191 2003 1191 1497 \nQ 1191 994 1439 701 \nQ 1688 409 2113 409 \nQ 2538 409 2786 701 \nQ 3034 994 3034 1497 \nQ 3034 2003 2786 2293 \nQ 2538 2584 2113 2584 \nz\nM 3366 4563 \nL 3366 3988 \nQ 3128 4100 2886 4159 \nQ 2644 4219 2406 4219 \nQ 1781 4219 1451 3797 \nQ 1122 3375 1075 2522 \nQ 1259 2794 1537 2939 \nQ 1816 3084 2150 3084 \nQ 2853 3084 3261 2657 \nQ 3669 2231 3669 1497 \nQ 3669 778 3244 343 \nQ 2819 -91 2113 -91 \nQ 1303 -91 875 529 \nQ 447 1150 447 2328 \nQ 447 3434 972 4092 \nQ 1497 4750 2381 4750 \nQ 2619 4750 2861 4703 \nQ 3103 4656 3366 4563 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-30\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-36\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_78\">\n     <g id=\"line2d_150\">\n      <g>\n       <use xlink:href=\"#m0df215f179\" x=\"266.0285\" y=\"86.589086\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_30\">\n      <!-- 0.8 -->\n      <g transform=\"translate(273.0285 90.388305) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \nQ 1584 2216 1326 1975 \nQ 1069 1734 1069 1313 \nQ 1069 891 1326 650 \nQ 1584 409 2034 409 \nQ 2484 409 2743 651 \nQ 3003 894 3003 1313 \nQ 3003 1734 2745 1975 \nQ 2488 2216 2034 2216 \nz\nM 1403 2484 \nQ 997 2584 770 2862 \nQ 544 3141 544 3541 \nQ 544 4100 942 4425 \nQ 1341 4750 2034 4750 \nQ 2731 4750 3128 4425 \nQ 3525 4100 3525 3541 \nQ 3525 3141 3298 2862 \nQ 3072 2584 2669 2484 \nQ 3125 2378 3379 2068 \nQ 3634 1759 3634 1313 \nQ 3634 634 3220 271 \nQ 2806 -91 2034 -91 \nQ 1263 -91 848 271 \nQ 434 634 434 1313 \nQ 434 1759 690 2068 \nQ 947 2378 1403 2484 \nz\nM 1172 3481 \nQ 1172 3119 1398 2916 \nQ 1625 2713 2034 2713 \nQ 2441 2713 2670 2916 \nQ 2900 3119 2900 3481 \nQ 2900 3844 2670 4047 \nQ 2441 4250 2034 4250 \nQ 1625 4250 1398 4047 \nQ 1172 3844 1172 3481 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-30\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-38\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_79\">\n     <g id=\"line2d_151\">\n      <g>\n       <use xlink:href=\"#m0df215f179\" x=\"266.0285\" y=\"59.924933\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_31\">\n      <!-- 1.0 -->\n      <g transform=\"translate(273.0285 63.724152) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-31\" d=\"M 794 531 \nL 1825 531 \nL 1825 4091 \nL 703 3866 \nL 703 4441 \nL 1819 4666 \nL 2450 4666 \nL 2450 531 \nL 3481 531 \nL 3481 0 \nL 794 0 \nL 794 531 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"LineCollection_1\"/>\n   <g id=\"patch_183\">\n    <path d=\"M 257.7125 226.244933 \nL 261.8705 226.244933 \nL 266.0285 226.244933 \nL 266.0285 59.924933 \nL 261.8705 59.924933 \nL 257.7125 59.924933 \nL 257.7125 226.244933 \nz\n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n  </g>\n </g>\n <defs>\n  <clipPath id=\"p575ba2994c\">\n   <rect x=\"20.5625\" y=\"8.342076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p957ad9238d\">\n   <rect x=\"58.825357\" y=\"8.342076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p13684a52de\">\n   <rect x=\"97.088214\" y=\"8.342076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p1abe17972a\">\n   <rect x=\"135.351071\" y=\"8.342076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p99f9d13d26\">\n   <rect x=\"173.613929\" y=\"8.342076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"pbbf4777ddc\">\n   <rect x=\"211.876786\" y=\"8.342076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"pdd02c00f72\">\n   <rect x=\"20.5625\" y=\"55.862076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p6af1523c07\">\n   <rect x=\"58.825357\" y=\"55.862076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"pae6153d92f\">\n   <rect x=\"97.088214\" y=\"55.862076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p2cd4f0bb44\">\n   <rect x=\"135.351071\" y=\"55.862076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p0d754d2a73\">\n   <rect x=\"173.613929\" y=\"55.862076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"pd7f4098616\">\n   <rect x=\"211.876786\" y=\"55.862076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p37a9cd4af8\">\n   <rect x=\"20.5625\" y=\"103.382076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p94312a9f28\">\n   <rect x=\"58.825357\" y=\"103.382076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p107ab9e58e\">\n   <rect x=\"97.088214\" y=\"103.382076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"pff69955f02\">\n   <rect x=\"135.351071\" y=\"103.382076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p6bc373f8d4\">\n   <rect x=\"173.613929\" y=\"103.382076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p79269eded6\">\n   <rect x=\"211.876786\" y=\"103.382076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p33c571d1ce\">\n   <rect x=\"20.5625\" y=\"150.902076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p57cb5492df\">\n   <rect x=\"58.825357\" y=\"150.902076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"pa4ed43525a\">\n   <rect x=\"97.088214\" y=\"150.902076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p4f84ea0f34\">\n   <rect x=\"135.351071\" y=\"150.902076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p97b08c5e75\">\n   <rect x=\"173.613929\" y=\"150.902076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"pb1577dfa55\">\n   <rect x=\"211.876786\" y=\"150.902076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p614d8b9236\">\n   <rect x=\"20.5625\" y=\"198.422076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p4402941511\">\n   <rect x=\"58.825357\" y=\"198.422076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p829ab20925\">\n   <rect x=\"97.088214\" y=\"198.422076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p4817fb5466\">\n   <rect x=\"135.351071\" y=\"198.422076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p969553b572\">\n   <rect x=\"173.613929\" y=\"198.422076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"pdbe48585c4\">\n   <rect x=\"211.876786\" y=\"198.422076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p751b9ae30f\">\n   <rect x=\"20.5625\" y=\"245.942076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"pf600e04a76\">\n   <rect x=\"58.825357\" y=\"245.942076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"pf032d206ec\">\n   <rect x=\"97.088214\" y=\"245.942076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p8da7c071d6\">\n   <rect x=\"135.351071\" y=\"245.942076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"pded4c849c7\">\n   <rect x=\"173.613929\" y=\"245.942076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n  <clipPath id=\"p6cd1d0f08d\">\n   <rect x=\"211.876786\" y=\"245.942076\" width=\"31.885714\" height=\"31.885714\"/>\n  </clipPath>\n </defs>\n</svg>\n"
     },
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=vision-transformer-md-9 fw=all -->\n",
    "\n",
    "Each map's darkest cell is its own position, which is mere self-similarity;\n",
    "the question is what surrounds it. The answer, after ten epochs, is: the\n",
    "first faint traces of the grid and no more. The printed statistic makes it\n",
    "precise: averaged over positions, immediate grid neighbors score slightly\n",
    "positive cosine similarity while distant positions score slightly negative,\n",
    "and several maps show a diffuse halo around their own cell or a weak band\n",
    "along their own column — but the values are small and the pattern is easy\n",
    "to miss without the summary statistics. Compare this with the crisp\n",
    "row-and-column structure that emerges in fully trained ViTs, and the state\n",
    "of our model is clear: the geometry it was never told is beginning to show\n",
    "in its embeddings, and at this scale of data and training it has not\n",
    "gotten far. Keep that in mind as we turn to an architecture that does not\n",
    "need to learn the geometry at all.\n",
    "\n",
    "### A Convolutional Baseline at the Same Budget\n",
    "\n",
    "The ViT above has about 6.5 million parameters. To make the inductive-bias\n",
    "comparison fair we build a CNN with the same budget, from the `Residual`\n",
    "blocks of that section: the familiar $7 \\times 7$ stem, then seven\n",
    "residual blocks whose channel widths are chosen so that the total lands at\n",
    "the ViT's parameter count. It trains on the identical data pipeline for the\n",
    "identical ten epochs, with the learning rate that that section\n",
    "established for this family."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "vision-transformer-a-convolutional-baseline-at-the-same-budget-1"
   },
   "source": [
    "class CompactResNet(d2l.Classifier):\n",
    "    \"\"\"A ResNet-style CNN parameter-matched to the ViT above.\"\"\"\n",
    "    def __init__(self, lr=0.01, num_classes=10):\n",
    "        super().__init__()\n",
    "        self.save_hyperparameters()\n",
    "        self.net = nn.Sequential(\n",
    "            nn.LazyConv2d(64, kernel_size=7, stride=2, padding=3),\n",
    "            nn.LazyBatchNorm2d(), nn.ReLU(),\n",
    "            nn.MaxPool2d(kernel_size=3, stride=2, padding=1),\n",
    "            d2l.Residual(64), d2l.Residual(64),\n",
    "            d2l.Residual(128, strides=2), d2l.Residual(128),\n",
    "            d2l.Residual(256, strides=2), d2l.Residual(256),\n",
    "            d2l.Residual(512, strides=2),\n",
    "            nn.AdaptiveAvgPool2d((1, 1)), nn.Flatten(),\n",
    "            nn.LazyLinear(num_classes))\n",
    "\n",
    "    def forward(self, X):\n",
    "        return self.net(X)"
   ],
   "execution_count": 10,
   "outputs": []
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "vision-transformer-a-convolutional-baseline-at-the-same-budget-2"
   },
   "source": [
    "cnn = CompactResNet()\n",
    "trainer = d2l.Trainer(max_epochs=10, num_gpus=1)\n",
    "trainer.fit(cnn, data)\n",
    "cnn_params = sum(p.numel() for p in cnn.parameters() if p.requires_grad)\n",
    "cnn_acc = float(cnn.board.data['val_acc'][-1].y)\n",
    "print(f'CNN: {cnn_params/1e6:.1f}M parameters, '\n",
    "      f'validation accuracy {cnn_acc:.2f}')\n",
    "print(f'ViT: {vit_params/1e6:.1f}M parameters, '\n",
    "      f'validation accuracy {vit_acc:.2f}')"
   ],
   "execution_count": 11,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "CNN: 6.5M parameters, validation accuracy 0.91\nViT: 6.5M parameters, validation accuracy 0.86\n"
     ]
    },
    {
     "output_type": "display_data",
     "data": {
      "image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n <metadata>\n  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n   <cc:Work>\n    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n    <dc:date>2026-07-19T14:09:15.618721</dc:date>\n    <dc:format>image/svg+xml</dc:format>\n    <dc:creator>\n     <cc:Agent>\n      <dc:title>Matplotlib v3.10.8, https://matplotlib.org/</dc:title>\n     </cc:Agent>\n    </dc:creator>\n   </cc:Work>\n  </rdf:RDF>\n </metadata>\n <defs>\n  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n </defs>\n <g id=\"figure_1\">\n  <g id=\"patch_1\">\n   <path d=\"M 0 183.35625 \nL 238.965625 183.35625 \nL 238.965625 0 \nL 0 0 \nz\n\" style=\"fill: #ffffff\"/>\n  </g>\n  <g id=\"axes_1\">\n   <g id=\"patch_2\">\n    <path d=\"M 30.103125 145.8 \nL 225.403125 145.8 \nL 225.403125 7.2 \nL 30.103125 7.2 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g id=\"matplotlib.axis_1\">\n    <g id=\"xtick_1\">\n     <g id=\"line2d_1\">\n      <defs>\n       <path id=\"m90cf23a0e0\" d=\"M 0 0 \nL 0 3.5 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#m90cf23a0e0\" x=\"30.103125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_1\">\n      <!-- 0 -->\n      <g transform=\"translate(26.921875 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \nQ 1547 4250 1301 3770 \nQ 1056 3291 1056 2328 \nQ 1056 1369 1301 889 \nQ 1547 409 2034 409 \nQ 2525 409 2770 889 \nQ 3016 1369 3016 2328 \nQ 3016 3291 2770 3770 \nQ 2525 4250 2034 4250 \nz\nM 2034 4750 \nQ 2819 4750 3233 4129 \nQ 3647 3509 3647 2328 \nQ 3647 1150 3233 529 \nQ 2819 -91 2034 -91 \nQ 1250 -91 836 529 \nQ 422 1150 422 2328 \nQ 422 3509 836 4129 \nQ 1250 4750 2034 4750 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-30\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_2\">\n     <g id=\"line2d_2\">\n      <g>\n       <use xlink:href=\"#m90cf23a0e0\" x=\"69.163125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_2\">\n      <!-- 2 -->\n      <g transform=\"translate(65.981875 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \nL 3431 531 \nL 3431 0 \nL 469 0 \nL 469 531 \nQ 828 903 1448 1529 \nQ 2069 2156 2228 2338 \nQ 2531 2678 2651 2914 \nQ 2772 3150 2772 3378 \nQ 2772 3750 2511 3984 \nQ 2250 4219 1831 4219 \nQ 1534 4219 1204 4116 \nQ 875 4013 500 3803 \nL 500 4441 \nQ 881 4594 1212 4672 \nQ 1544 4750 1819 4750 \nQ 2544 4750 2975 4387 \nQ 3406 4025 3406 3419 \nQ 3406 3131 3298 2873 \nQ 3191 2616 2906 2266 \nQ 2828 2175 2409 1742 \nQ 1991 1309 1228 531 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-32\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_3\">\n     <g id=\"line2d_3\">\n      <g>\n       <use xlink:href=\"#m90cf23a0e0\" x=\"108.223125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_3\">\n      <!-- 4 -->\n      <g transform=\"translate(105.041875 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \nL 825 1625 \nL 2419 1625 \nL 2419 4116 \nz\nM 2253 4666 \nL 3047 4666 \nL 3047 1625 \nL 3713 1625 \nL 3713 1100 \nL 3047 1100 \nL 3047 0 \nL 2419 0 \nL 2419 1100 \nL 313 1100 \nL 313 1709 \nL 2253 4666 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-34\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_4\">\n     <g id=\"line2d_4\">\n      <g>\n       <use xlink:href=\"#m90cf23a0e0\" x=\"147.283125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_4\">\n      <!-- 6 -->\n      <g transform=\"translate(144.101875 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \nQ 1688 2584 1439 2293 \nQ 1191 2003 1191 1497 \nQ 1191 994 1439 701 \nQ 1688 409 2113 409 \nQ 2538 409 2786 701 \nQ 3034 994 3034 1497 \nQ 3034 2003 2786 2293 \nQ 2538 2584 2113 2584 \nz\nM 3366 4563 \nL 3366 3988 \nQ 3128 4100 2886 4159 \nQ 2644 4219 2406 4219 \nQ 1781 4219 1451 3797 \nQ 1122 3375 1075 2522 \nQ 1259 2794 1537 2939 \nQ 1816 3084 2150 3084 \nQ 2853 3084 3261 2657 \nQ 3669 2231 3669 1497 \nQ 3669 778 3244 343 \nQ 2819 -91 2113 -91 \nQ 1303 -91 875 529 \nQ 447 1150 447 2328 \nQ 447 3434 972 4092 \nQ 1497 4750 2381 4750 \nQ 2619 4750 2861 4703 \nQ 3103 4656 3366 4563 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-36\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_5\">\n     <g id=\"line2d_5\">\n      <g>\n       <use xlink:href=\"#m90cf23a0e0\" x=\"186.343125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_5\">\n      <!-- 8 -->\n      <g transform=\"translate(183.161875 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \nQ 1584 2216 1326 1975 \nQ 1069 1734 1069 1313 \nQ 1069 891 1326 650 \nQ 1584 409 2034 409 \nQ 2484 409 2743 651 \nQ 3003 894 3003 1313 \nQ 3003 1734 2745 1975 \nQ 2488 2216 2034 2216 \nz\nM 1403 2484 \nQ 997 2584 770 2862 \nQ 544 3141 544 3541 \nQ 544 4100 942 4425 \nQ 1341 4750 2034 4750 \nQ 2731 4750 3128 4425 \nQ 3525 4100 3525 3541 \nQ 3525 3141 3298 2862 \nQ 3072 2584 2669 2484 \nQ 3125 2378 3379 2068 \nQ 3634 1759 3634 1313 \nQ 3634 634 3220 271 \nQ 2806 -91 2034 -91 \nQ 1263 -91 848 271 \nQ 434 634 434 1313 \nQ 434 1759 690 2068 \nQ 947 2378 1403 2484 \nz\nM 1172 3481 \nQ 1172 3119 1398 2916 \nQ 1625 2713 2034 2713 \nQ 2441 2713 2670 2916 \nQ 2900 3119 2900 3481 \nQ 2900 3844 2670 4047 \nQ 2441 4250 2034 4250 \nQ 1625 4250 1398 4047 \nQ 1172 3844 1172 3481 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-38\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_6\">\n     <g id=\"line2d_6\">\n      <g>\n       <use xlink:href=\"#m90cf23a0e0\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_6\">\n      <!-- 10 -->\n      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-31\" d=\"M 794 531 \nL 1825 531 \nL 1825 4091 \nL 703 3866 \nL 703 4441 \nL 1819 4666 \nL 2450 4666 \nL 2450 531 \nL 3481 531 \nL 3481 0 \nL 794 0 \nL 794 531 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-30\" transform=\"translate(63.623047 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"text_7\">\n     <!-- epoch -->\n     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \nL 3597 1613 \nL 953 1613 \nQ 991 1019 1311 708 \nQ 1631 397 2203 397 \nQ 2534 397 2845 478 \nQ 3156 559 3463 722 \nL 3463 178 \nQ 3153 47 2828 -22 \nQ 2503 -91 2169 -91 \nQ 1331 -91 842 396 \nQ 353 884 353 1716 \nQ 353 2575 817 3079 \nQ 1281 3584 2069 3584 \nQ 2775 3584 3186 3129 \nQ 3597 2675 3597 1894 \nz\nM 3022 2063 \nQ 3016 2534 2758 2815 \nQ 2500 3097 2075 3097 \nQ 1594 3097 1305 2825 \nQ 1016 2553 972 2059 \nL 3022 2063 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \nL 1159 -1331 \nL 581 -1331 \nL 581 3500 \nL 1159 3500 \nL 1159 2969 \nQ 1341 3281 1617 3432 \nQ 1894 3584 2278 3584 \nQ 2916 3584 3314 3078 \nQ 3713 2572 3713 1747 \nQ 3713 922 3314 415 \nQ 2916 -91 2278 -91 \nQ 1894 -91 1617 61 \nQ 1341 213 1159 525 \nz\nM 3116 1747 \nQ 3116 2381 2855 2742 \nQ 2594 3103 2138 3103 \nQ 1681 3103 1420 2742 \nQ 1159 2381 1159 1747 \nQ 1159 1113 1420 752 \nQ 1681 391 2138 391 \nQ 2594 391 2855 752 \nQ 3116 1113 3116 1747 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \nQ 1497 3097 1228 2736 \nQ 959 2375 959 1747 \nQ 959 1119 1226 758 \nQ 1494 397 1959 397 \nQ 2419 397 2687 759 \nQ 2956 1122 2956 1747 \nQ 2956 2369 2687 2733 \nQ 2419 3097 1959 3097 \nz\nM 1959 3584 \nQ 2709 3584 3137 3096 \nQ 3566 2609 3566 1747 \nQ 3566 888 3137 398 \nQ 2709 -91 1959 -91 \nQ 1206 -91 779 398 \nQ 353 888 353 1747 \nQ 353 2609 779 3096 \nQ 1206 3584 1959 3584 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \nL 3122 2828 \nQ 2878 2963 2633 3030 \nQ 2388 3097 2138 3097 \nQ 1578 3097 1268 2742 \nQ 959 2388 959 1747 \nQ 959 1106 1268 751 \nQ 1578 397 2138 397 \nQ 2388 397 2633 464 \nQ 2878 531 3122 666 \nL 3122 134 \nQ 2881 22 2623 -34 \nQ 2366 -91 2075 -91 \nQ 1284 -91 818 406 \nQ 353 903 353 1747 \nQ 353 2603 823 3093 \nQ 1294 3584 2113 3584 \nQ 2378 3584 2631 3529 \nQ 2884 3475 3122 3366 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \nL 3513 0 \nL 2938 0 \nL 2938 2094 \nQ 2938 2591 2744 2837 \nQ 2550 3084 2163 3084 \nQ 1697 3084 1428 2787 \nQ 1159 2491 1159 1978 \nL 1159 0 \nL 581 0 \nL 581 4863 \nL 1159 4863 \nL 1159 2956 \nQ 1366 3272 1645 3428 \nQ 1925 3584 2291 3584 \nQ 2894 3584 3203 3211 \nQ 3513 2838 3513 2113 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-65\"/>\n      <use xlink:href=\"#DejaVuSans-70\" transform=\"translate(61.523438 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" transform=\"translate(125 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(186.181641 0)\"/>\n      <use xlink:href=\"#DejaVuSans-68\" transform=\"translate(241.162109 0)\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_2\">\n    <g id=\"ytick_1\">\n     <g id=\"line2d_7\">\n      <defs>\n       <path id=\"mf8b8119e04\" d=\"M 0 0 \nL -3.5 0 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#mf8b8119e04\" x=\"30.103125\" y=\"117.105055\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_8\">\n      <!-- 0.2 -->\n      <g transform=\"translate(7.2 120.904274) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \nL 1344 794 \nL 1344 0 \nL 684 0 \nL 684 794 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-30\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-32\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_2\">\n     <g id=\"line2d_8\">\n      <g>\n       <use xlink:href=\"#mf8b8119e04\" x=\"30.103125\" y=\"88.24644\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_9\">\n      <!-- 0.4 -->\n      <g transform=\"translate(7.2 92.045658) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-34\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_3\">\n     <g id=\"line2d_9\">\n      <g>\n       <use xlink:href=\"#mf8b8119e04\" x=\"30.103125\" y=\"59.387824\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_10\">\n      <!-- 0.6 -->\n      <g transform=\"translate(7.2 63.187043) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-36\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_4\">\n     <g id=\"line2d_10\">\n      <g>\n       <use xlink:href=\"#mf8b8119e04\" x=\"30.103125\" y=\"30.529209\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_11\">\n      <!-- 0.8 -->\n      <g transform=\"translate(7.2 34.328427) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-30\"/>\n       <use xlink:href=\"#DejaVuSans-2e\" transform=\"translate(63.623047 0)\"/>\n       <use xlink:href=\"#DejaVuSans-38\" transform=\"translate(95.410156 0)\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"line2d_11\">\n    <path d=\"M 34.954394 51.550736 \nL 44.698573 95.847563 \nL 54.442752 105.203626 \nL 64.186931 108.787156 \nL 73.93111 114.629059 \nL 83.675289 115.879932 \nL 93.419468 119.835464 \nL 103.163647 120.649918 \nL 112.907826 125.366481 \nL 122.652006 124.876203 \nL 132.396185 129.803648 \nL 142.140364 127.301544 \nL 151.884543 132.882659 \nL 161.628722 131.024866 \nL 171.372901 135.809521 \nL 181.11708 134.308996 \nL 190.861259 138.720823 \nL 200.605438 136.114292 \nL 210.349618 139.5 \nL 220.093797 138.470106 \n\" clip-path=\"url(#p9b8d85fff4)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n   </g>\n   <g id=\"line2d_12\">\n    <path d=\"M 49.633125 50.60075 \nL 69.163125 104.24628 \nL 88.693125 99.559932 \nL 108.223125 103.487937 \nL 127.753125 109.035711 \nL 147.283125 110.37815 \nL 166.813125 108.437893 \nL 186.343125 107.930519 \nL 205.873125 100.014791 \nL 225.403125 102.099355 \n\" clip-path=\"url(#p9b8d85fff4)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #ff7f0e; stroke-width: 1.5\"/>\n   </g>\n   <g id=\"line2d_13\">\n    <path d=\"M 49.633125 36.630842 \nL 69.163125 16.58221 \nL 88.693125 17.852194 \nL 108.223125 16.610749 \nL 127.753125 14.88414 \nL 147.283125 14.470325 \nL 166.813125 14.042241 \nL 186.343125 13.5 \nL 205.873125 15.226608 \nL 225.403125 14.527403 \n\" clip-path=\"url(#p9b8d85fff4)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #2ca02c; stroke-width: 1.5\"/>\n   </g>\n   <g id=\"patch_3\">\n    <path d=\"M 30.103125 145.8 \nL 30.103125 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_4\">\n    <path d=\"M 225.403125 145.8 \nL 225.403125 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_5\">\n    <path d=\"M 30.103125 145.8 \nL 225.403125 145.8 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_6\">\n    <path d=\"M 30.103125 7.2 \nL 225.403125 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"legend_1\">\n    <g id=\"patch_7\">\n     <path d=\"M 138.8125 100.434375 \nL 218.403125 100.434375 \nQ 220.403125 100.434375 220.403125 98.434375 \nL 220.403125 54.565625 \nQ 220.403125 52.565625 218.403125 52.565625 \nL 138.8125 52.565625 \nQ 136.8125 52.565625 136.8125 54.565625 \nL 136.8125 98.434375 \nQ 136.8125 100.434375 138.8125 100.434375 \nz\n\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n    </g>\n    <g id=\"line2d_14\">\n     <path d=\"M 140.8125 60.664063 \nL 150.8125 60.664063 \nL 160.8125 60.664063 \n\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n    </g>\n    <g id=\"text_12\">\n     <!-- train_loss -->\n     <g transform=\"translate(168.8125 64.164063) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \nL 1172 3500 \nL 2356 3500 \nL 2356 3053 \nL 1172 3053 \nL 1172 1153 \nQ 1172 725 1289 603 \nQ 1406 481 1766 481 \nL 2356 481 \nL 2356 0 \nL 1766 0 \nQ 1100 0 847 248 \nQ 594 497 594 1153 \nL 594 3053 \nL 172 3053 \nL 172 3500 \nL 594 3500 \nL 594 4494 \nL 1172 4494 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \nQ 2534 3019 2420 3045 \nQ 2306 3072 2169 3072 \nQ 1681 3072 1420 2755 \nQ 1159 2438 1159 1844 \nL 1159 0 \nL 581 0 \nL 581 3500 \nL 1159 3500 \nL 1159 2956 \nQ 1341 3275 1631 3429 \nQ 1922 3584 2338 3584 \nQ 2397 3584 2469 3576 \nQ 2541 3569 2628 3553 \nL 2631 2963 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \nQ 1497 1759 1228 1600 \nQ 959 1441 959 1056 \nQ 959 750 1161 570 \nQ 1363 391 1709 391 \nQ 2188 391 2477 730 \nQ 2766 1069 2766 1631 \nL 2766 1759 \nL 2194 1759 \nz\nM 3341 1997 \nL 3341 0 \nL 2766 0 \nL 2766 531 \nQ 2569 213 2275 61 \nQ 1981 -91 1556 -91 \nQ 1019 -91 701 211 \nQ 384 513 384 1019 \nQ 384 1609 779 1909 \nQ 1175 2209 1959 2209 \nL 2766 2209 \nL 2766 2266 \nQ 2766 2663 2505 2880 \nQ 2244 3097 1772 3097 \nQ 1472 3097 1187 3025 \nQ 903 2953 641 2809 \nL 641 3341 \nQ 956 3463 1253 3523 \nQ 1550 3584 1831 3584 \nQ 2591 3584 2966 3190 \nQ 3341 2797 3341 1997 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \nL 1178 3500 \nL 1178 0 \nL 603 0 \nL 603 3500 \nz\nM 603 4863 \nL 1178 4863 \nL 1178 4134 \nL 603 4134 \nL 603 4863 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \nL 3513 0 \nL 2938 0 \nL 2938 2094 \nQ 2938 2591 2744 2837 \nQ 2550 3084 2163 3084 \nQ 1697 3084 1428 2787 \nQ 1159 2491 1159 1978 \nL 1159 0 \nL 581 0 \nL 581 3500 \nL 1159 3500 \nL 1159 2956 \nQ 1366 3272 1645 3428 \nQ 1925 3584 2291 3584 \nQ 2894 3584 3203 3211 \nQ 3513 2838 3513 2113 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-5f\" d=\"M 3263 -1063 \nL 3263 -1509 \nL -63 -1509 \nL -63 -1063 \nL 3263 -1063 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \nL 1178 4863 \nL 1178 0 \nL 603 0 \nL 603 4863 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \nL 2834 2853 \nQ 2591 2978 2328 3040 \nQ 2066 3103 1784 3103 \nQ 1356 3103 1142 2972 \nQ 928 2841 928 2578 \nQ 928 2378 1081 2264 \nQ 1234 2150 1697 2047 \nL 1894 2003 \nQ 2506 1872 2764 1633 \nQ 3022 1394 3022 966 \nQ 3022 478 2636 193 \nQ 2250 -91 1575 -91 \nQ 1294 -91 989 -36 \nQ 684 19 347 128 \nL 347 722 \nQ 666 556 975 473 \nQ 1284 391 1588 391 \nQ 1994 391 2212 530 \nQ 2431 669 2431 922 \nQ 2431 1156 2273 1281 \nQ 2116 1406 1581 1522 \nL 1381 1569 \nQ 847 1681 609 1914 \nQ 372 2147 372 2553 \nQ 372 3047 722 3315 \nQ 1072 3584 1716 3584 \nQ 2034 3584 2315 3537 \nQ 2597 3491 2834 3397 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-74\"/>\n      <use xlink:href=\"#DejaVuSans-72\" transform=\"translate(39.208984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(80.322266 0)\"/>\n      <use xlink:href=\"#DejaVuSans-69\" transform=\"translate(141.601562 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6e\" transform=\"translate(169.384766 0)\"/>\n      <use xlink:href=\"#DejaVuSans-5f\" transform=\"translate(232.763672 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" transform=\"translate(282.763672 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" transform=\"translate(310.546875 0)\"/>\n      <use xlink:href=\"#DejaVuSans-73\" transform=\"translate(371.728516 0)\"/>\n      <use xlink:href=\"#DejaVuSans-73\" transform=\"translate(423.828125 0)\"/>\n     </g>\n    </g>\n    <g id=\"line2d_15\">\n     <path d=\"M 140.8125 75.620313 \nL 150.8125 75.620313 \nL 160.8125 75.620313 \n\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #ff7f0e; stroke-width: 1.5\"/>\n    </g>\n    <g id=\"text_13\">\n     <!-- val_loss -->\n     <g transform=\"translate(168.8125 79.120313) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-76\" d=\"M 191 3500 \nL 800 3500 \nL 1894 563 \nL 2988 3500 \nL 3597 3500 \nL 2284 0 \nL 1503 0 \nL 191 3500 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-76\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(59.179688 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" transform=\"translate(120.458984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-5f\" transform=\"translate(148.242188 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" transform=\"translate(198.242188 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" transform=\"translate(226.025391 0)\"/>\n      <use xlink:href=\"#DejaVuSans-73\" transform=\"translate(287.207031 0)\"/>\n      <use xlink:href=\"#DejaVuSans-73\" transform=\"translate(339.306641 0)\"/>\n     </g>\n    </g>\n    <g id=\"line2d_16\">\n     <path d=\"M 140.8125 90.576563 \nL 150.8125 90.576563 \nL 160.8125 90.576563 \n\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #2ca02c; stroke-width: 1.5\"/>\n    </g>\n    <g id=\"text_14\">\n     <!-- val_acc -->\n     <g transform=\"translate(168.8125 94.076563) scale(0.1 -0.1)\">\n      <use xlink:href=\"#DejaVuSans-76\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(59.179688 0)\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" transform=\"translate(120.458984 0)\"/>\n      <use xlink:href=\"#DejaVuSans-5f\" transform=\"translate(148.242188 0)\"/>\n      <use xlink:href=\"#DejaVuSans-61\" transform=\"translate(198.242188 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(259.521484 0)\"/>\n      <use xlink:href=\"#DejaVuSans-63\" transform=\"translate(314.501953 0)\"/>\n     </g>\n    </g>\n   </g>\n  </g>\n </g>\n <defs>\n  <clipPath id=\"p9b8d85fff4\">\n   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n  </clipPath>\n </defs>\n</svg>\n"
     },
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=vision-transformer-md-10 fw=all -->\n",
    "\n",
    "Same parameter count, same data, same number of epochs — and the CNN wins\n",
    "by several points, about 90–92% against the ViT's 86–87%, a margin well\n",
    "beyond the run-to-run noise of these ten-epoch runs. The gap is what the\n",
    "convolutional prior is worth at this scale. Locality and translation\n",
    "equivariance are a strong prior the CNN has by construction; the\n",
    "transformer must buy the same facts with data, and we just watched how\n",
    "slowly that purchase proceeds: after ten epochs its position embeddings\n",
    "have barely begun to encode the grid. Nothing here says transformers are\n",
    "worse at vision. It says that *at 60,000 images* the architecture with\n",
    "the stronger prior wins,\n",
    "and it sets the stakes for the discussion below, where the data budget\n",
    "grows by four orders of magnitude and the verdict flips.\n",
    "\n",
    "## Summary and Discussion\n",
    "\n",
    "A vision transformer demonstrates, by construction, that the transformer\n",
    "block does not care what its tokens are: one strided convolution turns an\n",
    "image into a sequence, and everything downstream is the encoder stack this\n",
    "chapter already built. The block is the pre-norm, GELU configuration of\n",
    "that section; the only genuinely new parts are the patch\n",
    "embedding, the “&lt;cls&gt;” readout, and position embeddings learned from\n",
    "scratch.\n",
    "\n",
    "The experiments carry the real content. At matched parameter count, data,\n",
    "and training budget, the CNN wins, and it wins because of what it does not\n",
    "have to learn: convolution builds in locality and translation equivariance\n",
    "(that section), while the transformer must extract both from\n",
    "60,000 images. Scale flips the verdict. Trained on 300 million images, ViTs\n",
    "beat the best ResNets in image classification\n",
    "[@Dosovitskiy.Beyer.Kolesnikov.ea.2021]: given enough data, learned\n",
    "structure overtakes built-in structure, and the architecture with fewer\n",
    "commitments has more room to improve. The crossover does not even require\n",
    "new data — DeiT showed that aggressive augmentation and distillation make\n",
    "ViTs competitive on ImageNet-1k alone [@touvron2021training],\n",
    "manufacturing by transformation the invariances that convolution would have\n",
    "supplied by design. A complementary line, Swin transformers, reinstates\n",
    "convolution-like priors (local attention windows, hierarchical resolution)\n",
    "partly to escape the quadratic cost of global attention\n",
    "(that section) at high resolution [@liu2021swin].\n",
    "\n",
    "In the years since, the plain ViT, not its convolution-flavored variants,\n",
    "has become the standard vision backbone. Contrastively pretrained ViT\n",
    "encoders such as CLIP's [@radford2021learning] supply the image side\n",
    "of most vision-language models, and detection and segmentation systems\n",
    "routinely sit on ViT features; the Image Models part (that section)\n",
    "takes up these applications.\n",
    "\n",
    "## Exercises\n",
    "\n",
    "1. How does the value of `img_size` affect training time? Predict what\n",
    "   halving `patch_size` to 8 does to the sequence length and to the cost of\n",
    "   an attention layer (that section), then measure the\n",
    "   time per epoch.\n",
    "1. Instead of projecting the “&lt;cls&gt;” token representation to the\n",
    "   output, project the averaged patch representations. Implement this change\n",
    "   and see how it affects the accuracy.\n",
    "1. Add data augmentation — random horizontal flips and random crops — to the\n",
    "   training pipeline and retrain both the ViT and the CNN. Does the gap\n",
    "   between them shrink? Relate what you find to the DeiT recipe.\n",
    "1. Apply a fixed random permutation to the 36 patch tokens at test time\n",
    "   (after patch embedding, leaving the position embeddings in place) and\n",
    "   measure the accuracy of the trained ViT. What does the size of the drop\n",
    "   tell you about how much the model relies on its position embeddings?\n",
    "1. To fine-tune a trained ViT at a higher resolution, the standard trick is\n",
    "   to interpolate its position embeddings: reshape the $36$ patch embeddings\n",
    "   to $6 \\times 6 \\times d$, resize spatially to the new grid, and flatten\n",
    "   back. Implement this for $144 \\times 144$ inputs (a $9 \\times 9$ grid)\n",
    "   and verify that the model still classifies far better than chance\n",
    "   without any retraining."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "name": "d2l-pytorch",
   "display_name": "d2l (PyTorch)",
   "language": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
