{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "---\n",
    "execute:\n",
    "  enabled: false\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=channels-md-1 fw=all -->\n",
    "\n",
    "# Multiple Input and Multiple Output Channels\n",
    "\n",
    "While we described the multiple channels\n",
    "that comprise each image (e.g., color images have the standard RGB channels\n",
    "to indicate the amount of red, green and blue) and convolutional layers for multiple channels in that section,\n",
    "until now, we simplified all of our numerical examples\n",
    "by working with just a single input and a single output channel.\n",
    "This allowed us to think of our inputs, convolution kernels,\n",
    "and outputs each as two-dimensional tensors.\n",
    "\n",
    "When we add channels into the mix,\n",
    "our inputs and hidden representations\n",
    "both become three-dimensional tensors.\n",
    "For example, each RGB input image has shape $3\\times h\\times w$.\n",
    "We refer to this axis, with a size of 3, as the *channel* dimension. The notion of\n",
    "channels is as old as CNNs themselves: for instance LeNet-5 [@LeCun.Jackel.Bottou.ea.1995] uses them. \n",
    "In this section, we will take a deeper look\n",
    "at convolution kernels with multiple input and multiple output channels."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "channels-multiple-input-and-multiple-output-channels"
   },
   "source": [
    "from d2l import mxnet as d2l\n",
    "from mxnet import np, npx\n",
    "from mxnet.gluon import nn\n",
    "npx.set_np()"
   ],
   "execution_count": 1,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=channels-md-2 fw=all -->\n",
    "\n",
    "## Multiple Input Channels\n",
    "\n",
    "When the input data contains multiple channels,\n",
    "we need to construct a convolution kernel\n",
    "with the same number of input channels as the input data,\n",
    "so that it can perform cross-correlation with the input data.\n",
    "Assuming that the number of channels for the input data is $c_\\textrm{i}$,\n",
    "the number of input channels of the convolution kernel also needs to be $c_\\textrm{i}$. If our convolution kernel's window shape is $k_\\textrm{h}\\times k_\\textrm{w}$,\n",
    "then, when $c_\\textrm{i}=1$, we can think of our convolution kernel\n",
    "as just a two-dimensional tensor of shape $k_\\textrm{h}\\times k_\\textrm{w}$.\n",
    "\n",
    "However, when $c_\\textrm{i}>1$, we need a kernel\n",
    "that contains a tensor of shape $k_\\textrm{h}\\times k_\\textrm{w}$ for *every* input channel. Concatenating these $c_\\textrm{i}$ tensors together\n",
    "yields a convolution kernel of shape $c_\\textrm{i}\\times k_\\textrm{h}\\times k_\\textrm{w}$.\n",
    "Since the input and convolution kernel each have $c_\\textrm{i}$ channels,\n",
    "we can perform a cross-correlation operation\n",
    "on the two-dimensional tensor of the input\n",
    "and the two-dimensional tensor of the convolution kernel\n",
    "for each channel, adding the $c_\\textrm{i}$ results together\n",
    "(summing over the channels)\n",
    "to yield a two-dimensional tensor.\n",
    "This is the result of a two-dimensional cross-correlation\n",
    "between a multi-channel input and\n",
    "a multi-input-channel convolution kernel.\n",
    "\n",
    "the figure provides an example \n",
    "of a two-dimensional cross-correlation with two input channels.\n",
    "The shaded portions are the first output element\n",
    "as well as the input and kernel tensor elements used for the output computation:\n",
    "$(1\\times1+2\\times2+4\\times3+5\\times4)+(0\\times0+1\\times1+3\\times2+4\\times3)=56$.\n",
    "\n",
    "![Cross-correlation computation with two input channels.](../img/conv-multi-in.svg)\n",
    "\n",
    "\n",
    "To make sure we really understand what is going on here,\n",
    "we can implement cross-correlation operations with multiple input channels ourselves.\n",
    "Notice that all we are doing is performing a cross-correlation operation\n",
    "per channel and then adding up the results."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "channels-multiple-input-channels-1"
   },
   "source": [
    "def corr2d_multi_in(X, K):\n",
    "    # Iterate through the 0th dimension (channel) of K first, then add them up\n",
    "    return sum(d2l.corr2d(x, k) for x, k in zip(X, K))"
   ],
   "execution_count": 2,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=channels-md-3 fw=all -->\n",
    "\n",
    "We can construct the input tensor `X` and the kernel tensor `K`\n",
    "corresponding to the values in the figure\n",
    "to validate the output of the cross-correlation operation."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "channels-multiple-input-channels-2"
   },
   "source": [
    "X = d2l.tensor([[[0.0, 1.0, 2.0], [3.0, 4.0, 5.0], [6.0, 7.0, 8.0]],\n",
    "               [[1.0, 2.0, 3.0], [4.0, 5.0, 6.0], [7.0, 8.0, 9.0]]])\n",
    "K = d2l.tensor([[[0.0, 1.0], [2.0, 3.0]], [[1.0, 2.0], [3.0, 4.0]]])\n",
    "\n",
    "corr2d_multi_in(X, K)"
   ],
   "execution_count": 3,
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "array([[ 56.,  72.],\n       [104., 120.]])"
     },
     "metadata": {},
     "execution_count": 3
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=channels-md-4 fw=all -->\n",
    "\n",
    "## Multiple Output Channels\n",
    "\n",
    "Regardless of the number of input channels,\n",
    "so far we always ended up with one output channel.\n",
    "However, as we discussed in that section,\n",
    "it turns out to be essential to have multiple channels at each layer.\n",
    "In the most popular neural network architectures,\n",
    "we actually increase the channel dimension\n",
    "as we go deeper in the neural network,\n",
    "typically downsampling to trade off spatial resolution\n",
    "for greater *channel depth*.\n",
    "Intuitively, you could think of each channel\n",
    "as responding to a different set of features.\n",
    "The reality is a bit more complicated than this. A naive interpretation would suggest \n",
    "that representations are learned independently per pixel or per channel. \n",
    "Instead, channels are optimized to be jointly useful.\n",
    "This means that rather than mapping a single channel to an edge detector, it may simply mean \n",
    "that some direction in channel space corresponds to detecting edges.\n",
    "\n",
    "Denote by $c_\\textrm{i}$ and $c_\\textrm{o}$ the number\n",
    "of input and output channels, respectively,\n",
    "and by $k_\\textrm{h}$ and $k_\\textrm{w}$ the height and width of the kernel.\n",
    "To get an output with multiple channels,\n",
    "we can create a kernel tensor\n",
    "of shape $c_\\textrm{i}\\times k_\\textrm{h}\\times k_\\textrm{w}$\n",
    "for *every* output channel.\n",
    "We concatenate them on the output channel dimension,\n",
    "so that the shape of the convolution kernel\n",
    "is $c_\\textrm{o}\\times c_\\textrm{i}\\times k_\\textrm{h}\\times k_\\textrm{w}$.\n",
    "In cross-correlation operations,\n",
    "the result on each output channel is calculated\n",
    "from the convolution kernel corresponding to that output channel\n",
    "and takes input from all channels in the input tensor.\n",
    "\n",
    "We implement a cross-correlation function\n",
    "to calculate the output of multiple channels as shown below."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "channels-multiple-output-channels-1"
   },
   "source": [
    "def corr2d_multi_in_out(X, K):\n",
    "    # Iterate through the 0th dimension of K, and each time, perform\n",
    "    # cross-correlation operations with input X. All of the results are\n",
    "    # stacked together\n",
    "    return d2l.stack([corr2d_multi_in(X, k) for k in K], 0)"
   ],
   "execution_count": 4,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=channels-md-5 fw=all -->\n",
    "\n",
    "We construct a trivial convolution kernel with three output channels\n",
    "by concatenating the kernel tensor for `K` with `K+1` and `K+2`."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "channels-multiple-output-channels-2"
   },
   "source": [
    "K = d2l.stack((K, K + 1, K + 2), 0)\n",
    "K.shape"
   ],
   "execution_count": 5,
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "(3, 2, 2, 2)"
     },
     "metadata": {},
     "execution_count": 5
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=channels-md-6 fw=all -->\n",
    "\n",
    "Below, we perform cross-correlation operations\n",
    "on the input tensor `X` with the kernel tensor `K`.\n",
    "Now the output contains three channels.\n",
    "The result of the first channel is consistent\n",
    "with the result of the previous input tensor `X`\n",
    "and the multi-input channel,\n",
    "single-output channel kernel."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "channels-multiple-output-channels-3"
   },
   "source": [
    "corr2d_multi_in_out(X, K)"
   ],
   "execution_count": 6,
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "array([[[ 56.,  72.],\n        [104., 120.]],\n\n       [[ 76., 100.],\n        [148., 172.]],\n\n       [[ 96., 128.],\n        [192., 224.]]])"
     },
     "metadata": {},
     "execution_count": 6
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=channels-md-7 fw=all -->\n",
    "\n",
    "## $1\\times 1$ Convolutional Layer\n",
    "\n",
    "At first, a $1 \\times 1$ convolution, i.e., $k_\\textrm{h} = k_\\textrm{w} = 1$,\n",
    "does not seem to make much sense.\n",
    "After all, a convolution correlates adjacent pixels.\n",
    "A $1 \\times 1$ convolution obviously does not.\n",
    "Nonetheless, they are popular operations that are sometimes included\n",
    "in the designs of complex deep networks [@Lin.Chen.Yan.2013; @Szegedy.Ioffe.Vanhoucke.ea.2017].\n",
    "Let's see in some detail what it actually does.\n",
    "\n",
    "Because the minimum window is used,\n",
    "the $1\\times 1$ convolution loses the ability\n",
    "of larger convolutional layers\n",
    "to recognize patterns consisting of interactions\n",
    "among adjacent elements in the height and width dimensions.\n",
    "The only computation of the $1\\times 1$ convolution occurs\n",
    "on the channel dimension.\n",
    "\n",
    "the figure shows the cross-correlation computation\n",
    "using the $1\\times 1$ convolution kernel\n",
    "with 3 input channels and 2 output channels.\n",
    "Note that the inputs and outputs have the same height and width.\n",
    "Each element in the output is derived\n",
    "from a linear combination of elements *at the same position*\n",
    "in the input image.\n",
    "You could think of the $1\\times 1$ convolutional layer\n",
    "as constituting a fully connected layer applied at every single pixel location\n",
    "to transform the $c_\\textrm{i}$ corresponding input values into $c_\\textrm{o}$ output values.\n",
    "Because this is still a convolutional layer,\n",
    "the weights are tied across pixel location.\n",
    "Thus the $1\\times 1$ convolutional layer requires $c_\\textrm{o}\\times c_\\textrm{i}$ weights\n",
    "(plus the bias). Also note that convolutional layers are typically followed \n",
    "by nonlinearities. This ensures that $1 \\times 1$ convolutions cannot simply be \n",
    "folded into other convolutions. \n",
    "\n",
    "![The cross-correlation computation uses the $1\\times 1$ convolution kernel with three input channels and two output channels. The input and output have the same height and width.](../img/conv-1x1.svg)\n",
    "\n",
    "Let's check whether this works in practice:\n",
    "we implement a $1 \\times 1$ convolution\n",
    "using a fully connected layer.\n",
    "The only thing is that we need to make some adjustments\n",
    "to the data shape before and after the matrix multiplication."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "channels-1-times-1-convolutional-layer-1"
   },
   "source": [
    "def corr2d_multi_in_out_1x1(X, K):\n",
    "    c_i, h, w = X.shape\n",
    "    c_o = K.shape[0]\n",
    "    X = d2l.reshape(X, (c_i, h * w))\n",
    "    K = d2l.reshape(K, (c_o, c_i))\n",
    "    # Matrix multiplication in the fully connected layer\n",
    "    Y = d2l.matmul(K, X)\n",
    "    return d2l.reshape(Y, (c_o, h, w))"
   ],
   "execution_count": 7,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=channels-md-8 fw=all -->\n",
    "\n",
    "When performing $1\\times 1$ convolutions,\n",
    "the above function is equivalent to the previously implemented cross-correlation function `corr2d_multi_in_out`.\n",
    "Let's check this with some sample data."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "channels-1-times-1-convolutional-layer-2"
   },
   "source": [
    "X = d2l.normal(0, 1, (3, 3, 3))\n",
    "K = d2l.normal(0, 1, (2, 3, 1, 1))\n",
    "Y1 = corr2d_multi_in_out_1x1(X, K)\n",
    "Y2 = corr2d_multi_in_out(X, K)\n",
    "assert float(d2l.reduce_sum(d2l.abs(Y1 - Y2))) < 1e-5"
   ],
   "execution_count": 8,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=channels-md-9 fw=all -->\n",
    "\n",
    "## Grouped, Depthwise, and Depthwise-Separable Convolutions\n",
    "\n",
    "So far every output channel has depended on every input channel:\n",
    "the kernel carries $c_\\textrm{o} \\times c_\\textrm{i} \\times k_\\textrm{h} \\times k_\\textrm{w}$\n",
    "weights, and both parameters and compute scale with the product\n",
    "$c_\\textrm{i} \\cdot c_\\textrm{o}$. A *grouped convolution* relaxes this.\n",
    "Split the $c_\\textrm{i}$ input channels into $g$ groups of $c_\\textrm{i}/g$\n",
    "channels each, split the $c_\\textrm{o}$ output channels likewise,\n",
    "and let each output group convolve over its own input group only.\n",
    "Viewed as a map on channels, the kernel becomes block-diagonal:\n",
    "$g$ independent blocks of size $(c_\\textrm{o}/g) \\times (c_\\textrm{i}/g)$\n",
    "replace the single dense $c_\\textrm{o} \\times c_\\textrm{i}$ pattern of connections.\n",
    "The parameter and operation count drops from\n",
    "$c_\\textrm{i} c_\\textrm{o} k_\\textrm{h} k_\\textrm{w}$\n",
    "to $g \\cdot (c_\\textrm{i}/g)(c_\\textrm{o}/g) k_\\textrm{h} k_\\textrm{w} = c_\\textrm{i} c_\\textrm{o} k_\\textrm{h} k_\\textrm{w} / g$,\n",
    "a factor of $g$. The price is that no information flows between groups\n",
    "within the layer, so architectures built on grouped convolutions\n",
    "restore the exchange elsewhere, typically with $1\\times 1$ convolutions\n",
    "between grouped layers. ResNeXt (that section) turns exactly\n",
    "this trade-off into a design principle.\n",
    "\n",
    "Pushing the idea to its extreme, $g = c_\\textrm{i} = c_\\textrm{o}$,\n",
    "gives the *depthwise convolution*: each channel is filtered by its own\n",
    "$k_\\textrm{h} \\times k_\\textrm{w}$ kernel and no channel mixing happens at all.\n",
    "This is the mirror image of the $1\\times 1$ convolution from\n",
    "that section, which mixes channels but ignores spatial structure.\n",
    "Composing the two, a depthwise $k \\times k$ convolution followed by a\n",
    "*pointwise* $1\\times 1$ convolution, yields the *depthwise-separable\n",
    "convolution* [@chollet2017xception; @howard2017mobilenet].\n",
    "the figure contrasts it with the dense layer it replaces:\n",
    "the depthwise stage looks at neighborhoods within each channel,\n",
    "the pointwise stage recombines channels at each position.\n",
    "\n",
    "![Dense convolution mixes all input channels; a depthwise convolution filters each channel separately and a pointwise convolution mixes them.](../img/arch-conv-depthwise.svg)\n",
    "\n",
    "How much does the factorization save? For a $k \\times k$ kernel producing\n",
    "an $h \\times w$ output, the dense convolution costs\n",
    "$h w \\cdot c_\\textrm{i} c_\\textrm{o} k^2$ multiplications.\n",
    "The separable pair costs $h w \\cdot c_\\textrm{i} k^2$ for the depthwise stage\n",
    "plus $h w \\cdot c_\\textrm{i} c_\\textrm{o}$ for the pointwise stage. The ratio is\n",
    "\n",
    "$$\n",
    "\\frac{h w \\cdot c_\\textrm{i} k^2 + h w \\cdot c_\\textrm{i} c_\\textrm{o}}{h w \\cdot c_\\textrm{i} c_\\textrm{o} k^2}\n",
    "= \\frac{1}{c_\\textrm{o}} + \\frac{1}{k^2}.\n",
    "$$\n",
    "\n",
    "For $k = 3$ and a large number of output channels this is close to $1/9$:\n",
    "the separable layer is roughly eight to nine times cheaper, in parameters\n",
    "and in operations alike. The saving is not free. A depthwise-separable\n",
    "layer can only express convolutions that factor into a per-channel spatial\n",
    "filter followed by a channel mixture, a strict subset of dense convolutions.\n",
    "In practice the accuracy given up is small relative to the compute saved,\n",
    "which is why the factorization anchors the mobile architectures of\n",
    "that section and appears, with larger kernels, in\n",
    "ConvNeXt (that section).\n",
    "\n",
    "Let's verify the arithmetic. We build a dense $3 \\times 3$ convolution with\n",
    "128 input and output channels and its depthwise-separable factorization,\n",
    "then compare parameter counts; the equation predicts a\n",
    "ratio of $(1/128 + 1/9)^{-1} \\approx 8.4$. We also check that both map an\n",
    "input to an output of the same shape."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "channels-grouped-depthwise-and-depthwise-separable-convolutions"
   },
   "source": [
    "c_i, c_o, k = 128, 128, 3\n",
    "X = d2l.normal(0, 1, (1, c_i, 32, 32))\n",
    "dense = nn.Conv2D(c_o, kernel_size=k, padding=1, use_bias=False)\n",
    "depthwise = nn.Conv2D(c_i, kernel_size=k, padding=1, groups=c_i,\n",
    "                      use_bias=False)\n",
    "pointwise = nn.Conv2D(c_o, kernel_size=1, use_bias=False)\n",
    "for layer in (dense, depthwise, pointwise):\n",
    "    layer.initialize()\n",
    "assert dense(X).shape == pointwise(depthwise(X)).shape\n",
    "p_dense = dense.weight.data().size\n",
    "p_sep = depthwise.weight.data().size + pointwise.weight.data().size\n",
    "p_dense, p_sep, p_dense / p_sep"
   ],
   "execution_count": 9,
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "(147456, 17536, 8.408759124087592)"
     },
     "metadata": {},
     "execution_count": 9
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=channels-md-10 fw=all -->\n",
    "\n",
    "## Discussion\n",
    "\n",
    "Channels let a convolutional network maintain many learned features at every\n",
    "location and mix them through learned linear maps and nonlinearities. They offer\n",
    "a practical trade-off between the parameter reduction arising from translation\n",
    "equivariance and locality and the need for expressive image models.\n",
    "\n",
    "Note, though, that this flexibility comes at a price. Given an image of size $(h \\times w)$, the cost for computing a $k \\times k$ convolution is $\\mathcal{O}(h \\cdot w \\cdot k^2)$. For $c_\\textrm{i}$ and $c_\\textrm{o}$ input and output channels respectively this increases to $\\mathcal{O}(h \\cdot w \\cdot k^2 \\cdot c_\\textrm{i} \\cdot c_\\textrm{o})$. For a $256 \\times 256$ pixel image with a $5 \\times 5$ kernel and $128$ input and output channels respectively this amounts to over 53 billion operations (we count multiplications and additions separately). Later on we will encounter effective strategies to cut down on the cost, e.g., by requiring the channel-wise operations to be block-diagonal, leading to architectures such as ResNeXt [@Xie.Girshick.Dollar.ea.2017]. The depthwise-separable factorization of that section is the extreme point of that strategy: by the equation it cuts the cost by a factor of $(1/c_\\textrm{o} + 1/k^2)^{-1}$, here about $21\\times$. \n",
    "\n",
    "## Exercises\n",
    "\n",
    "1. Assume that we have two convolution kernels of size $k_1$ and $k_2$, respectively \n",
    "   (with no nonlinearity in between).\n",
    "    1. Prove that the result of the operation can be expressed by a single convolution.\n",
    "    1. What is the dimensionality of the equivalent single convolution?\n",
    "    1. Is the converse true, i.e., can you always decompose a convolution into two smaller ones?\n",
    "1. Assume an input of shape $c_\\textrm{i}\\times h\\times w$ and a convolution kernel of shape \n",
    "   $c_\\textrm{o}\\times c_\\textrm{i}\\times k_\\textrm{h}\\times k_\\textrm{w}$, padding of $(p_\\textrm{h}, p_\\textrm{w})$, and stride of $(s_\\textrm{h}, s_\\textrm{w})$.\n",
    "    1. What is the computational cost (multiplications and additions) for the forward propagation?\n",
    "    1. What is the memory footprint?\n",
    "    1. What is the memory footprint for the backward computation?\n",
    "    1. What is the computational cost for the backpropagation?\n",
    "1. By what factor does the number of calculations increase if we double both the number of input channels \n",
    "   $c_\\textrm{i}$ and the number of output channels $c_\\textrm{o}$? What happens if we double the padding?\n",
    "1. Are the variables `Y1` and `Y2` in the final example of this section exactly the same? Why?\n",
    "1. Express convolutions as a matrix multiplication, even when the convolution window is not $1 \\times 1$. \n",
    "1. Your task is to implement fast convolutions with a $k \\times k$ kernel. One of the algorithm candidates \n",
    "   is to scan horizontally across the source, reading a $k$-wide strip and computing the $1$-wide output strip \n",
    "   one value at a time. The alternative is to read a $k + \\Delta$ wide strip and compute a $\\Delta$-wide \n",
    "   output strip. Why is the latter preferable? Is there a limit to how large you should choose $\\Delta$?\n",
    "1. A grouped convolution with $g$ groups (that section) acts on channels as a block-diagonal matrix with $g$ blocks.\n",
    "    1. By what factor does grouping reduce the number of parameters and the computational cost, compared to a dense convolution with the same $c_\\textrm{i}$, $c_\\textrm{o}$, and kernel size?\n",
    "    1. What is the downside of having $g$ groups? How could you fix it, at least partly, without giving up the savings entirely?\n",
    "1. Consider a block of two dense $3 \\times 3$ convolutions, each with $c$ input and $c$ output channels, the building block of VGG (that section). Now replace each of the two convolutions by its depthwise-separable counterpart.\n",
    "    1. Compute the number of parameters and the number of multiplications on an $h \\times w$ input for both variants.\n",
    "    1. Which of the two stages, depthwise or pointwise, dominates the cost of the separable block? What does this suggest about where to spend additional capacity?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- d2l:prose id=channels-md-11 fw=mxnet -->\n",
    "\n",
    "[Discussions](https://d2l.discourse.group/t/69)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "name": "d2l-mxnet",
   "display_name": "d2l (MXNet)",
   "language": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
