A plain convolution always shrinks its input. With an n \times n image and a k \times k kernel:
n \times n \;\longrightarrow\; (n - k + 1) \times (n - k + 1).
Stack ten 5×5 layers on a 240×240 image:
240 \to 236 \to 232 \to \ldots \to 200.
30% of the area gone — all of it from the boundary. Two knobs to control this: padding (fight the shrink) and stride (lean into it, on purpose).
The boundary problem
Even before stacking, single convs use boundary pixels much less than central ones. Each pixel only contributes when the kernel window covers it — interior pixels appear in many windows, corner pixels in just one:
Pixel utilization: 1×1, 2×2, and 3×3 kernels. Larger kernels make the boundary problem worse.
Information at the edges is systematically underweighted. Padding fixes both the shrinking and the underweighting.
Padding: add zeros around the input
Add a “frame” of zero-valued pixels around the input. The kernel can now slide further, including positions where its window hangs off the original image:
3×3 input padded to 5×5; 2×2 kernel gives a 4×4 output. The shaded element is 0{\cdot}0 + 0{\cdot}1 + 0{\cdot}2 + 0{\cdot}3 = 0.
For odd k, (k-1)/2 is an integer — we can pad symmetrically, the same on both sides, and the output position (i, j) corresponds to a window centered on input (i, j). Clean to reason about.
Even k forces a left/right asymmetry — pad floor on one side, ceil on the other.
That’s why every modern CNN you see uses 3×3 kernels with padding 1, or 5×5 with padding 2, or 7×7 with padding 3.
Padding helper
Define a helper to wrap input/output reshaping, then ask the basic question: 8×8 input, 3×3 kernel, padding=1 — what’s the output shape?
from mxnet import np, npxfrom mxnet.gluon import nnnpx.set_np()
# We define a helper function to calculate convolutions. It initializes # the convolutional layer weights and performs corresponding dimensionality # elevations and reductions on the input and outputdef comp_conv2d(conv2d, X): conv2d.initialize()# (1, 1) indicates that batch size and the number of channels are both 1 X = X.reshape((1, 1) + X.shape) Y = conv2d(X)# Strip the first two dimensions: examples and channelsreturn Y.reshape(Y.shape[2:])# 1 row and column is padded on either side, so a total of 2 rows or columns are addedconv2d = nn.Conv2D(1, kernel_size=3, padding=1)X = np.random.uniform(size=(8, 8))comp_conv2d(conv2d, X).shape
Same shape — the padded conv is “shape-preserving”. With an asymmetric kernel, mirror the asymmetry in the padding:
# We use a convolution kernel with height 5 and width 3. The padding on# either side of the height and width are 2 and 1, respectivelyconv2d = nn.Conv2D(1, kernel_size=(5, 3), padding=(2, 1))comp_conv2d(conv2d, X).shape
Stride: skipping positions on purpose
The opposite problem: sometimes the input is huge and we want to shrink fast. Move the kernel by s > 1 at each step — skipping intermediate positions:
Cross-correlation with vertical stride 3 and horizontal stride 2. The kernel jumps three rows down and two columns right.
Computational benefit: s\times fewer positions to evaluate in each direction, so s_h s_w \times fewer operations. Statistical benefit: aggressive downsampling forces the network to summarize.
