• convolution in a CNN: slide a small filter ( kernel ) over the input, take the dot product at each position, get a feature map
    • technically cross correlation, deep learning libraries don’t flip the kernel; the signal processing convolution does, and by the convolution theorem it’s a product in the Fourier domain
    • output size: for input , kernel , padding , stride
  • example: 5x5 image, 3x3 vertical edge filter, stride 1, no padding gives a 3x3 map
12031
45678
10121
01340
34512
10-1
10-1
10-1
  • top left output: left column minus right column , so little horizontal change there
  • why it works for images: the right inductive biases
    • local connectivity: each output only sees a small patch, its receptive field, which grows with depth
    • weight sharing: one filter everywhere, far fewer parameters than a dense layer
    • translation equivariance: shift the image and the feature map shifts with it; pooling buys some invariance
  • typical CNN: conv, ReLU, pooling, repeated, then flatten into dense layers and Softmax
    • early layers learn edge and blob detectors, later layers parts and objects, loosely like simple/complex cells in V1 ( Hubel & Wiesel, the original inspiration )
  • landmark nets: LeNet-5 ( digits, 1998 ), AlexNet ( ImageNet 2012 ), VGG ( stacked 3x3s ), ResNet ( skip connections let it train 100+ layers )
  • vision transformers swap the locality prior for Self Attention over patches; with enough data they match or beat CNNs, which is a pretty bitter lesson for hand built priors