Table of Contents
What a CNN Is Good At
Convolutional neural networks, or CNNs, are neural networks designed to work well with images by taking advantage of spatial structure. Images are not just arbitrary vectors, they have local neighborhoods where nearby pixels are related, and they often contain repeated patterns like edges and textures that can appear in many locations. CNNs build representations by applying the same learnable pattern detector across the whole image, which gives them a strong inductive bias for vision tasks and usually makes them more parameter efficient than fully connected networks on image shaped data.
A typical CNN turns an input image tensor into a sequence of increasingly abstract feature maps, then maps those features to a final prediction such as a class label. Earlier layers tend to detect simple patterns like edges, later layers combine those into parts and objects. You will implement the specific operations, such as convolution and pooling, in later chapters within this section, but here the goal is to understand the overall CNN workflow and the shape conventions you will rely on in PyTorch.
Image Tensors and Shape Conventions in PyTorch
In PyTorch, image tensors are commonly represented in channels first format. A single image is shaped $(C, H, W)$ where $C$ is the number of channels, $H$ is height, and $W$ is width. A batch of images is shaped $(N, C, H, W)$ where $N$ is the batch size. Many common bugs in CNN code come from mixing up this order or accidentally providing images in $(H, W, C)$ format, which is common in other libraries.
A CNN in PyTorch almost always expects images shaped $(N, C, H, W)$. If your data is $(N, H, W, C)$, you must permute it before feeding it to the model.
For typical natural images, $C = 3$ for RGB. For grayscale images, $C = 1$. The values are usually floating point and often normalized, which is covered in the dedicated preprocessing chapter. Here, what matters is that the CNN layers will transform these tensors while preserving the idea of channel and spatial dimensions.
The High Level CNN Pipeline
A practical CNN for image classification usually follows a pattern. The network begins with a stack of convolutional blocks that repeatedly transform $(N, C, H, W)$ into new feature maps with a different number of channels and usually smaller spatial size. After several blocks, the network converts the final feature maps into a vector representation per image, then uses a small classifier head to produce logits.
Conceptually, the pipeline looks like this. You start with an input image batch $x$ shaped $(N, C, H, W)$. You apply a feature extractor, which returns a tensor shaped roughly like $(N, C', H', W')$. You reduce the spatial dimensions, either by flattening $(C' \cdot H' \cdot W')$ or by using global pooling to get $(N, C')$. You then apply one or more linear layers to get logits shaped $(N, K)$ where $K$ is the number of classes.
For multi class classification in PyTorch, the model should output raw logits, not softmax probabilities, when you plan to use nn.CrossEntropyLoss.
This section will later cover how convolutions, padding, stride, and pooling determine $H'$ and $W'$. At this level, you should keep track of how tensors flow through the model and where shape changes typically happen.
Local Receptive Fields and Weight Sharing
Two ideas make CNNs different from a plain multilayer perceptron when working with images. The first is the local receptive field, meaning a neuron in a convolutional layer looks at a small patch of the input image or feature map, not the entire image at once. The second is weight sharing, meaning the same small set of weights is used to scan across all spatial locations. This lets the network learn a detector like a vertical edge and apply it anywhere in the image.
As you stack layers, the effective receptive field grows, so deeper layers can integrate information from larger regions of the original image. This supports hierarchical feature learning without requiring fully connected connections to every pixel.
Feature Maps and Channels as Learned Detectors
The output channels of a convolutional layer can be thought of as different learned detectors, each producing a feature map over spatial positions. If a layer outputs $C'$ channels, it has learned $C'$ kinds of features at that stage of the network. As you go deeper, $C'$ often increases while $H$ and $W$ decrease, which is a common design pattern because high level concepts can be represented with fewer spatial locations but more semantic channels.
A practical mental model is that CNNs trade spatial resolution for semantic richness. You will see this explicitly when you build a basic CNN and observe the tensor shapes after each layer.
Downsampling and the Transition to a Classifier Head
CNNs often include operations that reduce spatial size, called downsampling. This makes computation cheaper and increases the effective receptive field. Downsampling can happen via strided convolutions or pooling layers, which will be covered later. Eventually, you need to turn a grid of features into a fixed size vector per image, so you can produce predictions independent of the input resolution, within whatever constraints your architecture imposes.
There are two common strategies. One is flattening, which turns $(N, C', H', W')$ into $(N, C' H' W')$ and then uses linear layers. The other is global average pooling, which averages over $H'$ and $W'$ to get $(N, C')$ and then applies a linear layer. Global pooling tends to reduce parameters and can improve robustness, but the right choice depends on the architecture and task.
A Minimal CNN Skeleton in PyTorch
A CNN in PyTorch is typically an nn.Module with a feature extractor part and a head part. The exact layers will be introduced in subsequent chapters, but it helps to see a high level template so you recognize the structure when reading code.
A typical skeleton looks like a module with self.features containing convolutional blocks, and self.classifier containing the final mapping to logits. In forward, you apply the feature extractor, reduce spatial dimensions, then apply the classifier. When you implement your own, you will frequently print shapes during early debugging to ensure $(N, C, H, W)$ stays consistent.
If your final layer outputs $(N, K)$ logits, do not apply softmax inside forward during training when using CrossEntropyLoss. Apply softmax only for reporting probabilities at inference time.
Where This Fits in the Rest of the Section
This chapter established the big picture of how CNNs process image tensors, why they are a good match for images, and what shape conventions you will follow in PyTorch. The next chapters in this section will make that concrete by defining convolution operations, padding and stride, pooling, building a basic CNN module, then expanding into augmentation and transfer learning workflows.