KAHIBARO Bookshelves
Discord Login Register

9.4. Data Augmentation for Vision

What Data Augmentation Is and Why It Matters

Data augmentation is the practice of generating varied training examples from your existing images by applying label preserving transformations. The goal is to help a convolutional neural network learn features that are stable under changes it will naturally encounter at inference time, such as small shifts, different lighting, or slight rotations. Augmentation is most useful when you have limited data or when your dataset does not fully capture the real world variability you expect at deployment.

Augmentation should generally be applied only to the training set. Validation and test sets should represent the real input distribution you want to measure, so you typically use only deterministic preprocessing there, such as resizing and normalization.

Apply random augmentations to training data only. Validation and test data should not use randomness, otherwise metrics become noisy and misleading.

When Augmentation Helps, and When It Hurts

Augmentation helps when the transformation preserves the class label. For example, if you classify dogs versus cats, a horizontal flip usually keeps the label correct. If you classify left versus right hands, flipping changes the meaning and can corrupt labels.

Augmentation can also hurt when it creates unrealistic images that the model overfits to as artifacts, or when it changes the underlying task. Large rotations for digit recognition can turn a 6 into a 9. Aggressive cropping can remove the object completely. Color jitter can break tasks where color is the target signal, such as distinguishing ripe versus unripe fruit if color is the defining feature.

Only use augmentations that preserve the label for your task. If an augmentation can change the correct answer, it injects label noise and can reduce accuracy.

Common Vision Augmentations and Their Intent

Random resized cropping encourages robustness to framing and scale. In practice it is one of the most effective augmentations for natural images, but the crop parameters must be chosen to avoid removing the object too often.

Horizontal flipping teaches left right invariance. Vertical flipping is less common because many datasets have an implicit up direction.

Small rotations and affine transforms help with camera tilt and viewpoint changes. Keep them mild unless your domain truly includes large rotations.

Color jitter changes brightness, contrast, saturation, and hue to reduce reliance on exact lighting. Use caution in domains where color carries label information.

Gaussian blur and noise can improve robustness to sensor noise and compression, but they can also wash out fine details if overused.

Random erasing, also called cutout, removes a patch of pixels, encouraging the model to use multiple cues rather than one discriminative region.

MixUp and CutMix combine two images and their labels. These are powerful regularizers but they change the supervision format and are easiest when your labels are one hot or can be mixed. They are more advanced than basic geometric and color augmentations, but still common in image classification.

Implementing Augmentation with torchvision.transforms

In beginner PyTorch workflows, you typically use torchvision transforms inside your Dataset so they are applied on the fly each time an image is loaded. This gives effectively infinite variety without storing augmented copies.

A common pattern is to define separate transforms for training and validation. The training transform includes randomness, the validation transform is deterministic.

python
import torchvision.transforms as T
train_tfms = T.Compose([
    T.RandomResizedCrop(224, scale=(0.7, 1.0)),
    T.RandomHorizontalFlip(p=0.5),
    T.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2, hue=0.05),
    T.ToTensor(),
    T.Normalize(mean=(0.485, 0.456, 0.406),
                std=(0.229, 0.224, 0.225)),
])
val_tfms = T.Compose([
    T.Resize(256),
    T.CenterCrop(224),
    T.ToTensor(),
    T.Normalize(mean=(0.485, 0.456, 0.406),
                std=(0.229, 0.224, 0.225)),
])

The order matters. You typically perform geometric and color operations while the image is still in PIL or uint8 form, then convert to a tensor, then normalize. If you normalize first and then apply color jitter, the effect is not what you expect because you are no longer operating in the original pixel space.

A typical safe ordering is: geometric transforms, then color transforms, then ToTensor(), then Normalize(...). Do not randomly augment after normalization unless you know exactly why.

Integrating Augmentations into a Dataset and DataLoader

Many torchvision datasets accept a transform= argument. For example, with ImageFolder, the transform is applied to each image when it is loaded.

python
from torchvision.datasets import ImageFolder
from torch.utils.data import DataLoader
train_ds = ImageFolder("data/train", transform=train_tfms)
val_ds   = ImageFolder("data/val",   transform=val_tfms)
train_loader = DataLoader(train_ds, batch_size=64, shuffle=True, num_workers=4, pin_memory=True)
val_loader   = DataLoader(val_ds,   batch_size=64, shuffle=False, num_workers=4, pin_memory=True)

Shuffling is important for training. The randomness from augmentation does not replace the need to shuffle batches.

Augmentation Versus Normalization and Resizing

Resizing and normalization are usually considered preprocessing, not augmentation, because they are applied consistently and are not meant to diversify the dataset. Resizing ensures a consistent input size for the network. Normalization standardizes channels to a stable numeric scale that many pretrained models expect.

Augmentation creates variability. It is normal to use both: deterministic preprocessing for every split, plus random augmentation only for the training split.

Practical Tuning Guidelines

Augmentation strength is a hyperparameter. If training accuracy is very high but validation accuracy is much lower, slightly stronger augmentation can reduce overfitting. If both training and validation accuracy are low, augmentation may be too strong or the model may be underpowered, in which case you should reduce augmentation intensity before assuming the model architecture is the issue.

A simple way to sanity check augmentations is to visualize a few augmented samples and ask whether a human would still assign the same label confidently. If the answer is no, the augmentation is likely too aggressive.

Always visually inspect augmented samples early. If you would hesitate to label them, your model will struggle and your labels may be effectively corrupted.

Determinism and Reproducibility With Random Augmentations

Because training augmentations are random, two runs can produce slightly different results even with the same code. If you need reproducibility for debugging, you can set seeds and control DataLoader worker seeding, but expect some remaining variability, especially on GPU. The key point for augmentation is to keep validation deterministic so evaluation remains comparable across runs.

A Minimal Recommended Starter Recipe

For a first image classification CNN, a strong default is random resized crop, random horizontal flip, and mild color jitter, followed by tensor conversion and normalization. Start mild, confirm that the augmented images remain realistic, then increase strength only if you see overfitting and the task is label invariant to those transformations.

Views: 0

Comments

Please login to add a comment.

Don't have an account? Register now!

Where to Move

Move chapter:

  • ☰ 1. Orientation and Learning Setup
    • ☰ 1.1. Course Goals and How to Use This Outline
      • ☰ 1.2. What You Need to Know Before Starting
        • ☰ 1.3. Installing Python and PyTorch
          • ☰ 1.4. Choosing CPU vs CUDA vs Apple Silicon
            • ☰ 1.5. Using Jupyter, VS Code, and Scripts
              • ☰ 1.6. Reproducibility, Seeds, and Determinism
              • ☰ 2. Python and PyTorch Fundamentals
                • ☰ 2.1. Tensors and Basic Operations
                  • ☰ 2.2. Tensor Shapes, Dtypes, and Devices
                    • ☰ 2.3. Indexing, Slicing, and Broadcasting
                      • ☰ 2.4. Autograd and Computational Graphs
                        • ☰ 2.5. Common Tensor Pitfalls and Debugging
                          • ☰ 2.6. Saving and Loading Tensors
                          • ☰ 3. Neural Network Building Blocks
                            • ☰ 3.1. Linear Layers and Affine Transforms
                              • ☰ 3.2. Activation Functions
                                • ☰ 3.3. Loss Functions
                                  • ☰ 3.4. Optimizers and Learning Rates
                                    • ☰ 3.5. Regularization Basics
                                      • ☰ 3.6. Weight Initialization Concepts
                                      • ☰ 4. First Models End to End
                                        • ☰ 4.1. A Simple Regression Model
                                          • ☰ 4.2. A Simple Classification Model
                                            • ☰ 4.3. Data Splitting and Validation Sets
                                              • ☰ 4.4. Training Loop Structure
                                                • ☰ 4.5. Tracking Loss and Metrics
                                                  • ☰ 4.6. Basic Overfitting and Underfitting Checks
                                                  • ☰ 5. Data Handling with PyTorch
                                                    • ☰ 5.1. Datasets and DataLoaders
                                                      • ☰ 5.2. Transforms and Preprocessing
                                                        • ☰ 5.3. Collate Functions and Batching
                                                          • ☰ 5.4. Shuffling, Sampling, and Class Imbalance
                                                            • ☰ 5.5. Working with Tabular, Text, and Image Data
                                                              • ☰ 5.6. Data Pipeline Performance Basics
                                                              • ☰ 6. Training Loops in Practice
                                                                • ☰ 6.1. Forward Pass and Backward Pass
                                                                  • ☰ 6.2. Gradient Zeroing and Accumulation
                                                                    • ☰ 6.3. Device Placement and Mixed Precision Basics
                                                                      • ☰ 6.4. Learning Rate Schedules
                                                                        • ☰ 6.5. Gradient Clipping
                                                                          • ☰ 6.6. Logging with TensorBoard
                                                                          • ☰ 7. Debugging and Model Troubleshooting
                                                                            • ☰ 7.1. Verifying Shapes and NaNs
                                                                              • ☰ 7.2. Checking Gradients and Vanishing Gradients
                                                                                • ☰ 7.3. Sanity Checks and Overfit a Small Batch
                                                                                  • ☰ 7.4. Common Causes of Training Instability
                                                                                    • ☰ 7.5. Interpreting Learning Curves
                                                                                      • ☰ 7.6. Practical Tips for Faster Iteration
                                                                                      • ☰ 8. Feedforward Networks for Structured Data
                                                                                        • ☰ 8.1. Feature Scaling and Normalization
                                                                                          • ☰ 8.2. Embeddings for Categorical Features
                                                                                            • ☰ 8.3. Multi-Layer Perceptrons in PyTorch
                                                                                              • ☰ 8.4. Choosing Hidden Sizes and Depth
                                                                                                • ☰ 8.5. Evaluation Metrics for Classification and Regression
                                                                                                  • ☰ 8.6. Model Export and Inference Basics
                                                                                                  • ☰ 9. Convolutional Neural Networks for Images
                                                                                                    • ☰ 9.1. Convolutions, Padding, and Stride
                                                                                                      • ☰ 9.2. Pooling and Feature Maps
                                                                                                        • ☰ 9.3. Building a Basic CNN in PyTorch
                                                                                                          • ☰ 9.4. Data Augmentation for Vision
                                                                                                            • ☰ 9.5. Transfer Learning with Pretrained Models
                                                                                                              • ☰ 9.6. Fine-Tuning and Freezing Layers
                                                                                                              • ☰ 10. Recurrent and Sequence Models
                                                                                                                • ☰ 10.1. Sequences, Tokens, and Padding
                                                                                                                  • ☰ 10.2. RNNs, LSTMs, and GRUs
                                                                                                                    • ☰ 10.3. Handling Variable-Length Sequences
                                                                                                                      • ☰ 10.4. Teacher Forcing and Sequence Losses
                                                                                                                        • ☰ 10.5. Basic Text Classification Pipeline
                                                                                                                          • ☰ 10.6. Simple Sequence-to-Sequence Overview
                                                                                                                          • ☰ 11. Attention and Transformers Fundamentals
                                                                                                                            • ☰ 11.1. Why Attention Helps
                                                                                                                              • ☰ 11.2. Self-Attention Intuition
                                                                                                                                • ☰ 11.3. Transformer Building Blocks
                                                                                                                                  • ☰ 11.4. Using Transformer Models in PyTorch
                                                                                                                                    • ☰ 11.5. Fine-Tuning for Text Classification
                                                                                                                                      • ☰ 11.6. Managing Tokenizers and Batching
                                                                                                                                      • ☰ 12. Model Evaluation and Experiment Management
                                                                                                                                        • ☰ 12.1. Train, Validation, and Test Best Practices
                                                                                                                                          • ☰ 12.2. Cross-Validation Basics
                                                                                                                                            • ☰ 12.3. Calibration and Thresholding
                                                                                                                                              • ☰ 12.4. Error Analysis and Confusion Matrices
                                                                                                                                                • ☰ 12.5. Experiment Tracking Concepts
                                                                                                                                                  • ☰ 12.6. Comparing Models Fairly
                                                                                                                                                  • ☰ 13. Saving, Loading, and Deployment Readiness
                                                                                                                                                    • ☰ 13.1. Saving Model Weights and Full Checkpoints
                                                                                                                                                      • ☰ 13.2. Loading for Inference and Resuming Training
                                                                                                                                                        • ☰ 13.3. TorchScript and Tracing Basics
                                                                                                                                                          • ☰ 13.4. Exporting to ONNX Overview
                                                                                                                                                            • ☰ 13.5. Batch Inference and Latency Considerations
                                                                                                                                                              • ☰ 13.6. Reproducible Packaging of Code and Weights
                                                                                                                                                              • ☰ 14. Performance and Scaling Basics
                                                                                                                                                                • ☰ 14.1. Profiling and Bottleneck Identification
                                                                                                                                                                  • ☰ 14.2. DataLoader Optimization
                                                                                                                                                                    • ☰ 14.3. GPU Utilization and Memory Management
                                                                                                                                                                      • ☰ 14.4. Gradient Accumulation for Large Batches
                                                                                                                                                                        • ☰ 14.5. Distributed Training Concepts Overview
                                                                                                                                                                          • ☰ 14.6. Practical Guidelines for Faster Training
                                                                                                                                                                          • ☰ 15. Best Practices and Next Steps
                                                                                                                                                                            • ☰ 15.1. Choosing Hyperparameters Systematically
                                                                                                                                                                              • ☰ 15.2. Regularization and Generalization Toolkit
                                                                                                                                                                                • ☰ 15.3. Responsible Use and Dataset Considerations
                                                                                                                                                                                  • ☰ 15.4. Reading PyTorch Documentation Effectively
                                                                                                                                                                                    • ☰ 15.5. Suggested Projects for Beginners
                                                                                                                                                                                      • ☰ 15.6. Pathways to Advanced Topics