Table of Contents
What This Section Covers
Saving and loading is the bridge between training a model once and being able to use it again reliably, whether that is for evaluation, continued training, sharing with someone else, or deploying into an application. In PyTorch, this section is about understanding the main artifacts you can persist, how those artifacts map to common workflows like resuming training or running inference, and what extra constraints appear when you move from a notebook into something that must run predictably in production.
This section is also where training oriented code starts to separate into two modes. One mode is training mode, where you care about optimizer state, learning rate schedules, randomness, and continuity. The other mode is inference mode, where you care about stable outputs, speed, memory, and compatibility with the runtime environment.
The Two Most Common Things to Save
Most practical PyTorch projects save either model weights alone or a full training checkpoint. Model weights means the parameters and buffers of the model, typically via the model state dictionary. A full checkpoint means not only the model weights, but also everything needed to resume training from the same point, which commonly includes optimizer state, the current epoch or step, scheduler state, and sometimes additional metadata like the best validation score so far.
Rule: For long training runs you care about resuming, save full checkpoints. For deployment focused inference, save only what you need to run inference, usually the model weights plus the exact model code and preprocessing.
Training Continuity vs Inference Portability
A subtle but important distinction is that the best format for resuming training is not always the best format for deployment. A full checkpoint is excellent for training continuity because it captures optimizer internal buffers and learning rate scheduler counters that affect future updates. However, deployment portability often benefits from a representation that is less dependent on Python code structure and training time components, which is why later chapters in this section cover TorchScript, tracing, and ONNX at a high level.
You should also plan for the fact that deployment is not only about the model file. The input preprocessing and output postprocessing logic are part of the model contract. If you normalize inputs during training, you must apply the same normalization during inference. If you map class indices to label strings, you must preserve the mapping.
Rule: A saved model without the exact preprocessing and label mapping is incomplete. Treat preprocessing parameters, vocabulary files, tokenizers, and label mappings as versioned artifacts.
Environment and Compatibility Considerations
Saving and loading happens across machines, operating systems, and hardware backends. Even when the tensor data is identical, differences in device, dtype, and PyTorch version can affect what loads cleanly and how it runs. This section sets the stage for a few common concerns that will be handled in the child chapters, like loading a GPU trained model on CPU for inference, exporting for runtimes that do not run Python, and choosing between eager execution and exported graphs.
At a high level, you want to keep these constraints in mind. Deployment environments may not have CUDA. They may not have the same Python package versions. They may require a single file artifact. They may require fast batch inference, not single item prediction. The rest of this section breaks those concerns into specific techniques and patterns.
What You Should Be Able to Do After This Section
By the end of the chapters under this heading, you should be able to save model weights and full checkpoints, load them correctly for either inference or resuming training, export models into forms that are easier to deploy, and reason about performance constraints like batching and latency. You should also be able to package code, weights, and preprocessing in a way that makes your results reproducible for yourself and for others.