KAHIBARO Bookshelves
Discord Login Register

2.6. Saving and Loading Tensors

Why saving and loading tensors matters

Tensors are the raw material of PyTorch work. You might want to save a preprocessed dataset tensor to avoid repeating expensive preprocessing, cache embeddings, store intermediate results for debugging, or checkpoint a long running computation. In this chapter you will learn the practical mechanics of saving and loading tensors, how to preserve device and dtype correctly, and how to avoid common file format mistakes.

The core APIs: `torch.save` and `torch.load`

PyTorch’s standard way to serialize tensors is torch.save and torch.load. Under the hood this uses Python pickling plus PyTorch’s storage format, which is designed to preserve tensor metadata such as shape, dtype, and device information.

A minimal example looks like this:

python
import torch
x = torch.randn(3, 4)
torch.save(x, "x.pt")
y = torch.load("x.pt")
print(y.shape, y.dtype, y.device)

torch.save can write a single tensor, or a Python object that contains tensors, such as a dict or list. This is often the most convenient way to bundle multiple related tensors.

Saving multiple tensors together

A common pattern is to save a dictionary of tensors, possibly with a few simple metadata values.

python
import torch
data = {
    "features": torch.randn(100, 20),
    "labels": torch.randint(0, 5, (100,)),
    "mean": torch.tensor(0.123),   # tensors are fine
    "std": torch.tensor(0.987),
}
torch.save(data, "dataset_cache.pt")
loaded = torch.load("dataset_cache.pt")
X = loaded["features"]
y = loaded["labels"]

You can also save tuples and lists, but dicts are easier to extend later without breaking your loading code.

Only load .pt files you trust. torch.load uses pickle and can execute arbitrary code during loading if the file is malicious.

File extensions and conventions

PyTorch does not require a particular extension, but common conventions are .pt and .pth. You can use either for tensors and PyTorch saved objects. Prefer a consistent naming scheme, for example train_features.pt, embeddings_cache.pt, or checkpoint_step_1000.pt.

Device handling when loading, `map_location`

If you save a CUDA tensor and later load on a machine without a GPU, you must map the tensors to CPU during load. Use map_location.

python
import torch
# Suppose this was saved from a GPU machine
obj = torch.load("x.pt", map_location="cpu")

You can also map to a specific GPU.

python
obj = torch.load("x.pt", map_location="cuda:0")

If you want to load on CPU first and move later, that is also fine and is often a clean approach for portability.

python
obj = torch.load("x.pt", map_location="cpu")
obj = obj.to("cuda:0")

Do not assume that a saved tensor will load onto the device you want. Use map_location explicitly in code that needs to run on different hardware.

Dtype preservation and intentional casting after load

torch.save preserves dtype. If you need a different dtype for training or inference, cast after loading.

python
x = torch.load("x.pt", map_location="cpu")
x_fp16 = x.to(torch.float16)
x_fp32 = x.to(torch.float32)

This is especially useful when you store tensors in a compact dtype to save disk space, then convert at runtime.

Saving and loading with paths

Use pathlib.Path for robust path handling.

python
from pathlib import Path
import torch
path = Path("artifacts") / "tensor_cache.pt"
path.parent.mkdir(parents=True, exist_ok=True)
torch.save(torch.arange(10), path)
t = torch.load(path)

CPU tensors, GPU tensors, and portability

Saving GPU tensors is allowed, but for maximum portability you can move to CPU before saving.

python
x = x.detach().cpu()
torch.save(x, "x_cpu.pt")

This is a good default when saving data artifacts that will be shared across different environments.

If a tensor requires gradients, saving it will also save enough information to restore it as a tensor with requires_grad set. For cached data and features, you usually want detached tensors. Use x = x.detach() before saving.

Memory considerations when loading large tensors

torch.load will materialize tensors in memory. For very large saved tensors, ensure you have enough RAM or GPU memory for the loaded object. A simple practical tactic is to store separate files for very large components so you can load only what you need, for example features.pt and labels.pt instead of one combined file.

When not to use `torch.save` for tensors

torch.save is excellent for PyTorch workflows, but it is not the best interchange format for non PyTorch environments. If you need to share arrays with other tools, you might prefer NumPy formats. In that case you can convert with tensor.cpu().numpy() and use NumPy saving, but that is outside the scope of this chapter.

Quick checklist

Rule of thumb: save detached CPU tensors for portability, and always use map_location when loading in code that may run on different devices.

Views: 0

Comments

Please login to add a comment.

Don't have an account? Register now!

Where to Move

Move chapter:

  • ☰ 1. Orientation and Learning Setup
    • ☰ 1.1. Course Goals and How to Use This Outline
      • ☰ 1.2. What You Need to Know Before Starting
        • ☰ 1.3. Installing Python and PyTorch
          • ☰ 1.4. Choosing CPU vs CUDA vs Apple Silicon
            • ☰ 1.5. Using Jupyter, VS Code, and Scripts
              • ☰ 1.6. Reproducibility, Seeds, and Determinism
              • ☰ 2. Python and PyTorch Fundamentals
                • ☰ 2.1. Tensors and Basic Operations
                  • ☰ 2.2. Tensor Shapes, Dtypes, and Devices
                    • ☰ 2.3. Indexing, Slicing, and Broadcasting
                      • ☰ 2.4. Autograd and Computational Graphs
                        • ☰ 2.5. Common Tensor Pitfalls and Debugging
                          • ☰ 2.6. Saving and Loading Tensors
                          • ☰ 3. Neural Network Building Blocks
                            • ☰ 3.1. Linear Layers and Affine Transforms
                              • ☰ 3.2. Activation Functions
                                • ☰ 3.3. Loss Functions
                                  • ☰ 3.4. Optimizers and Learning Rates
                                    • ☰ 3.5. Regularization Basics
                                      • ☰ 3.6. Weight Initialization Concepts
                                      • ☰ 4. First Models End to End
                                        • ☰ 4.1. A Simple Regression Model
                                          • ☰ 4.2. A Simple Classification Model
                                            • ☰ 4.3. Data Splitting and Validation Sets
                                              • ☰ 4.4. Training Loop Structure
                                                • ☰ 4.5. Tracking Loss and Metrics
                                                  • ☰ 4.6. Basic Overfitting and Underfitting Checks
                                                  • ☰ 5. Data Handling with PyTorch
                                                    • ☰ 5.1. Datasets and DataLoaders
                                                      • ☰ 5.2. Transforms and Preprocessing
                                                        • ☰ 5.3. Collate Functions and Batching
                                                          • ☰ 5.4. Shuffling, Sampling, and Class Imbalance
                                                            • ☰ 5.5. Working with Tabular, Text, and Image Data
                                                              • ☰ 5.6. Data Pipeline Performance Basics
                                                              • ☰ 6. Training Loops in Practice
                                                                • ☰ 6.1. Forward Pass and Backward Pass
                                                                  • ☰ 6.2. Gradient Zeroing and Accumulation
                                                                    • ☰ 6.3. Device Placement and Mixed Precision Basics
                                                                      • ☰ 6.4. Learning Rate Schedules
                                                                        • ☰ 6.5. Gradient Clipping
                                                                          • ☰ 6.6. Logging with TensorBoard
                                                                          • ☰ 7. Debugging and Model Troubleshooting
                                                                            • ☰ 7.1. Verifying Shapes and NaNs
                                                                              • ☰ 7.2. Checking Gradients and Vanishing Gradients
                                                                                • ☰ 7.3. Sanity Checks and Overfit a Small Batch
                                                                                  • ☰ 7.4. Common Causes of Training Instability
                                                                                    • ☰ 7.5. Interpreting Learning Curves
                                                                                      • ☰ 7.6. Practical Tips for Faster Iteration
                                                                                      • ☰ 8. Feedforward Networks for Structured Data
                                                                                        • ☰ 8.1. Feature Scaling and Normalization
                                                                                          • ☰ 8.2. Embeddings for Categorical Features
                                                                                            • ☰ 8.3. Multi-Layer Perceptrons in PyTorch
                                                                                              • ☰ 8.4. Choosing Hidden Sizes and Depth
                                                                                                • ☰ 8.5. Evaluation Metrics for Classification and Regression
                                                                                                  • ☰ 8.6. Model Export and Inference Basics
                                                                                                  • ☰ 9. Convolutional Neural Networks for Images
                                                                                                    • ☰ 9.1. Convolutions, Padding, and Stride
                                                                                                      • ☰ 9.2. Pooling and Feature Maps
                                                                                                        • ☰ 9.3. Building a Basic CNN in PyTorch
                                                                                                          • ☰ 9.4. Data Augmentation for Vision
                                                                                                            • ☰ 9.5. Transfer Learning with Pretrained Models
                                                                                                              • ☰ 9.6. Fine-Tuning and Freezing Layers
                                                                                                              • ☰ 10. Recurrent and Sequence Models
                                                                                                                • ☰ 10.1. Sequences, Tokens, and Padding
                                                                                                                  • ☰ 10.2. RNNs, LSTMs, and GRUs
                                                                                                                    • ☰ 10.3. Handling Variable-Length Sequences
                                                                                                                      • ☰ 10.4. Teacher Forcing and Sequence Losses
                                                                                                                        • ☰ 10.5. Basic Text Classification Pipeline
                                                                                                                          • ☰ 10.6. Simple Sequence-to-Sequence Overview
                                                                                                                          • ☰ 11. Attention and Transformers Fundamentals
                                                                                                                            • ☰ 11.1. Why Attention Helps
                                                                                                                              • ☰ 11.2. Self-Attention Intuition
                                                                                                                                • ☰ 11.3. Transformer Building Blocks
                                                                                                                                  • ☰ 11.4. Using Transformer Models in PyTorch
                                                                                                                                    • ☰ 11.5. Fine-Tuning for Text Classification
                                                                                                                                      • ☰ 11.6. Managing Tokenizers and Batching
                                                                                                                                      • ☰ 12. Model Evaluation and Experiment Management
                                                                                                                                        • ☰ 12.1. Train, Validation, and Test Best Practices
                                                                                                                                          • ☰ 12.2. Cross-Validation Basics
                                                                                                                                            • ☰ 12.3. Calibration and Thresholding
                                                                                                                                              • ☰ 12.4. Error Analysis and Confusion Matrices
                                                                                                                                                • ☰ 12.5. Experiment Tracking Concepts
                                                                                                                                                  • ☰ 12.6. Comparing Models Fairly
                                                                                                                                                  • ☰ 13. Saving, Loading, and Deployment Readiness
                                                                                                                                                    • ☰ 13.1. Saving Model Weights and Full Checkpoints
                                                                                                                                                      • ☰ 13.2. Loading for Inference and Resuming Training
                                                                                                                                                        • ☰ 13.3. TorchScript and Tracing Basics
                                                                                                                                                          • ☰ 13.4. Exporting to ONNX Overview
                                                                                                                                                            • ☰ 13.5. Batch Inference and Latency Considerations
                                                                                                                                                              • ☰ 13.6. Reproducible Packaging of Code and Weights
                                                                                                                                                              • ☰ 14. Performance and Scaling Basics
                                                                                                                                                                • ☰ 14.1. Profiling and Bottleneck Identification
                                                                                                                                                                  • ☰ 14.2. DataLoader Optimization
                                                                                                                                                                    • ☰ 14.3. GPU Utilization and Memory Management
                                                                                                                                                                      • ☰ 14.4. Gradient Accumulation for Large Batches
                                                                                                                                                                        • ☰ 14.5. Distributed Training Concepts Overview
                                                                                                                                                                          • ☰ 14.6. Practical Guidelines for Faster Training
                                                                                                                                                                          • ☰ 15. Best Practices and Next Steps
                                                                                                                                                                            • ☰ 15.1. Choosing Hyperparameters Systematically
                                                                                                                                                                              • ☰ 15.2. Regularization and Generalization Toolkit
                                                                                                                                                                                • ☰ 15.3. Responsible Use and Dataset Considerations
                                                                                                                                                                                  • ☰ 15.4. Reading PyTorch Documentation Effectively
                                                                                                                                                                                    • ☰ 15.5. Suggested Projects for Beginners
                                                                                                                                                                                      • ☰ 15.6. Pathways to Advanced Topics