Table of Contents
What this section is for
This section is your foundation for everything that comes later in the course. Deep learning in PyTorch is mostly about manipulating tensors, moving them across devices, and letting PyTorch compute gradients so your model can learn. If you are comfortable with these basics, the later chapters about models, training loops, and debugging become much easier because you can reason about shapes, types, and where values come from.
You will not build full neural networks yet. Instead, you will learn the small set of core ideas that show up in every real project: how tensors behave, how their metadata affects computation, how indexing and broadcasting work, how automatic differentiation works, and how to save intermediate results safely.
How to study this section
Read and run the code as you go. Almost every beginner problem in PyTorch is a mismatch between what you think a tensor looks like and what it actually looks like. The fastest way to learn is to print shapes, dtypes, and devices constantly until it becomes second nature. When something fails, you should be able to answer three questions immediately: what is the tensor shape, what is the dtype, and what device is it on.
As you work through these chapters, keep a small scratch file or notebook where you try tiny experiments. For example, create two tensors, add them, slice them, call .backward(), and inspect .grad. The goal is not to memorize function names, it is to build intuition about what PyTorch is doing.
The mental model you are building
PyTorch code is usually organized around a few repeated steps. You create tensors to represent data and parameters. You apply operations to build a computation that produces outputs and a loss. PyTorch records those operations so it can compute gradients. You use gradients to update parameters. Even before you learn training loops, this section teaches the two key layers of that process: tensor computation and gradient computation.
A tensor is not just numbers. It is numbers plus important metadata. Shape tells you how many dimensions and how big each dimension is. Dtype tells you how values are represented, such as floating point or integer. Device tells you where the tensor lives, such as CPU or GPU. Many errors are just violations of expectations about this metadata.
Autograd is the other core idea. When you mark tensors as requiring gradients and use differentiable operations, PyTorch builds a computational graph. Calling .backward() asks PyTorch to compute derivatives of a scalar loss with respect to any tensor that participated in that graph and has requires_grad=True.
Rule to internalize: always track shape, dtype, and device. Most PyTorch bugs are one of these three mismatching what an operation expects.
What you will be able to do after this section
After completing this section’s chapters, you should be able to create tensors, reshape them, and perform common arithmetic and linear algebra operations. You should be able to control dtype and device deliberately, and understand why that matters. You should be able to use indexing, slicing, and broadcasting without guessing. You should understand what it means for a tensor to require gradients, what .backward() does, and why gradients accumulate. You should be able to debug the most common beginner issues such as shape mismatches, accidental integer tensors in places that need floats, and saving and loading tensors reliably.
This is the minimum toolkit you need before you start building models. Once these fundamentals feel natural, the rest of the course becomes less about fighting errors and more about learning modeling ideas.