Table of Contents
The Goal of This Chapter
Regression means predicting a real number, like a house price or a temperature. In this chapter you will build your first complete PyTorch model that learns a simple numeric relationship from data, train it with a basic loop, and confirm that it is actually learning. The focus here is on an end to end example, not on explaining tensors, autograd, optimizers, or loss functions in depth, since those have their own chapters.
A Tiny Regression Dataset
To keep the first example clear, we will generate synthetic data where the true relationship is linear. The model’s job is to recover that relationship from noisy observations.
We will create inputs $x$ and targets $y$ with
$$y = 3x + 2 + \epsilon$$
where $\epsilon$ is small random noise.
For regression, your targets should be floating point tensors, typically torch.float32. If you accidentally use integer targets, training can behave incorrectly or silently underperform.
Implementing a Minimal Regression Model
A linear regression model is just an affine transform, $y = wx + b$. In PyTorch, nn.Linear(1, 1) represents exactly that mapping from one input feature to one output value.
Set a seed for repeatability, create the dataset tensors, define the model, pick a loss, and pick an optimizer.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Synthetic dataset
N = 200
x = torch.rand(N, 1) * 2 - 1 # values in [-1, 1], shape [N, 1]
noise = 0.1 * torch.randn(N, 1)
y = 3.0 * x + 2.0 + noise # shape [N, 1]
# Model: y_hat = w*x + b
model = nn.Linear(1, 1)
# Loss and optimizer
loss_fn = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
Shape consistency matters. Here both x and y are shaped [N, 1]. If you use [N] for one and [N, 1] for the other, broadcasting can happen and produce a loss value that looks valid but is not what you intended.
A First Training Loop
Training repeats three core steps: compute predictions, compute loss, update parameters. The important detail is that PyTorch accumulates gradients by default, so you must zero them each iteration.
epochs = 200
for epoch in range(epochs):
# Forward pass
y_hat = model(x)
loss = loss_fn(y_hat, y)
# Backward pass and update
optimizer.zero_grad()
loss.backward()
optimizer.step()
if (epoch + 1) % 50 == 0:
print(f"epoch {epoch+1:3d} | loss = {loss.item():.6f}")
Always call optimizer.zero_grad() before loss.backward() in a standard training loop. If you forget, gradients accumulate across epochs and your updates will be wrong.
Checking What the Model Learned
After training, you can inspect the learned parameters. For nn.Linear(1, 1), weight has shape [1, 1] and bias has shape [1].
w = model.weight.item()
b = model.bias.item()
print("learned w:", w)
print("learned b:", b)
If training worked, w should be close to 3 and b should be close to 2, not exact because of noise and limited data.
Making a Few Predictions
To see the model behave like a function, feed a few values of $x$ and print the predicted $y$.
model.eval()
with torch.no_grad():
x_test = torch.tensor([[-1.0], [0.0], [1.0]])
y_pred = model(x_test)
print(torch.cat([x_test, y_pred], dim=1)) # columns: x, predicted y
When you are evaluating or doing inference, use model.eval() and wrap the forward pass in torch.no_grad(). This prevents unnecessary gradient tracking and makes results more predictable when you later introduce layers that behave differently during training.
What You Have Now
You have a complete regression workflow: data, model, loss, optimizer, training loop, and a simple evaluation step. The same skeleton will carry over to more complex models, larger datasets, and different tasks, with changes mainly in the model definition, the loss function, and how you feed data in batches.