12. Model Evaluation and Experiment Management

Why evaluation and experiment management matter

Training a neural network is only half the job. The other half is knowing whether it truly learned something that will generalize to new data, and being able to reproduce and compare results without guesswork. Model evaluation is the set of practices that tell you how well a model performs and how reliable that performance is. Experiment management is the set of habits that let you change one thing at a time, keep records, and make fair comparisons so you can improve systematically instead of by accident.

For beginners, the biggest risk is not that the model performs poorly, but that you measure performance in a misleading way, such as evaluating on the same data you trained on, tuning hyperparameters based on the test set, or comparing runs that differ in multiple hidden ways. This chapter sets the map for the evaluation and tracking topics that follow in this section.

The three roles of data: train, validation, and test

In supervised learning, you typically separate data into three conceptual roles. Training data is used to fit model parameters, meaning the weights updated by gradient descent. Validation data is used to make decisions about the training process and design choices, such as selecting hyperparameters, picking an epoch via early stopping, choosing architectures, and deciding thresholds. Test data is used only once you are done making choices, as an estimate of real world performance.

A helpful mental model is that the validation set is for learning about your process, while the test set is for judging your final product. When you repeatedly look at test performance and adjust your approach, the test set stops being a clean estimate because your choices become indirectly tailored to it.

What “good evaluation” looks like

Good evaluation is consistent, comparable, and relevant. Consistent means the same preprocessing, the same metric definitions, and the same data splits when you want to compare models. Comparable means you control randomness and record enough details that you can reproduce results or explain why they differ. Relevant means the metric reflects the task objective, such as accuracy for balanced classification, F1 for class imbalance, mean absolute error for robust regression, or calibrated probabilities when decisions depend on confidence.

In practice, evaluation quality often depends more on the experimental setup than on any single metric. A slightly less perfect metric measured correctly is more useful than a sophisticated metric measured with leakage or inconsistent splits.

Data leakage and subtle evaluation traps

Data leakage happens when information from outside the training data influences training in a way that would not be available in production. Leakage can be obvious, such as training on the test set, but it is often subtle, such as fitting normalization statistics on the full dataset before splitting, computing target dependent features using the whole dataset, or applying text vocabulary building using all data including validation and test.

Another common trap is repeated validation probing. If you evaluate on the validation set thousands of times while tuning many choices, the validation set becomes less representative. That is why this section later discusses best practices, and in some settings, cross validation.

Metrics as functions of predictions

At a high level, a metric is a function that maps model outputs and targets to a number. For regression you might compute error magnitudes, for classification you compare predicted labels or probabilities to true labels. The key is that you must define exactly what the model outputs mean at evaluation time. Some models output logits, not probabilities. Some pipelines output per class scores that need a softmax. Some tasks need thresholds, top k decisions, or cost sensitive tradeoffs. Without a precise definition, two runs can report “accuracy” that is not actually computed the same way.

When metrics involve thresholds, such as turning probabilities into positive or negative predictions, the threshold is part of the model selection process and typically belongs to validation, not test. When metrics involve averaging across batches, you should make sure you are computing a dataset level metric rather than a simple average of per batch metrics that may weight batches unevenly.

Uncertainty, variance, and why one number is not enough

A single score from a single split is a sample, not a truth. Performance varies due to random initialization, minibatch order, and how the split happens. That variance is often large for small datasets. You can manage this by using fixed splits for fair comparisons, running multiple seeds and reporting mean and standard deviation, or using cross validation. The goal is not to eliminate randomness, but to measure and control it so you can distinguish real improvements from noise.

A practical rule is that if a change improves validation performance by a tiny amount that is smaller than the typical run to run variation, you should be skeptical until you confirm with repeated runs or stronger evidence.

Fair comparisons between experiments

To compare models fairly, you want to change one thing at a time and keep everything else constant. This includes the data split, preprocessing, training epochs, optimizer settings, batch size, and random seeds. If you change multiple things at once, you may get a better score but you will not know why.

Fair comparison also includes using the same evaluation protocol. If one run reports the best validation epoch and another reports the last epoch, the results are not comparable. If one run uses a different threshold, it is not comparable. If one run uses test time augmentation or an ensemble, it is not comparable to a single model unless you state that clearly.

Experiment tracking as a habit, not a tool

Tools like TensorBoard, Weights and Biases, MLflow, or simple CSV logs can help, but the core is the habit of recording what you did. At minimum, each run should have a unique identifier and a record of the dataset version, split strategy, model architecture choices, preprocessing, training configuration, random seed, and the resulting metrics. Without that, you cannot learn efficiently from your own work because you cannot reliably repeat or extend a result.

A lightweight approach is to save a config object for every run and write metrics to a structured file. Even if you later adopt a tracking platform, the mental model stays the same, you are building a history of decisions and outcomes.

Choosing evaluation frequency and avoiding feedback loops

Evaluating too frequently can slow training and can encourage over tuning to the validation set. Evaluating too rarely can hide issues such as divergence, overfitting, or data pipeline bugs. A common approach is to evaluate once per epoch for many tasks, then use additional checks such as a small subset evaluation for quick iteration.

When you monitor metrics during training, you are creating a feedback loop, you observe, adjust, and try again. That is normal. The key is to keep the test set outside this loop. The validation set is inside the loop, so you should treat it as a resource you can overuse. This is one reason why later topics like cross validation, calibration, and error analysis are valuable, because they help you learn more from the same data without biasing your final test estimate as much.

Connecting evaluation to decisions

Evaluation is not only about reporting. It is about decisions. You will use evaluation to decide whether to collect more data, change the loss function, adjust class weighting, tune learning rates, redesign the architecture, or change the threshold. Different decisions require different views of performance. Aggregate metrics tell you overall quality, while error analysis reveals what is failing, such as specific classes, rare cases, or particular input conditions.

In this section of the course, the upcoming chapters will focus on specific best practices and techniques, including train validation test procedures, cross validation, calibration and thresholding, error analysis with confusion matrices, experiment tracking concepts, and how to compare models fairly. This chapter’s goal is to anchor those topics in the core idea: evaluation is a disciplined protocol, and experiment management is the structure that makes progress repeatable.

Views: 98