KAHIBARO
17.3 Statistical Uncertainties
Table of Contents
Counting statistics
In many ROOT analyses each bin of a histogram, or each event count in a region, represents a number of independent trials where an event either occurs or does not occur. In this situation the appropriate model is usually the Poisson distribution. If the expected mean number of events is $\mu$, then the probability to observe $n$ events is
$$
P(n \mid \mu) = \frac{\mu^n}{n!} e^{-\mu}.
$$
For Poisson statistics the variance of the observed count is equal to the mean:
$$
\mathrm{Var}(n) = \mu.
$$
When you measure $n$ events, your best estimate of the mean is $\hat{\mu} = n$, so the estimated variance and standard deviation of the count are
$$
\sigma_n^2 \approx n, \qquad \sigma_n \approx \sqrt{n}.
$$
For a simple event count with Poisson statistics, the statistical uncertainty on the count is
$$\Delta n = \sqrt{n}.$$
This rule is used everywhere in ROOT-based analyses. For example, the vertical error bar on a histogram bin content $N_{\text{bin}}$ is by default taken as $\sqrt{N_{\text{bin}}}$, if the histogram has been filled with unit weights.
When events are filled into a histogram with weights, for example using efficiency or luminosity weights, the situation changes. Let $w_i$ be the weight of the $i$-th event that falls into a particular bin, and let the bin content be
$$
B = \sum_{i=1}^{N} w_i.
$$
If the events are independent, the variance of the bin content is
$$
\sigma_B^2 = \sum_{i=1}^{N} w_i^2.
$$
In ROOT, when you call Sumw2() on a histogram before filling, the histogram stores this sum of squared weights and uses it to compute bin errors.
For a histogram with weights and Sumw2() enabled, the bin error is
$$\Delta B = \sqrt{\sum_{i} w_i^2}.$$
Without Sumw2(), ROOT assumes unit weights and sets $\Delta B = \sqrt{B}$.
For very small counts, such as $n = 0, 1, 2$, the approximation $\Delta n = \sqrt{n}$ becomes less accurate if you need precise confidence intervals. There are more refined formulas and tables for low-count Poisson uncertainties, but for many introductory analyses the $\sqrt{n}$ rule is sufficient.
When you add or subtract independent counts, their uncertainties combine accordingly. If you have two independent Poisson counts $n_1$ and $n_2$ with uncertainties $\sqrt{n_1}$ and $\sqrt{n_2}$, then for the sum $S = n_1 + n_2$,
$$
\sigma_S^2 = \sigma_{n_1}^2 + \sigma_{n_2}^2 = n_1 + n_2,
$$
so $\sigma_S = \sqrt{n_1 + n_2}$. For the difference $D = n_1 - n_2$ the same variance formula applies as long as the counts are independent:
$$
\sigma_D^2 = n_1 + n_2.
$$
These rules are built into ROOT operations on histograms. When you use methods like Add(), Add(h1,h2, a, b) or Divide(), ROOT automatically propagates the bin errors using the statistical formulas that correspond to independent quantities. This is one of the reasons to always let the histogram manage its errors correctly from the start.
Propagation of uncertainties
In a ROOT analysis you rarely use raw counts alone. You usually compute derived quantities, such as efficiencies, cross sections, normalized yields, ratios, and corrected measurements. Each of these quantities depends on one or more measured variables that have their own statistical uncertainties. To estimate the uncertainty on the derived quantity from the uncertainties of its inputs you use error propagation.
Suppose you have a quantity
$$
y = f(x_1, x_2, \dots, x_n),
$$
where each $x_i$ has an uncertainty $\sigma_{x_i}$. If the uncertainties are small and the $x_i$ are statistically independent, the variance of $y$ is approximated by
$$
\sigma_y^2 \approx \sum_{i=1}^{n} \left( \frac{\partial f}{\partial x_i} \right)^2 \sigma_{x_i}^2.
$$
For a function $y = f(x_1,\dots,x_n)$ of independent variables, the standard error propagation formula is
$$
\sigma_y^2 = \sum_{i}\left(\frac{\partial f}{\partial x_i}\right)^2 \sigma_{x_i}^2,
$$
with partial derivatives evaluated at the measured values.
This formula is the basis for almost all analytic uncertainty calculations in simple ROOT macros. Below are some very common special cases that appear frequently in analyses.
If you have a sum or difference
$$
y = x_1 \pm x_2,
$$
then
$$
\sigma_y^2 = \sigma_{x_1}^2 + \sigma_{x_2}^2.
$$
If you have a product or a ratio
$$
y = x_1 x_2 \quad \text{or} \quad y = \frac{x_1}{x_2},
$$
the relative uncertainties combine approximately as
$$
\left( \frac{\sigma_y}{y} \right)^2 = \left( \frac{\sigma_{x_1}}{x_1} \right)^2 + \left( \frac{\sigma_{x_2}}{x_2} \right)^2.
$$
A particularly important case is the efficiency or fraction
$$
\epsilon = \frac{N_{\text{pass}}}{N_{\text{total}}},
$$
where $N_{\text{pass}}$ events pass a selection out of $N_{\text{total}}$ total events. If each event either passes or fails, and events are independent, the number of passing events follows a binomial distribution. The binomial variance for $\epsilon$ is
$$
\sigma_\epsilon^2 = \frac{\epsilon (1 - \epsilon)}{N_{\text{total}}}.
$$
This result can also be derived from error propagation using $N_{\text{pass}}$ and $N_{\text{total}}$ and the binomial model. In ROOT, when you compute an efficiency with classes designed for that purpose, such as TEfficiency, this binomial uncertainty is what is usually used.
In many situations you need to propagate uncertainties numerically rather than analytically. For example, if $f$ is complicated or depends on the result of a ROOT fit, you can use the fit covariance matrix. If a fit returns parameters $p_i$ with covariance matrix elements $V_{ij}$, and you define
$$
y = g(p_1, p_2, \dots),
$$
the general propagation formula with correlations is
$$
\sigma_y^2 = \sum_{i,j} \frac{\partial g}{\partial p_i} \frac{\partial g}{\partial p_j} V_{ij}.
$$
ROOT stores this covariance matrix inside fit results and makes it accessible, so later chapters that work with fits use this formula implicitly when computing uncertainties on derived quantities.
Although systematic uncertainties are conceptually different from statistical ones, the same mathematical propagation rules apply to them. You usually evaluate how a given systematic effect shifts your result, interpret that shift as an uncertainty contribution, and combine several independent contributions in quadrature. The key distinction is that statistical uncertainties decrease when you collect more data, while systematic contributions usually do not.
In practice, when writing ROOT code for statistical uncertainties, you will often rely on ROOT to handle propagation when combining histograms or using built in classes for efficiencies and fits. Understanding the analytic formulas remains important, because it tells you when ROOT's default behavior is appropriate and when you must implement your own uncertainty calculations for specific derived quantities.
Views: 11