KAHIBARO
42.6. Statistical Validation
Table of Contents
Number of events
Every GATE simulation is based on Monte Carlo sampling, so results always contain statistical noise. For beginners, the first practical question is usually: how many events are enough? There is no universal number. The right choice depends on how small you need your statistical uncertainty to be, and in which quantity.
In practice, one event in GATE usually corresponds to one primary particle, for example one gamma photon, one positron decay, or one proton entering a phantom. When you score any quantity (counts, dose, energy, fluence) in a voxel or detector element, you are collecting contributions from many independent events. Under typical conditions, the number of counts in a given bin follows Poisson statistics.
If a bin (for example, a dose voxel or a detector channel) receives $N$ independent counts, the expected statistical relative uncertainty in that bin behaves approximately like $1 / \sqrt{N}$. This means that to reduce the uncertainty by a factor of 10, you must increase the number of contributing events by a factor of 100. This scaling has important consequences when planning simulations.
For Poisson-distributed counts $N$:
$$\sigma_N = \sqrt{N}, \quad \text{relative uncertainty} = \frac{\sigma_N}{N} = \frac{1}{\sqrt{N}}.$$
To divide the relative uncertainty by a factor $k$, you must multiply $N$ by $k^2$.
In validation, you usually do not care about the total number of primaries on its own. Instead, you care about the number of useful events in the region or quantity you are comparing with reference data. For example, you might run $10^9$ primaries, but if only $10^3$ events reach a small region of interest, the statistical uncertainty there will still be large. It is therefore good practice to check statistics specifically in:
- Regions of interest, such as a tumor volume, central voxels of a depth dose curve, or the field of view of a detector.
- Tails and low-signal regions of distributions, because there the number of counts can be small even in a large overall simulation.
GATE actors often give you direct access to the number of events per bin. For example, dose actors can output both the dose and its uncertainty, from which you can infer the effective number of contributing events in each voxel. If only a few voxels in a profile show very large uncertainties, it is possible that the total number of events is adequate globally, but not for those specific regions.
To choose an initial number of events, a simple and practical workflow is:
- Run a short test simulation with a modest number of primaries.
- Inspect the statistical uncertainty in the quantities you will use for validation, such as depth dose curves or energy spectra.
- Estimate how much you need to increase the number of events to reach a target uncertainty. If the relative uncertainty in a bin is currently $u_\text{current}$ and your target is $u_\text{target}$, then the required factor in the number of events is approximately:
$$\left(\frac{u_\text{current}}{u_\text{target}}\right)^2.$$ - Repeat with the increased number of events and confirm that the uncertainty is now acceptable.
This approach avoids running overly long simulations without evidence that you actually need the extra statistics.
Monte Carlo uncertainty
Statistical validation requires explicit attention to Monte Carlo uncertainty. When you compare simulation results with analytical models or experimental measurements, you must account for the fact that your simulated results have finite precision due to the limited number of sampled events.
For many quantities in radiation transport and imaging, the assumption of Poisson statistics is a good approximation. If a bin in a histogram or image has $N$ independent counts, the standard deviation of that count is $\sqrt{N}$. The same logic extends to quantities that are scaled by a constant factor, such as dose per primary or counts per unit time, as long as you correctly propagate the scaling.
For example, if you estimate the mean of a quantity $x$ from $N$ independent samples $x_i$ as:
$$\bar{x} = \frac{1}{N} \sum_{i=1}^{N} x_i,$$
the standard error on the mean is:
$$\sigma_{\bar{x}} = \frac{\sigma_x}{\sqrt{N}},$$
where $\sigma_x$ is the standard deviation of the underlying distribution of $x$. In simple counting problems, $x_i$ can be 0 or 1, and this formula reduces to the familiar Poisson expression.
For a Poisson count $N$ in a bin:
$$\sigma_N = \sqrt{N}, \quad \sigma_\text{rel} = \frac{1}{\sqrt{N}}.$$
For a scaled quantity $Q = c \, N$:
$$\sigma_Q = c \sqrt{N}, \quad \sigma_{Q,\text{rel}} = \frac{1}{\sqrt{N}}.$$
In GATE, many actors directly provide both the scored value and its statistical uncertainty. For instance, dose actors can output a dose image and a corresponding uncertainty image. The voxel-by-voxel relative uncertainty is then simply:
$$u_\text{rel} = \frac{\sigma_\text{dose}}{\text{dose}}.$$
When you perform validation, it is important to overlay or visualize these uncertainties alongside the dose or count distributions. Two common patterns you will see are:
- Low statistical uncertainty in high-dose or high-count regions, where many events contribute.
- High statistical uncertainty in low-dose or low-count regions, such as deep tails of a depth dose curve or edges of a field.
When comparing a simulated dose profile to a reference measurement, you should interpret differences in light of these uncertainties. If the difference between simulation and measurement in a voxel is smaller than about two times the Monte Carlo standard deviation, it can be explained by statistical noise alone. Larger systematic differences that persist even with small statistical uncertainty indicate potential modeling or geometry problems.
Monte Carlo uncertainty also affects derived quantities. For example, if you determine a Bragg peak position by fitting a depth dose curve, the fit parameters will have uncertainties that depend on the statistical uncertainties of the dose values. Similarly, when you compute global metrics for validation, such as gamma index passing rates or average relative error, you should remember that these metrics are themselves affected by Monte Carlo noise in each bin.
A practical approach for beginners is to define a target relative uncertainty in the key validation quantity, often in the range of 1 to 3 percent in the region of interest, then adjust the number of events until the GATE uncertainty outputs confirm that you are close to this target. For less critical regions or for initial development work, larger uncertainties may be acceptable, which will save computation time.
Finally, when you repeat simulations with different random seeds, you can use the variation between runs as an independent check of the estimated Monte Carlo uncertainty. If the run-to-run variations are consistent with the uncertainties reported by GATE, this supports the reliability of your statistical validation.
Views: 9