KAHIBARO
40.5. Optimizing Output
Table of Contents
Reducing unnecessary data
Output configuration has a very large impact on both speed and memory usage in GATE. Every extra variable you save must be written by many threads, buffered, sent to disk, and later read back for analysis. For large simulations the main bottleneck is often not physics but I/O.
The first principle is to decide in advance which physical questions you want to answer. From that, derive the minimum set of actors and branches you truly need. Recording everything “just in case” is almost always a mistake for large Monte Carlo runs.
A practical strategy is to start with a rich output configuration in a small test run, analyze which quantities you actually use, then remove unused branches and actors before launching production runs. For ROOT outputs this typically means carefully choosing which branches are created by each actor and which variables are stored in each branch. For image outputs it means limiting dose or fluence maps to the regions and resolutions that matter for your analysis.
Many actors allow you to control their detail level. A phase space actor, for instance, can often be configured to record only position and energy instead of including direction, particle history, and step information. A dose actor can be configured with a coarser voxel grid if you do not need fine spatial resolution. Similarly, for hits and singles you can disable rarely used fields such as detailed process flags or parent IDs when you do not intend to use them.
Filters are another powerful way to avoid unnecessary data. You can attach volume filters so that an actor records only interactions in specific detector elements or organ volumes. You can also use particle filters to keep only gammas or protons, and energy filters to discard very low energy deposits that are irrelevant for your observable. This avoids generating long lists of entries that will be discarded later in analysis and directly reduces both the number of records and their size.
It is also common to split a large simulation into multiple targeted runs instead of collecting “everything” at once. For example, you might run one simulation that produces detailed phase-space data at a scoring plane, and a second simulation that uses that phase space as input and only records coincedences. Each run then outputs only what is necessary for its specific purpose.
Time-resolved output requires special care. If you enable very fine temporal binning for dose or fluence images, the number of output images can grow quickly, and each can be large. Before enabling such features, check whether you really need that temporal resolution or whether a few time points or a single integrated result are sufficient.
Finally, remember that output operations slow down multithreaded execution because they must be synchronized when writing to shared files. Fewer, smaller writes usually give better scalability. When possible, prefer actors and formats that buffer data per thread and merge only occasionally, rather than writing at every step or event.
Only record the quantities that are directly needed to answer your scientific question. Use actor filters, reduced branch content, and coarser grids to cut unnecessary data at the simulation stage.
File sizes
Once you have minimized which data are produced, the next concern is how large each output file becomes. Large ROOT trees or 3D images consume disk space quickly and can make later analysis slow or impractical.
The three main contributors to file size are the number of entries, the number of variables per entry, and the precision used to store each variable. Reducing entries and variables is covered by output reduction. Precision control is an additional tool. Many quantities do not require double precision. Configuring actors to store floats instead of doubles, or integers where appropriate, can significantly shrink files while preserving all information that matters physically.
Spatial resolution and field of view have a strong effect on image file sizes. A 3D dose map with $512 \times 512 \times 512$ voxels produces 134 million voxels for a single image. If you instead use $256 \times 256 \times 256$ voxels, the number of voxels decreases by a factor of 8. Before choosing a voxel grid, consider the smallest feature you need to resolve and scale the grid accordingly. For many problems a moderate voxel size is fully adequate.
You can also restrict grids and actors to smaller regions. Instead of scoring dose or fluence over the full world volume, limit the scoring region to the phantom, patient, or detector. This reduces the number of voxels and the amount of memory that must be allocated and later written to disk.
ROOT files support compression, which is typically enabled by default. Compression trades CPU time for smaller files. In multithreaded simulations this extra CPU cost is usually acceptable. You can often adjust the compression level to find a good compromise between size and speed. Higher compression levels mean smaller files but slower writing. For extremely large trees it is usually beneficial to keep a moderate compression level that balances both aspects.
For ROOT trees with many branches, splitting and basket sizes also influence file size and read performance. A higher split level organizes branches more effectively, which can help when reading only a subset of variables later. Appropriate basket sizes can reduce overhead. These are advanced configuration knobs, but they can make a difference in large-scale production.
Some image formats, such as MHD and NIfTI, allow compression through associated compressed data files. When writing dose or activity images, enabling compression will reduce disk usage without changing the contained values. For long-term storage and sharing of results compressed images are usually preferable.
Managing file sizes also includes structuring your results. It is often better to produce several smaller files than one enormous file. For instance, you might write separate ROOT files per run, per slice, or per projection angle. This allows you to analyze or move a subset of data without handling a massive monolithic file. It can also make it easier to resume or repeat only specific portions of a study.
Before launching large campaigns on HPC systems, estimate expected file sizes using smaller test runs. For example, run a short simulation, note the output size per event, and extrapolate to the planned number of events. Check that the resulting total fits within available storage quotas. If not, adjust your configuration while the simulation is still in the design phase.
Keep voxel grids as small as physically reasonable, use appropriate data precision and compression, and always estimate total file sizes from test runs before starting large simulations.
Views: 8