29.5. Multithreading

Table of Contents

Multithreading in Geant4 Performance Tuning

Multithreading is one of the most effective ways to accelerate Geant4 simulations on modern CPUs. In this chapter you will learn what changes when Geant4 is run with multiple threads, how this affects performance, and what you should do in your code to take full advantage of it without introducing subtle bugs.

Role of Multithreading in Performance

A Geant4 simulation typically spends most of its time transporting particles through geometry and processing physics interactions. Each event is statistically independent of the others, so different events can be simulated at the same time on different CPU cores.

Multithreading lets Geant4 create several worker threads. Each worker simulates different events, using the same geometry and physics configuration. On a multi core processor this reduces the wallclock time for a given number of events, sometimes nearly in proportion to the number of cores, provided that your code and analysis are thread safe.

Key rule: In multithreaded Geant4, events are independent, and each worker thread has its own copy of user initializations and actions. Never assume a single global object is used by all threads unless it is designed to be shared safely.

If your simulation is CPU dominated and not strongly limited by input or output, enabling multithreading is usually the simplest way to improve performance.

Master and Worker Threads

Geant4 uses a master worker model for multithreading. This model is central to understanding where and how your user code runs.

Master Thread Responsibilities

The master thread is created first. It is responsible for:

Configuring and constructing the overall simulation. It builds the geometry and physics list once in a special master context.

Managing worker threads. It starts, coordinates, and stops them.

Running master side user actions. For example, a BeginOfRunAction and EndOfRunAction can be executed in a master specific context, separate from per thread actions.

The master thread does not usually process events itself. Instead, it distributes event ranges to workers, which then perform event level simulation.

Worker Thread Responsibilities

Each worker thread is responsible for:

Owning its own copies of user geometry, physics, and user action classes. The framework clones or re creates these per thread.

Simulating a subset of events. Each worker works independently on different events.

Collecting per thread results, such as histograms or ntuple rows, that will later be merged or written.

Geant4 internally ensures that worker threads can operate concurrently without modifying shared internal state. However, your own code must respect the same principle.

Important: Never modify geometry or physics from worker threads during a run. Geometry and physics must be fully defined before /run/initialize, and then treated as read only during event processing.

How Geant4 Uses Multiple Threads

When you build your application with multithreading enabled and use G4MTRunManager instead of the serial run manager, Geant4 creates one master and a number of worker threads.

The core behavior is:

You configure the simulation and call /run/initialize in the usual way.
You set the number of threads, for example from C++ or with a macro command.
When you call /run/beamOn N, the run manager divides the requested number of events among worker threads.
Each worker processes its events completely independently, including particle tracking, user actions, and analysis filling.
At the end of the run, per thread results are combined in the master, then written or finalized.

Because each worker has its own copies of user classes, you do not need to manually fork or manage threads. You only need to ensure that your code is written in a thread safe way.

Setting the Thread Count

The number of threads can be controlled either in C++ in your main() or using macro commands.

At the C++ level, a typical pattern in main() is:

Use G4MTRunManager instead of G4RunManager when multithreading is enabled.
Call SetNumberOfThreads() before /run/initialize or Initialize().

For example, you might allow the user to specify the thread count in a macro, or you might detect the hardware concurrency and choose a default value.

Using too many threads can increase memory consumption and sometimes reduce performance if there are not enough physical cores. A good starting point is one thread per physical core, then adjust based on measurements.

In an interactive or batch macro you can set the threads with a command before initialization. You should not change the thread count during a run.

Rule: Set the number of threads before starting the run. Do not change the thread count between /run/beamOn calls without reinitializing as requested by Geant4.

Writing Thread Safe User Code

Thread safety is critical for correct multithreaded simulations. In Geant4, event level code is executed in several threads in parallel, so any shared data structures that are modified must be protected or avoided.

Local vs Shared Objects

Anything that is a data member of a user action class, such as RunAction, EventAction, or SteppingAction, is per thread, because each worker owns its own instance. This is usually safe and simple.

Objects that are declared as global variables, function static variables, or singletons that you create yourself, may be shared by all threads, which is dangerous if they are written from multiple threads.

As a rule of thumb, favor:

Local variables inside functions and methods.

Class member variables in user actions and detector or physics classes.

Objects allocated and owned by G4AnalysisManager and created through its interface, which are designed for multithreaded use.

Avoid direct use of shared global objects unless they are constant or protected by thread synchronization primitives, which is advanced and usually unnecessary for beginning users.

Avoiding Race Conditions

A race condition occurs when two threads modify or depend on the same piece of data without proper synchronization. In Geant4 this often happens when:

You increment a global counter from several threads.

You append to a standard library container that is shared between threads, such as a global std::vector.

You write to the same output file from more than one thread at a time.

To avoid this, keep per thread tallies inside per thread classes, such as RunAction or EventAction. Let Geant4 or G4AnalysisManager handle merging at the end of the run, rather than writing from many threads yourself.

Never write directly to the same file from multiple threads. Use the Geant4 analysis system or merge per thread results in the master and write from a single thread.

Geometry and Physics Modifications

Geometry objects and physics lists are replicated per thread for performance and safety. However, they are created from shared templates defined by the master. You must not modify geometry or physics during event processing, either from master or workers.

All DetectorConstruction and physics list configuration should be completed before /run/initialize. After that, treat these as read only for the remainder of the run.

Analysis and Output with Multithreading

Output and analysis usually require special attention in a multithreaded application. The Geant4 analysis manager is designed to simplify this.

Per Thread Accumulation

Each worker thread has its own G4AnalysisManager instance where histograms and ntuples are filled. This is automatic when you use the standard analysis interface.

During event processing, threads fill their own local analysis objects. At the end of the run, the analysis system merges these per thread objects into a single final result in the master, then writes the output file.

This model avoids races and makes it easy to write correct analysis code without manual synchronization.

When to Write Results

You should open and close your analysis output through G4AnalysisManager in the run actions. In multithreaded mode, a typical pattern is:

Open the file in the master or in BeginOfRunAction.

Fill histograms and ntuples from workers during events.

Merge and write final results in EndOfRunAction, where the master will finalize the output.

Do not open multiple different output files from worker threads unless you know exactly what you are doing. Let the analysis manager coordinate all of the writing.

Always let G4AnalysisManager handle merging and writing in multithreaded mode. Avoid manual file I/O in event, tracking, or stepping actions, because this can cause race conditions and corrupted files.

Measuring and Tuning Multithreaded Performance

Enabling multithreading is only the first step. To really improve performance you should measure and tune.

Speedup and Scaling

A useful quantity is the speedup factor when using multiple threads:

$$
S = \frac{T_1}{T_N}
$$

where $T_1$ is the time with one thread, and $T_N$ is the time with $N$ threads.

In an ideal case, $S$ is close to $N$. In practice, you may see less than linear scaling because of:

Shared resources such as memory bandwidth or cache.

Time spent in the master or in non parallelizable parts of the code.

Input and output overhead.

Differences in event complexity.

If scaling is poor, first check for unnecessary I/O, verbose logging, or shared data structures. These are frequent bottlenecks in multithreaded Geant4 applications.

Balancing Threads and Memory

Each worker thread maintains its own copies of geometry, physics objects, and analysis data. This increases memory usage roughly in proportion to the number of threads.

If your geometry is complex or your analysis includes many histograms and ntuples, a large thread count can exhaust available memory or cause excessive paging, which drastically reduces performance.

The practical steps are:

Measure memory usage for 1 thread.

Estimate memory usage times the number of threads and compare to available physical memory.

Reduce thread count or simplify geometry or analysis if memory is tight.

Minimizing Synchronization Overhead

If you introduce your own locks or mutexes for shared data, excessive synchronization can negate the benefits of multithreading. Try to:

Design code so that each thread works on independent data as much as possible.

Restrict any necessary locking to rare or short operations.

Avoid locking inside stepping or tracking actions that are executed very frequently.

In many beginner applications, it is possible to avoid manual synchronization entirely by using per thread data only and relying on the analysis system to merge.

Practical Tips for Beginners

To get good performance from multithreading without complications, beginners can follow a few simple practices:

Use G4MTRunManager and set the thread count once at startup.

Avoid global variables, especially mutable containers and counters. If you need a global constant, mark it as const.

Keep all analysis inside RunAction, EventAction, and Geant4 analysis classes. Do not open your own files from stepping or tracking actions.

Keep geometry and physics fixed for the duration of a run. If you need to change geometry or physics, end the run, configure, reinitialize, then start a new run.

Measure both wallclock time and memory usage for different thread counts, and choose the configuration that gives a good balance on your specific machine.

By following these guidelines, you can use multithreading to significantly reduce simulation time while maintaining correct and reproducible results.