Using precompiled software on clusters

Finding and Using Installed Software

On most clusters you use software that is already installed by the system administrators. This is called precompiled or centrally installed software. You do not download and build everything yourself. Learning how to discover and select these packages is essential, because it saves time and helps you get good performance with minimal effort.

Clusters often have many versions of the same package, built with different compilers and optimized for different CPUs or GPUs. Your task is to choose appropriate versions, load them into your environment, and combine them safely.

Relationship to Modules and Software Stacks

A software stack is the combination of compilers, MPI implementations, numerical libraries, and applications that work together on a given system. System administrators usually define several compatible stacks, and give you access to them through environment modules or a similar mechanism.

You will see tools such as module, lmod, or sometimes spack load. These tools do not install software when you run them. Instead, they adjust your environment so that precompiled software that is already on the system becomes visible and used by your programs.

This chapter focuses on how to work with that precompiled software in practice, assuming you already know the basics of modules and of what numerical libraries are from earlier chapters.

Discovering Available Software

On a typical cluster the main discovery commands come from the module system. A few common patterns appear across many systems, even if the exact naming differs.

To see all available modules you often use a command such as:

module avail

This lists module names grouped by category or directory. Categories often correspond to compilers, MPI implementations, libraries, and applications. The list can be very long, so interactive filtering is important.

You can usually search for a specific package name:

module spider fftw

or:

module avail fftw

Some systems use module spider for a deep search that also shows versions and dependency information, while module avail only lists modules that are immediately loadable with your current module path. If a search shows “hidden” or “inactive” modules, that often means you first need to load a base compiler or MPI module before you can use the higher level library.

Many centers also maintain documentation pages or wikis that list installed software, recommended modules for certain workloads, and example job scripts. It is usually worth consulting these pages to understand local naming conventions for compilers and stacks.

Understanding Module Names and Versions

Module names typically encode several pieces of information: package, version, and sometimes the compiler and MPI implementation that were used to build the software. For example:

FFTW/3.3.10-gompi-2023b

may indicate FFTW version 3.3.10, built with a toolchain called gompi from a stack release named 2023b. A different module might be:

FFTW/3.3.10-intel-2022a

which is the same library version, but compiled with the Intel compiler and possibly a different MPI stack.

The choice of module affects:

1. Which compiler you must use when you compile your own code.
2. Which MPI implementation you must link against.
3. Which CPU or GPU features the binary expects.

A basic rule is to keep your stack consistent. If you load a library built with GCC/12.2.0 and OpenMPI/4.1.5, you should compile your program with the same compiler and usually link against the same MPI.

If you are unsure, look at the full module name and the help text for the compiler, MPI, and library modules. Many module systems support:

module help FFTW/3.3.10-gompi-2023b

which typically describes dependencies and usage notes.

Loading and Unloading Software

Loading a precompiled package is usually as simple as:

module load FFTW/3.3.10-gompi-2023b

The module command adjusts environment variables such as PATH, LD_LIBRARY_PATH, and CPATH. After this, commands and libraries from FFTW become visible to compilers and at runtime.

To see what you currently have loaded:

module list

You can unload specific modules:

module unload FFTW

or reset to a clean state:

module purge

A clean environment is especially important when troubleshooting or when you change to a different stack. Many subtle problems come from accidentally loading libraries from conflicting stacks.

A common pattern when you begin a new session or job script is:

module purge
module load GCC/12.2.0
module load OpenMPI/4.1.5
module load FFTW/3.3.10-gompi-2023b

This ensures that your full environment is consistent and reproducible.

Inspecting How Modules Change Your Environment

To understand what a module is doing, you can inspect the changes it applies. Some systems provide:

module show FFTW/3.3.10-gompi-2023b

This prints the underlying script that modifies environment variables. You can see which directories are added to PATH, LD_LIBRARY_PATH, and CPATH, and sometimes which additional modules are loaded automatically.

Such inspection is useful when:

You want to know where header files live, for example to set -I include paths manually.

You need to confirm which library files the linker will see, for instance .so or .a paths.

You want to replicate a similar configuration in a custom build or container.

If you suspect that a module is causing conflicts, inspecting it can reveal overlapping paths or unexpected dependencies.

Linking Against Precompiled Libraries

When you compile your own code, you often want to link against the precompiled numerical libraries already installed. The exact commands will depend on the library and compiler, but the typical workflow is similar.

First, load the relevant modules:

module purge
module load GCC/12.2.0
module load OpenMPI/4.1.5
module load FFTW/3.3.10-gompi-2023b

Next, compile code that includes the library headers. For FFTW in C, you might write:

gcc mycode.c -o mycode -lfftw3 -lm

Because the module set CPATH and LIBRARY_PATH or LD_LIBRARY_PATH, the compiler and linker automatically find the header files and libraries. For some libraries you need explicit include flags such as -I or library search paths with -L. The module help text often shows recommended compile and link commands.

Some packages provide wrapper compilers that already include the correct flags. Typical examples are MPI compilers mpicc, mpicxx, or mpifort. If a library offers such wrappers, using them reduces the chance of errors.

Whenever possible, test a minimal example program to confirm that your link flags are correct and that your program runs successfully on the cluster.

Using Precompiled Applications and Tools

Many HPC clusters provide entire applications as precompiled binaries, not just libraries. Examples include visualization tools, CFD codes, molecular dynamics engines, and finite element packages.

The pattern is similar:

1. Search for the application module with module avail or module spider.
2. Load the specific version that matches your desired stack.
3. Run the application, usually from within a batch job script.

Some applications provide separate “serial” and “parallel” modules. The parallel versions require an MPI module and are typically launched with the scheduler’s parallel launch command, for example srun in SLURM.

Often the module help text contains example job scripts. These examples are valuable guides, because they are tuned for the local system and may include settings such as thread counts, environment variables, and optimal MPI launch flags.

Reproducibility with Precompiled Software

Because the module system can change over time, and because administrators may install new versions or retire old ones, it is important to record which modules you used.

You can embed the relevant module load commands directly in your job scripts. This has several benefits:

Your job sees the same environment every time it runs.

You can share the script with collaborators who can reproduce your run by using the same cluster.

If the cluster operators change the default modules, your script is still explicit, so it will either continue to work or fail in a visible way.

Some users also capture the output of module list in log files:

module list 2>&1 | tee modules_used.log

This records the exact stack used for a particular run, which is useful for performance comparisons and for scientific reproducibility.

If a module disappears, consult the documentation or support team to find the replacement stack. Often, centers maintain compatibility notes when transitioning between software releases.

Managing Conflicting or Incompatible Modules

It is common to encounter conflicts when loading multiple precompiled packages. Module systems usually detect obvious conflicts. For example, loading a module for a different compiler stack might produce a message such as “conflict with GCC/12.2.0, unloading.”

You should be aware of a few typical patterns:

You usually do not load more than one primary compiler module at a time.

Similarly, you usually select one MPI implementation per session or job.

Higher level modules, such as applications or numerical libraries, are often tied to specific lower level modules. Loading them may implicitly load the correct dependencies and unload incompatible ones.

To reduce confusion, work with a clear sequence:

1. module purge to ensure a clean start.
2. Load a compiler.
3. Load MPI, if needed.
4. Load numerical libraries and applications.

If you encounter subtle run time problems such as segmentation faults or incorrect results, one of the first troubleshooting steps is to check module list and confirm that your stack matches known working combinations from the system documentation.

Using Precompiled Libraries in Job Scripts

Using precompiled software affects how you write your job scripts. In noninteractive batch jobs, your script must construct the same environment that you used during compilation or testing.

A typical job script might begin with:

#!/bin/bash
#SBATCH --job-name=myjob
#SBATCH --time=01:00:00
#SBATCH --nodes=1
#SBATCH --ntasks=4
module purge
module load GCC/12.2.0
module load OpenMPI/4.1.5
module load FFTW/3.3.10-gompi-2023b
srun ./mycode

You should avoid relying on your interactive shell’s startup files to load modules, because the scheduler might run your job in a slightly different environment. The job script must be self contained.

If your program uses OpenMP or other thread based parallelism in combination with MPI or GPU libraries, you may also need to set environment variables such as OMP_NUM_THREADS after loading modules, so that the runtime can find the correct libraries and tune itself.

When Precompiled Software Is Not Enough

Sometimes the precompiled packages on a cluster do not cover your needs. For example, you might require:

A newer or older version of a library.

A library compiled with special options or patches.

An application with custom plugins.

In these cases, you can often build your own copy of the software in your home or project directory. Even then, it is useful to base your build on the cluster’s precompiled compilers and foundational libraries.

For instance, you can:

Load the system’s preferred compiler and MPI modules.

Use the installed BLAS, LAPACK, FFT, or HDF5 libraries as dependencies when you configure and build your custom software.

This keeps your build compatible with the rest of the system and helps you benefit from tuned low level libraries that the administrators already provide.

When you build custom software, consider writing your own small module file or using tools like Spack to manage and load it, so that your local stack stays organized and reproducible.

Performance Considerations with Precompiled Software

Precompiled software on clusters is usually built with architecture specific optimizations and configured to use the system’s fastest numerical kernels. Using these packages is often faster than compiling the same code with default settings on your own workstation.

However, performance still depends on choosing the right modules:

Library versions may vary in how well they exploit the cluster’s CPUs or GPUs.

Some modules are built for generic instruction sets, while others target specific processors.

GPU libraries may require a matching CUDA version or driver level.

System documentation often recommends specific modules for a given hardware generation. For example, a cluster with multiple CPU types may provide separate stacks tuned for each. Matching the recommended module stack for the nodes you use helps you reach the performance the system is designed to deliver.

If you experiment with different versions of a library to improve performance, always test with a representative workload and document the module versions you used so that you can reproduce and compare results.

Summary

Using precompiled software on clusters revolves around a few recurring actions: discovering what is installed, choosing compatible versions that match your compiler and MPI, loading those modules consistently, and linking or running your programs within that environment. By relying on the centrally installed stacks, you avoid most of the complexity of building and tuning libraries yourself, while still benefiting from high performance implementations that are tailored to the cluster hardware.