Introduction to HPC

Where to Move

Move chapter:

• ☰ Course Overview
  • ☰ What is High-Performance Computing
  • ☰ Why HPC is important in science, engineering, and industry
  • ☰ Examples of real-world HPC applications
  • ☰ Structure and goals of the course
• ☰ Fundamentals of Computer Architecture
  • ☰ CPUs, cores, and clock speeds
  • ☰ Memory hierarchy
    • ☰ Registers
    • ☰ Cache
    • ☰ Main memory (RAM)
  • ☰ Storage systems
  • ☰ GPUs and accelerators
  • ☰ SIMD and vectorization concepts
• ☰ Operating Systems and the Linux Environment
  • ☰ Why Linux dominates HPC
  • ☰ Basic Linux command line usage
  • ☰ Filesystems and directory structures
  • ☰ Environment modules
  • ☰ Software installation concepts
• ☰ HPC Clusters and Infrastructure
  • ☰ What is an HPC cluster
  • ☰ Login nodes
  • ☰ Compute nodes
  • ☰ Head and management nodes
  • ☰ Interconnects
    • ☰ Ethernet
    • ☰ InfiniBand
  • ☰ Shared memory systems
  • ☰ Distributed memory systems
  • ☰ Parallel filesystems
    • ☰ NFS
    • ☰ Lustre
    • ☰ GPFS
• ☰ Job Scheduling and Resource Management
  • ☰ Why job schedulers are needed
  • ☰ Batch systems overview
  • ☰ Introduction to SLURM
  • ☰ Writing job scripts
  • ☰ Submitting jobs
  • ☰ Monitoring jobs
  • ☰ Cancelling and modifying jobs
• ☰ Parallel Computing Concepts
  • ☰ Why parallel computing is needed
  • ☰ Types of parallelism
    • ☰ Task parallelism
    • ☰ Data parallelism
  • ☰ Strong scaling
  • ☰ Weak scaling
  • ☰ Amdahl’s Law
  • ☰ Gustafson’s Law
  • ☰ Load balancing
• ☰ Shared-Memory Parallel Programming
  • ☰ Introduction to OpenMP
  • ☰ Threads and thread management
  • ☰ Parallel regions
  • ☰ Work-sharing constructs
  • ☰ Synchronization mechanisms
  • ☰ Race conditions
  • ☰ Performance considerations
• ☰ Distributed-Memory Parallel Programming
  • ☰ Introduction to MPI
  • ☰ MPI processes
  • ☰ Point-to-point communication
  • ☰ Collective communication
  • ☰ Communicators
  • ☰ Process topologies
  • ☰ Performance pitfalls
• ☰ Hybrid Parallel Programming
  • ☰ Motivation for hybrid programming
  • ☰ Combining MPI and OpenMP
  • ☰ Node-level parallelism
  • ☰ Cluster-level parallelism
  • ☰ Common hybrid programming patterns
• ☰ GPU and Accelerator Computing
  • ☰ GPU architecture basics
  • ☰ Memory hierarchy on GPUs
  • ☰ Introduction to CUDA
  • ☰ Introduction to OpenACC
  • ☰ Performance considerations for accelerators
• ☰ Compilers and Build Systems
  • ☰ Common HPC compilers
    • ☰ GCC
    • ☰ Intel oneAPI
    • ☰ LLVM
  • ☰ Compiler optimization flags
  • ☰ Debug builds
  • ☰ Optimized builds
  • ☰ Introduction to Make
  • ☰ Introduction to CMake
• ☰ Performance Analysis and Optimization
  • ☰ Measuring performance
  • ☰ Benchmarking applications
  • ☰ Profiling tools
  • ☰ Memory optimization
  • ☰ Cache optimization
  • ☰ Vectorization strategies
  • ☰ Improving parallel efficiency
• ☰ Numerical Libraries and Software Stacks
  • ☰ Linear algebra libraries
    • ☰ BLAS
    • ☰ LAPACK
    • ☰ ScaLAPACK
  • ☰ Fast Fourier Transform libraries
  • ☰ Scientific software frameworks
  • ☰ Using precompiled software on clusters
• ☰ Data Management and I/O
  • ☰ Parallel I/O concepts
  • ☰ File formats used in HPC
  • ☰ Checkpointing strategies
  • ☰ Restart mechanisms
  • ☰ Managing large datasets
• ☰ Reproducibility and Software Environments
  • ☰ Software stacks
  • ☰ Containers in HPC
  • ☰ Introduction to Singularity and Apptainer
  • ☰ Best practices for reproducible workflows
• ☰ Debugging and Testing Parallel Programs
  • ☰ Common bugs in parallel programs
  • ☰ Deadlocks
  • ☰ Debugging tools for HPC
  • ☰ Testing strategies for parallel codes
• ☰ HPC in Practice
  • ☰ Typical HPC workflows
  • ☰ Developing code locally
  • ☰ Running applications on clusters
  • ☰ Case studies from science
  • ☰ Case studies from industry
• ☰ Ethics, Sustainability, and Green Computing
  • ☰ Energy consumption of HPC systems
  • ☰ Efficient resource usage
  • ☰ Job sizing and fair-share usage
  • ☰ Responsible computing practices
• ☰ Future Trends in HPC
  • ☰ Exascale computing
  • ☰ AI and machine learning in HPC
  • ☰ Heterogeneous architectures
  • ☰ Quantum computing and HPC integration
• ☰ Final Project and Hands-On Exercises
  • ☰ Designing an HPC application
  • ☰ Running large-scale simulations
  • ☰ Performance analysis and optimization report
  • ☰ Documentation and best practices summary