Description of Course
Parallel Programming in High-Performance Computing (HPC) course is designed to equip you with foundational knowledge and practical skills to harness the power of parallel computing. Modern scientific, industrial, and technological advancements demand immense computational power, often exceeding the capabilities of single-core processors. Parallel computing leverages multiple processors or cores to execute tasks simultaneously, drastically reducing computation times and enabling solutions to previously unsolvable problems. HPC and parallel programming are at the heart of advancements in areas like numerical problems, climate modeling, engineering simulations, finance simulations and more. Whether you’re a researcher, developer, or industry professional, mastering parallel programming will enable you to tackle complex computational challenges efficiently and scale your solutions to meet real-world demands.
The course was developed in autumn 2022 and has been conducted three times, tailored for Montenegrin students and industry partners affiliated with NCC Montenegro.
Course Content Overview
- Introduction to High-performance Computing (HPC): Fundamentals of HPC and its importance in computational tasks, Applications of HPC in scientific research, engineering, and artificial intelligence, Overview of the HPC landscape in Montenegro for research and business opportunities
- Parallel Computing Basics: Core concepts of parallel computing, Importance of parallelism in accelerating computations, Introduction to parallelization strategies: task parallelism and data parallelism
- Parallel Computing Architectures: Overview of shared memory, distributed memory, and hybrid architectures, Strengths, challenges, and applications of each architecture type, Case studies for different architectural approaches
- Programming Models for Parallel Computing: Introduction to Message Passing Interface (MPI) and Open Multiprocessing (OpenMP), Hybrid programming concepts combining MPI and OpenMP, Communication, synchronization, and workload distribution principles
- Linux Fundamentals for HPC: Linux environment basics for HPC programming, File systems, command line tools, and scripting for automation, setting up and managing a development environment, preparation for hands-on in upcoming modules
- Developing Parallel Programs: Writing parallel programs in C and Python, Implementing MPI and OpenMP for real problems, Debugging and profiling parallel applications
- Job Scheduling and HPC Resource Management: Introduction to job schedulers (Slurm), Managing computing resources effectively, Writing and submitting job scripts
- Performance Analysis and Optimization: Key performance metrics: speedup, efficiency, and scalability, Tools and techniques for analyzing program performance, Strategies for optimizing parallel programs
- Hybrid Computing Techniques: Combining shared and distributed memory approaches, Implementing hybrid solutions with MPI and OpenMP, Examples of hybrid computing applications in real world scenarios
- Real Applications of Parallel Computing: Practical exercises in domains such as data processing, simulations, and AI, Case studies showcasing the impact of HPC on complex problems, team based projects to solve real challenges
- Advanced Topics in Parallel Computing: Dynamic load balancing and scalability considerations, Emerging trends in HPC and AI integration, Overview of future directions and career opportunities in HPC
- Final Project incorporating all course modules, Presentation of project outcomes, Course review and preparation for continued learning in HPC
Who Should Attend?
This course is tailored for students, researchers, engineers, and professionals seeking to expand their expertise in parallel programming and high-performance computing. A basic understanding of programming (preferably in C and/or Python) and familiarity with Linux will be advantageous but not mandatory.