CSE 6230 : High Performance Parallel Computing - Spring’23

Course Objective

With computing pervasive everywhere from instrumenting with microcontrollers like Arduino, small form factor devices like cell phones, laptops to supercomputers programming exercising the complete capability of these devices is pivotal for the future. This course prepares you towards writing blazing fast code to crunch numbers from large graphs, scientific data from instruments, internet services data in a reasonable time. The course focuses on hands-on experience with programming high-performance computer systems, from single-processor multicore platforms up to large-scale distributed memory multi-CPU/GPU systems. Unlike CSE 6220: Intro to HPC, which presents theory, this course explores practice: current and emerging programming models, performance analysis, and low-level performance engineering for parallelism. We will focus on writing parallel code for different architectures, using the debugging and performance tools to understand performance bottlenecks, performance model your algorithm to model the peak and achievable performances. In the latter half of the course, we will focus on graph and machine learning applications. This will tease you with contemporary applications on leadership computing facilities on graphs, machine learning and simulations. In the final section of the course, we will discuss your course projects.

SCHEDULE

The class meets Tuesdays and Thursdays from 5:00-6:15pm in Ford Environment Science and Technology Building, Room L1255. Tuesday will be online and Thursday will be in person lecture. Each lab assignment is due before class (at 4:30pm) on the deadline day.

GRADING

This course is taught under the Georgia Tech Honor Code. In the context of this course, this means that all work that a student submits for individual assignments and exams will be his or her own work. For group assignments, project submissions, or of projects that are part of a larger on-going project, the final report has a required section in which each student must detail which portions of the work were completed individually and which portions were the work of others.

The course has three components:

Four Programming Assignments -30%

All assignments must run complying the output specifications. We will be downloading all the project codes and running a script to evaluate the project. If the code is not complying to specification, you can get 0 points as if it didnt run. The assignment winner is the one with the top-performing code. For each assignment, the assignment winners will be invited to share their work with the class on 3/16/23.

Mid term - 10%

Final Group Project - 60%

All deliverables and exams will be graded by TA. The project proposal and final presentation will be graded by both the TA and the course instructor.

PREREQUISITES

You should be comfortable with sequential programming in “mainstream” general-purpose compiled languages, such as Fortran, C, C++. Familiarity with basic algorithm analysis (i.e., big-O notation) is also helpful.

Preferred Courses -

The amounts of time students spend on this class greatly vary, based on their backgrounds, and what they may already know. Students with limited computer science background can find the homework assignments challenging, would take significant time and effort. Students will have 10 days to complete each homework assignment. Do not wait until the last day. It is critical to plan ahead and prepare for the significant time needed. Almost all homework assignments involve very large amount of programming tasks (which naturally means likely a lot of debugging will be needed, thus can be time consuming). You should be proficient in at least one high-level programming language (e.g., Python, C++, Java), and is efficient with debugging principles and practices. If not, we recommend first taking introductory computing course(s) before taking this course. For example, CSE 6040 for (OMS) Analytics students; CS 1301, CS 1331, CS 1332, CS 1371, etc. for an campus students.

Previous Courses

Office hours:

Thursday’s 2-3pm EST. Check the class Piazza for obtaining the zoom link.

TA and office hours:

Tuesday’s - 3:45-4:45pm. Check the class Piazza for obtaining the zoom link.

Announcements and Discussion

We use Piazza for all announcements and discussion. Everyone must join this class's Piazza (link available on Canvas). Double check that you are joining the correct Piazza! There are multiple concurrent course sections with the same name and course number taking place, e.g., online for OMSA and OMSCS, and campus for Atlanta-based students.

CSE 6230 Piazza Link

The fastest way to get help with homework assignments is to post your questions on Piazza. That way, not only our TAs and instructor can help, your peers can too. If you prefer that your question addresses to only our TAs and the instructor, you can use the private post feature (i.e., check the "Individual Students(s) / Instructors(s)" radio box). While we welcome everyone to share their experiences in tackling issues and helping each other out, but please do not post your answers, as that may affect the learning experience of your fellow classmates. For special cases such as failed submissions due to system errors, missing grades, failed file uploads, emergencies that prevent you from submitting, personal issues, you can contact the staff using a private Piazza post.

Homework and Assignment Submissions

Canvas will be used for submission of assignments and projects, but not for announcements or discussion. Plagiarism, Collaboration Policy, and Student Honor Code All course participants (myself, teaching assistants, and learners) are expected to know and abide by the Georgia Tech Academic Honor Code. Ethical behavior is extremely important in all facets of life. Plagiarism is a serious offense. You are responsible for completing your own work. You are not allowed to copy and paste, or paraphrase, or submit materials created or published by others, as if you created the materials. All materials submitted must be your own. You may discuss high-level ideas with other students at the "whiteboard" level (e.g., how cross validation works, use hashmap instead of array) and review any relevant materials online. However, each student must write up and submit his or her own answers. You must not put your code on public domain (e.g., public GitHub), because a (future) student could copy your code. That student obviously violates the honor code, and you may also be implicated. All incidents of suspected dishonesty, plagiarism, or violations of the Georgia Tech Honor Code will be subject to the institute’s Academic Integrity procedures (e.g., reported to and directly handled by the Office of Student Integrity (OSI)). Consequences can be severe, e.g., academic probation or dismissal, grade penalties, a 0 grade for assignments concerned, and prohibition from withdrawing from the class.

Late Policy and Due Dates

All assignments will be available on canvas after the class at 6pm on the announcement date. All assignment are due before the class at 4:55pm on the deadline Two days extension can be provided if notified a day before the deadline to the TA for a 20% penalty. No penalties for medical reasons or emergencies. And should they arise, you must contact the Dean of Students office. Doctor's notes, medical documentation, explanation of emergencies, etc. should be submitted to the Dean’s office. After their office receives the information, they will notify me on your behalf.

Attendance Policy

This is a hybrid course – with some in-class lectures and some over zoom/teams. All the classes will be recorded over zoom for future reference. We request the students to mute their mic and video while joining the lectures remotely. The zoom/teams lectures will also be projected in the class room. We will send the zoom link to all the registered students. Students who didnt get the email from us, send the TA or the instructor an email to get the link. Login regularly to complete your work, so that you do not have to spend a lot of time reviewing and refreshing yourself regarding the content.

Detailed Schedule

Day Title Lecture by Type Slides Recorded Video Notes Deadlines  
1/10/23 Introduction I Ramki and Shruti In Class and Zoom Slides Video      
1/12/23 Introduction II Ramki and Shruti In Class and Zoom Slides Video Assignment 0 Announcement    
1/17/23 OpenMP Ramki Zoom Slides Video Assignment 1 Announcement Assignment 0 Due  
1/19/23 MPI Ramki In Class and Zoom Slides Video      
1/24/23 CUDA Ramki Zoom Slides Video      
1/26/23 HIP and AMD devices Jakub Kurzak Zoom Slides Video Assignment 2 Announcement Assignment 1 Due  
1/31/23 Parallel IO Jong-Choi Youl Zoom Slides Video      
2/2/23 Logging and Debugging Ramki In Class and Zoom          
2/7/23 Performance Modeling Piyush Zoom          
2/9/23 Distributed Complexity Analysis Edgar Zoom       Assignment 3 Announcement Assignment 2 Due
2/14/23 Profiling Guest Lecture Zoom          
2/16/23 Machine Architecture Tools (Likwid, hwloc, perftools) Guest Lecture Zoom     Assignment 4 Announcement Assignment 3 Due  
2/21/23 Kokkos Damien Zoom          
2/23/23 HPC on Python Ramki            
2/28/23 Preparation for Midterm   In Class and Zoom       Assignment 4 Due  
3/2/23 Midterm   In Class only       Project Proposal Presentations due  
3/7/23 HPC Future Doug Kothe In class          
3/9/23 Project Proposal Students In Class and Zoom          
3/14/23 Project Proposal Students In Class and Zoom       Proposal Report Due  
3/16/23 Assignment Winners Lecture   In Class only          
3/21/23 Spring Break Spring Break Spring Break Spring Break     Spring Break  
3/23/23 Spring Break Spring Break Spring Break Spring Break     Spring Break  
3/28/23 HPC and Graph Algorithms Aydin Buluc Zoom          
3/30/23 HPC/ML - Linear Regression and Dimensionality Reduction Srinivas Eswar Zoom          
4/4/23 HPC/ML - Deep learning optimizations Guojing Cong Zoom          
4/6/23 Simulation I Spencer Byngelson         Demonstration I during office hours  
4/11/23 Climate Simulations Sarat Sreepathi            
4/13/23 Particle Acceleration Simulation Axel/Jean-Luc Vay         Demonstration II during office hours  
4/18/23 Final Presentation              
4/20/23 Final Presentation           Demonstration III during office hours  
4/25/23 Final Presentation              
4/27/23                
5/2/23             Final Report Due  
rss facebook twitter github gitlab scholar scholar youtube mail spotify lastfm instagram linkedin google google-plus pinterest medium vimeo stackoverflow reddit quora quora