With computing pervasive everywhere from instrumenting with microcontrollers like Arduino, small form factor devices like cell phones, laptops to supercomputers programming exercising the complete capability of these devices is pivotal for the future. This course prepares you towards writing blazing fast code to crunch numbers from large graphs, scientific data from instruments, internet services data in a reasonable time. The course focuses on hands-on experience with programming high-performance computer systems, from single-processor multicore platforms up to large-scale distributed memory multi-CPU/GPU systems. Unlike CSE 6220: Intro to HPC, which presents theory, this course explores practice: current and emerging programming models, performance analysis, and low-level performance engineering for parallelism. We will focus on writing parallel code for different architectures, using the debugging and performance tools to understand performance bottlenecks, performance model your algorithm to model the peak and achievable performances. In the latter half of the course, we will focus on graph and machine learning applications. This will tease you with contemporary applications on leadership computing facilities on graphs, machine learning and simulations. In the final section of the course, we will discuss your course projects.
The class meets Tuesdays and Thursdays from 5:00-6:15pm in Ford Environment Science and Technology Building, Room L1255. Tuesday will be online and Thursday will be in person lecture. Each lab assignment is due before class (at 4:30pm) on the deadline day.
This course is taught under the Georgia Tech Honor Code. In the context of this course, this means that all work that a student submits for individual assignments and exams will be his or her own work. For group assignments, project submissions, or of projects that are part of a larger on-going project, the final report has a required section in which each student must detail which portions of the work were completed individually and which portions were the work of others.
The course has three components:
Four Programming Assignments -30%
All assignments must run complying the output specifications. We will be downloading all the project codes and running a script to evaluate the project. If the code is not complying to specification, you can get 0 points as if it didnt run. The assignment winner is the one with the top-performing code. For each assignment, the assignment winners will be invited to share their work with the class on 3/16/23.
Mid term - 10%
Final Group Project - 60%
All deliverables and exams will be graded by TA. The project proposal and final presentation will be graded by both the TA and the course instructor.
You should be comfortable with sequential programming in “mainstream” general-purpose compiled languages, such as Fortran, C, C++. Familiarity with basic algorithm analysis (i.e., big-O notation) is also helpful.
Preferred Courses -
The amounts of time students spend on this class greatly vary, based on their backgrounds, and what they may already know. Students with limited computer science background can find the homework assignments challenging, would take significant time and effort. Students will have 10 days to complete each homework assignment. Do not wait until the last day. It is critical to plan ahead and prepare for the significant time needed. Almost all homework assignments involve very large amount of programming tasks (which naturally means likely a lot of debugging will be needed, thus can be time consuming). You should be proficient in at least one high-level programming language (e.g., Python, C++, Java), and is efficient with debugging principles and practices. If not, we recommend first taking introductory computing course(s) before taking this course. For example, CSE 6040 for (OMS) Analytics students; CS 1301, CS 1331, CS 1332, CS 1371, etc. for an campus students.
Thursday’s 2-3pm EST. Check the class Piazza for obtaining the zoom link.
Tuesday’s - 3:45-4:45pm. Check the class Piazza for obtaining the zoom link.
We use Piazza for all announcements and discussion. Everyone must join this class's Piazza (link available on Canvas). Double check that you are joining the correct Piazza! There are multiple concurrent course sections with the same name and course number taking place, e.g., online for OMSA and OMSCS, and campus for Atlanta-based students.
The fastest way to get help with homework assignments is to post your questions on Piazza. That way, not only our TAs and instructor can help, your peers can too. If you prefer that your question addresses to only our TAs and the instructor, you can use the private post feature (i.e., check the "Individual Students(s) / Instructors(s)" radio box). While we welcome everyone to share their experiences in tackling issues and helping each other out, but please do not post your answers, as that may affect the learning experience of your fellow classmates. For special cases such as failed submissions due to system errors, missing grades, failed file uploads, emergencies that prevent you from submitting, personal issues, you can contact the staff using a private Piazza post.
Canvas will be used for submission of assignments and projects, but not for announcements or discussion. Plagiarism, Collaboration Policy, and Student Honor Code All course participants (myself, teaching assistants, and learners) are expected to know and abide by the Georgia Tech Academic Honor Code. Ethical behavior is extremely important in all facets of life. Plagiarism is a serious offense. You are responsible for completing your own work. You are not allowed to copy and paste, or paraphrase, or submit materials created or published by others, as if you created the materials. All materials submitted must be your own. You may discuss high-level ideas with other students at the "whiteboard" level (e.g., how cross validation works, use hashmap instead of array) and review any relevant materials online. However, each student must write up and submit his or her own answers. You must not put your code on public domain (e.g., public GitHub), because a (future) student could copy your code. That student obviously violates the honor code, and you may also be implicated. All incidents of suspected dishonesty, plagiarism, or violations of the Georgia Tech Honor Code will be subject to the institute’s Academic Integrity procedures (e.g., reported to and directly handled by the Office of Student Integrity (OSI)). Consequences can be severe, e.g., academic probation or dismissal, grade penalties, a 0 grade for assignments concerned, and prohibition from withdrawing from the class.
All assignments will be available on canvas after the class at 6pm on the announcement date. All assignment are due before the class at 4:55pm on the deadline Two days extension can be provided if notified a day before the deadline to the TA for a 20% penalty. No penalties for medical reasons or emergencies. And should they arise, you must contact the Dean of Students office. Doctor's notes, medical documentation, explanation of emergencies, etc. should be submitted to the Dean’s office. After their office receives the information, they will notify me on your behalf.
This is a hybrid course – with some in-class lectures and some over zoom/teams. All the classes will be recorded over zoom for future reference. We request the students to mute their mic and video while joining the lectures remotely. The zoom/teams lectures will also be projected in the class room. We will send the zoom link to all the registered students. Students who didnt get the email from us, send the TA or the instructor an email to get the link. Login regularly to complete your work, so that you do not have to spend a lot of time reviewing and refreshing yourself regarding the content.
| Day | Title | Lecture by | Type | Slides | Recorded Video | Notes | Deadlines |
|---|---|---|---|---|---|---|---|
| 1/10/23 | Introduction I | Ramki and Shruti | In Class and Zoom | Slides | Video | ||
| 1/12/23 | Introduction II | Ramki and Shruti | In Class and Zoom | Slides | Video | Assignment 0 Announcement | |
| 1/17/23 | OpenMP | Ramki | Zoom | Slides | Video | Assignment 1 Announcement | Assignment 0 Due |
| 1/19/23 | MPI | Ramki | In Class and Zoom | Slides | Video | ||
| 1/24/23 | CUDA | Ramki | Zoom | Slides | Video | ||
| 1/26/23 | HIP and AMD devices | Jakub Kurzak | Zoom | Slides | Video | Assignment 2 Announcement | Assignment 1 Due |
| 1/31/23 | Parallel IO | Jong-Choi Youl | Zoom | Slides | Video | ||
| 2/2/23 | Debugging | Will Castillo, Elijah Mccarthy, Veronica (ORNL) | In Class and Zoom | Introduction,GDB, ROCGDB,DDT | Video | ||
| 2/7/23 | Performance Modeling | Piyush | Zoom | Slides | Video | ||
| 2/9/23 | Distributed Complexity Analysis | Edgar | Zoom | Slides | Video | Assignment 3 Announcement | Assignment 2 Due |
| 2/14/23 | Profiling | Nick Hagerty | Zoom | Slides | Video | ||
| 2/16/23 | Guest Lecture | Zoom | |||||
| 2/21/23 | Kokkos | Damien | Zoom | Slides | Video | ||
| 2/23/23 | HPC on Python | Ramki | Slides | Video | Assignment 4 Announcement | Assignment 3 Due | |
| 2/28/23 | Preparation for Midterm | Ramki | In Class and Zoom | Slides | Video | ||
| 3/2/23 | Midterm | In Class only | |||||
| 3/7/23 | HPC Future | Doug Kothe | In class | Assignment 4 Due | |||
| 3/9/23 | Machine Architecture Tools (Likwid, hwloc, perftools) | Thomas Gruber | In Class and Zoom | Slides | Video | ||
| 3/14/23 | Project Proposal | Students | In Class and Zoom | Slides | Video | Project Proposal Presentations due on 3/13 on canvas | |
| 3/16/23 | Project Proposal | Students | In Class only | Slides | Video | ||
| 3/21/23 | Spring Break | Spring Break | Spring Break | Spring Break | Spring Break | ||
| 3/23/23 | Spring Break | Spring Break | Spring Break | Spring Break | Spring Break | ||
| 3/28/23 | HPC and Graph Algorithms | Aydin Buluc | Zoom | Slides | Video | ||
| 3/30/23 | HPC/ML - Linear Regression and Dimensionality Reduction | Srinivas Eswar | Zoom | Slides, Notebook | Video | ||
| 4/4/23 | Climate Simulations | Sarat Sreepathi | Zoom | Slides | Video | ||
| 4/6/23 | Assignment Winners Lecture | Students | In Class | PDF Slides, Devashish Assignment2 | Video | ||
| 4/11/23 | HPC/ML - Deep learning optimizations | Guojing Cong | Zoom | Slides | Video | ||
| 4/13/23 | Particle Acceleration Simulation | Axel Huebl/Jean-Luc Vay | Zoom | Slides | Video | Demonstration I during office hours | |
| 4/18/23 | Final Presentation | Students | In Class | Slides | Video | ||
| 4/20/23 | Final Presentation | Students | In Class | Slides | No Video available | Demonstration II during office hours | |
| 4/25/23 | Final Presentation | Students | In class | Slides | Video | ||
| 4/27/23 | Demonstration III during office hours | ||||||
| 5/2/23 | Final Report Due |
For project details, please look at the proposal slides lecture on 3/14 and 3/14 and the final presentation