Professor: Peter Pacheco
Office: Harney 540
Phone: 422-6630
Email: user: peter, domain: usfca.edu
Office Hours: MWF 12:30-1:30 and by appointment

Course Syllabus

Programming Assignments

1. Programming Assignment 1
2. Programming Assignment 2
3. Programming Assignment 3. Note: The default value of k in method 3 has been changed from n/2 to n/4.
4. Programming Assignment 4
5. A list of possible projects for the final programming assignment. Here's a list of tentative presentation dates.
   • Monday, May 9:
     1. Kai and Chao
     2. Yumeng and Ludan
     3. Xintian and Lin
     4. Simao and Chen
     5. Neal and Leo
     6. Adrian
   • Wednesday, May 11:
     1. Calvin and Felix
     2. Srujana
     3. Krichaporn and Pakkapon
     4. Puneet and Basar
     5. Shan and Bobby

Seminar Papers

• The first three seminars will discuss " The Landscape of Parallel Computing Research: A View From Berkeley," by Krste Asanovic, et al.
  1. First presentation: Introduction, Motivation, Applications and Dwarfs, pp. 1-19. Presenters: Kai Chen and Xinli Feng (Felix).
  2. Second presentation: Hardware and Programming Models, pp. 20-36. Presenters: Chengzhi Liang (Calvin) and Chao Ren.
  3. Third presentation: Systems Software, Metrics for Success, and Conclusion, pp. 36-45. Presenters: Li-Hung Hsieh (Leo) and Yucheng Xiong (Neal).
• The fourth seminar will cover the paper Transactional Memory by James Larus and Christos Kozyrakis. Presenters: Puneet Pal and Srujana Tumu.
• The fifth seminar will cover the paper A View of Cloud Computing by Michael Armbrust, et al. Presenter: Basar Onat.
• The sixth seminar will cover pp. 1-15 of the paper JOSTLE: Parallel Multilevel Graph-Partitioning Software - An Overview by C. Walshaw and M. Cross. Presenter: Adrian Bivol.
• The seventh seminar will cover the paper Data Structures in the Multicore Age by Nir Shavit. Presenters: Krichaporn Srisupapak and Pakkapon Poopaka
• The eighth seminar will cover the paper Memory models: a case for rethinking parallel languages and hardware by Sarita V. Adve and Hans-J. Boehm. Presenters: Ludan Zhang and Yumeng Sun
• The ninth seminar will cover pp. 36-47 of the paper Prefix Sums and Their Application by Guy E. Blelloch. Presenters: Xintian Dong and Chen Chen.
• The tenth seminar will cover the paper Designing Efficient Sorting Algorithms for Manycore GPUs by Nadathur Satish, Mark Harris, and Michael Garland. Presenters: Shan Ye and Guannan He (Bobby).
• The eleventh seminar will cover the paper Languages for High-Productivity Computing: the DARPA HPCS Language Project by Ewing Lusk and Katherine Yelick. Presenters: Lin Wang and Simao Liu.

Additional Course Information

• How to set up a Subversion repository
• A PDF form for seminar paper evaluation
• A plain text form for seminar paper evaluation
• A short guide to the use of the Penguin cluster (corrected 2/1/11)
• Run-times of three implementations of the trapezoidal rule
• Speedups of three implementations of the trapezoidal rule
• Efficiencies of three implementations of the trapezoidal rule
• Performance of an MPI implementation of the trapezoidal rule
• Comparative performance of Allgather using block and cyclic distributions
• Run-times for Pthreads matrix-vector multiplication
• A Brief Guide to Using CUDA at USF
• CUDA Programming Guide
• CUDA Reference Manual
• CUDA-GDB User Manual
• CUDA SDK code samples
• NVIDIA GPU Computing Developer Home Page
• A key to the midterm
• Run times of MPI matrix-vector multiplication

Code

• Serial trapezoidal rule
• MPI implementation of the trapezoidal rule
• Pthreads implementation of the trapezoidal rule
• A macro for taking times in serial and Pthreads programs
• First OpenMP version of the trapezoidal rule: uses a parallel and a critical directive
• Second OpenMP version of the trapezoidal rule: uses a parallel directive and a reduction clause
• Third OpenMP version of the trapezoidal rule: uses a parallel for directive and explicit scoping
• MPI implementation of the trapezoidal rule. This version reports execution time.
• Pthreads implementation of the trapezoidal rule. This version reports execution time.
• Second OpenMP implementation of the trapezoidal rule. This version reports execution time.
• Program that compares the performance of Allgathers with block and cyclic distributions
• First Pthreads implementation of matrix-vector multiplication
• A CUDA program for adding two vectors
• A very short CUDA program illustrating the use of cuPrintf
• A CUDA program that implements a dot product using atomic operations. Note that atomicAdd for floating point arguments isn't available for our Nvidia cards. So this version uses an implementation taken from the CUDA Programming Guide
• A CUDA implementation of atomicAdd for floating point arguments.
• A more modular version of the dot product program that uses atomicAddf
• A version of the dot product program that uses a binary tree within the thread blocks.
• A version of the dot product program that uses a binary tree within the thread blocks and shared memory for storing the component products.
• A version of the dot product program that uses a different binary tree within the thread blocks.
• A version of the dot product program that uses page-locked mapped memory.
• A version of the dot product program that unrolls the for loop in the kernel.
• A program that illustrates the use of cuPrintf.cu
• MPI matrix-vector multiplication using a block row distribution
• MPI matrix-vector multiplication using a block column distribution
• MPI matrix-vector multiplication using a block submatrix distribution. This version builds row and column communicators using MPI's topology functions.
• Serial tree-search that uses recursion.
• Serial tree-search that doesn't use recursion
• Pthreads tree-search that uses a static partition of the search tree.
• MPI tree-search that uses a static partition of the search tree.
• Pthreads tree-search that uses a dynamic partition of the search tree.