High Performance Computing

Papers:

• Neural Acceleration of Graph Based Utility Functions for Sparse Matrices [pdf:]
• Neural Acceleration for Incomplete Factorization Preconditioning [pdf:]
• Grokking - Generalization Beyond Overfitting on Small Algorithmic Datasets - 2201.02177v1 [pdf:]
• Grokfast - Accelerated Grokking by Ampilfying Slow Gradients - 2405.20233v2 [pdf:]
• Evaluation of Pre-training LLMs on Leadership Class Supercomputers [pdf:]
• papers/Probability-Aware Selective Protection for Sparse Iterative Solver.pdf
• papers/Fiedler's Theory of Spectral Graph Partitioning.pdf
• books/Spectral Algorithms.pdf
• HPAC (High Performance Approximate Computing) [pdf:]
• HPAC-ML [pdf:]

Books:

• The Art of HPC, Volume 1 & 2: https://theartofhpc.com/
• Numerical Analysis - Mathematics of Scientific Computing by David Kincaid, Ward Cheney

To read:

• Symbolic Factorization of Sparse Matrix
• RCM
• Approximate Minimum Degree Ordering Algorithm (AMD)
• Nested Dissection (ND)
• Graph Based Utility Functions
  • vertex information
  • graph partition
  • subgraph identification
• Spectral Subarea convolution operator
• Spatial Subarea convolution operator
• ILU(k)
• ICHOL(k)
• Conjugate Gradient Method
• Generalized Minimal Residual Method
• Transformer Architecture [pg. 3]