I just received my copy of the newly published Using Advanced MPI: Modern Features of the Message-Passing Interface. Written by William Gropp, Torsten Hoefler, Ewing Lusk, and Rajeev Thakur, the book ...

MPI (Message Passing Interface) is the de facto standard distributed communications framework for scientific and commercial parallel distributed computing. The Intel MPI implementation is a core ...

While many application tuning guides focus on understanding the performance of an application on a single node, a very scalable application will use multiple node to achieve very high performance.

We have written much about large-scale deep learning implementations over the last couple of years, but one question that is being posed with increasing frequency is how these workloads (training in ...