NVIDIA Kepler to Ampere: ECHELON Reservoir Simulation Software Performance Scaling Through Five Generations of GPUs

The first results benchmarking ECHELON software on the newly released NVIDIA Ampere A100 are out, and they make a powerful and compelling case for continued performance scaling with GPU hardware capability (see Figure 1 below). This is the third time I’ve published a version of this chart. The first time was in 2016 with the Pascal release, then for Volta in 2018 and now for Ampere. The message is clear and thrilling, ECHELON software continues to scale linearly with memory bandwidth. As NVIDIA delivers more and more powerful chips, ECHELON software efficiently produces greater performance. Although we have improved the software through algorithmic optimizations, we have attempted to make these comparisons with the code fixed for a fair evaluation of the hardware impact.

As NVIDIA delivers more and more powerful chips, ECHELON reservoir simulation software efficiently produces greater performance.

In real terms, ECHELON reservoir simulation software has become about 5.5x faster in just 7 years from hardware advances. This kind of performance uplift from hardware alone has not been seen since the 1990s when CPU clock frequencies increased with every new generation. That trend came to a halt however in the mid 2000’s when clock speeds stalled, and chip vendors moved to multi-core designs. From then on, the burden of improving performance was left to software developers who had to modify codes for parallel operation.

Figure 1 - ECHELON software performance vs GPU memory bandwidth.

A recent Wall Street Journal article coined the phrase Huang's Law (after NVIDIA CEO Jensen Huang) to describe the phenomenal rise in AI computing power delivered by NVIDIA hardware year after year over the last decade. It is meant to contrast with Moore's law which is an empirical observation of long-term trends in CPU capability. Moore's law actually just states that the number of transistors on a chip doubles about every two years, which is still a valid observation. However, GPUs and CPUs deploy those transistors to different purposes and in recent years GPU architectures have delivered significantly greater tangible gains in software performance than CPUs. Figure 2 below tells the story by comparing the evolution of memory bandwidth and double-precision flops between GPUs (green) and CPUs(blue) over the last 12 years. For ECHELON reservoir simulation software and most other technical software applications, memory bandwidth is the key commodity for performance.

Figure 2 - Evolution of memory bandwidth(left) and double-precision flops (right) for GPUs(green) and CPUs(blue) from 2008 to 2020.

There has been much focus on the advances that GPUs have enabled in AI and machine learning but it's important to note that there are concomitant gains for those of us doing traditional technical and scientific computing. The exciting news is two-fold. First, as illustrated in Figure 2, in a chip to chip comparison, GPU performance as measured by both memory bandwidth and flops has grown at a faster rate than that of CPUs for 12 years now. The gap in performance is large and growing. Second, software like ECHELON that make full use of the GPU, in that it has been designed specifically for them from inception are on a different reservoir technology curve than their CPU competitors. Our performance is now governed by Huang's law. As we look to the coming decades with more emphasis on complex physics, higher resolution grids, and ensembles of models this data showing that simulation grounded in GPU power will continue to yield industry pacing gains is welcome news.