tNavigator on clusters
On the workstations the acceleration factor cannot be kept at the same level when the model dimension increases. The problem of memory access speed starts playing a major role in this case. Thus, despite more efficient computers, the simulation time cannot be reduced any further due to memory speed limitations.
This restriction can be removed with distributed memory CPU clusters that require MPI-based algorithms for data interaction between the nodes. This approach is implemented in most of the reservoir simulators. tNavigator contains a novel hybrid algorithm for parallel computations. It utilizes the MPI approach for task distribution between the cluster nodes, and system threads between the cores within each node (MPI+threads). This approach removes the restrictions by utilizing all resources as efficient as possible, and gives a tenfold greater acceleration factor compared to the market leaders on multicore CPU clusters.
In the Moscow office Rock Flow Dynamics utilizes the cluster with 137 nodes (2652 cores) for hydrodynamic simulation. 100 nodes: 2x Xeon E5 2680v2, 20 nodes: 2x Xeon 5650, 8 nodes: 2x Xeon E5 2630v4, 9 nodes: 2x Xeon E5 2680v4, Infiniband 56 Gb/s, RAM 12.8TB, 300TB HDD.
Example of parallel speed-up on clusters.
We are proud to show an example of tNavigator scalability on cluster «Lomonosov» in Moscow State University: 512 nodes, 4096 cores, Xeon X5670, Infiniband QDR 40Gb/s. The three-phase model has been calculated: 20 million active grid blocks, 39 wells. tNavigator delivered the acceleration 1328 times. As we know this is the world record in dynamic simulations. Results have been published in the paper SPE 163090.
We are actively working on developing more efficient parallel computations. We believe there are higher limits we can achieve.