Intel® Parallel Computing Center at Texas A&M University
- Honggao Liu, HPRC Director, TAMU
- Zhi Shang, HPRC Post-Doc, TAMU
TAMU's interdisciplinary High Performance Research Computing (HPRC) has a mission to infuse computational technologies into the research and creative activities of every academic discipline. The Intel PCC program at HPRC led by PI Liu will develop open-source software focusing on simulation of flows through micropores, such as those found in rocks involved in oil and gas extraction, by extending OpenFOAM, a popular open-source simulation software. Fig. 1 shows the digital micropores in the rocks in oil reservoir.
The primary focus is to modernize OpenFOAM to increase parallelism and scalability through optimizations that leverage cores, caches, threads, and vector capabilities of Intel Xeon Phi processors. For achieving the best speedup using Intel Xeon Phi processors, we are interested in increasing the parallelism by adding OpenMP where applicable, refactoring code to allow compilers to vectorize loops, and identifying candidate arrays for the high-bandwidth, on-package memory to making OpenFOAM run better on Knights Landing processors.
Gas, oil, water and sand particles all normally exist in a reservoir. The accurate information on the location of oil within porous media (such as rock and sand) is needed to help determine where to drill and to evaluate existing oil reservoirs. It is also important to know where extensive sand deposits exist within porous media. Therefore, there are lots of data processing and physics computing in analyzing the situation of the reservoir. With new methods available to address complex physical phenomena, and advances in powerful computing platforms, the ability to model fundamental flow physics at high resolution becomes both essential and possible. The MP-PIC (multiphase particle-in-cell) method was employed as the DPM (discrete particle modeling) model to perform the multiphase flow simulation in the porous media. Fig. 2 shows the distribution of the sand particles with the oil flows inside the porous media of the reservoir.
The multiphase flow is fundamental to many engineering and environmental processes related not only to oil & gas but also to chemical processing, energy, and geophysics. Simulation challenges are magnified by the complicated flow phenomena with interactions between different phases. Especially when the simulated domain is approaching industrial scale, the large number of grid plus the complicated flow phenomena will block practical application by traditional numerical simulations. Partly reason is due to current community codes, however, do not exploit HPC capabilities to the fullest and lack fully coupled physics.
The Intel PCC project at HPRC will undertake code performance scaling, profiling, and optimization for OpenFOAM on advanced HPC platforms, and develop the multiphysics coupled simulation algorithm suited for both fundamental physics needs as well as efficient usage of HPC resources. The research results will serve as the development methods for extending the current OpenFOAM programming frameworks through incorporation of modules using Intel Xeon Phi coprocessors and processors. Code modernization for scientific and industrial research is critical to advancing the pace of discovery and innovation. Modernizing OpenFOAM codes for Intel architecture will have broad and lasting impact on the community for years to come.
Fig. 3 shows the speed up of the large scale CFD simulations (51 million mesh cells with 40 million sand particles) on the oil reservoir using coupled DPM with CFD with OpenFOAM accelerated by Intel Xeon Phi Coprocessors using MPI, OpenMP and vectorization parallel programming. In Fig. 3, 1 node has 20 processors and 1 Xeon Phi Coprocessor has 61 cores.
- Zhi Shang, Honggao Liu, James A. Lupo. High Performance Computing of Multiphase Flows in Porous Media - Using OpenFOAM and the Intel Xeon Phi. Lambert Academic Publishing, OmniScriptum GmbH & Co. KG, Bahnhofstraße 28, 66111 Saarbrücken, Germany, 2017. .
- Zhi Shang, Honggao Liu, James A. Lupo. Evaluation of hybrid MPI-OpenMP on discrete particle modeling for large scale parallel computing with OpenFOAM. Asian Journal of Mathematics and Computer Research
- Zhi Shang, Honggao Liu. Simulating Multiphase Flows in Porous Media Using OpenFOAM on Intel Xeon Phi Knights Landing Processors. Practice & Experience in Advanced Research Computing (PEARC17), New Orleans, Louisiana, July 9 - 13, 2017, USA. (presentation + article)
- Zhi Shang, Honggao Liu. High Performance Computing of Multiphase Flow in Porous Media Network using OpenFOAM. Texas A&M Research Computing Week, College Station, Texas, June 5 - 9, 2017, USA (poster)
- Zhi Shang, Honggao Liu. High Performance Computing at Intel Xeon Phi Knights Landing Cluster with OpenFOAM. Poster at Texas A&M University booth, 29th Supercomputing Conference (SC16), Salt Lake City, Utah, November 13 - 18, 2016, USA. (poster)
- Zhi Shang, Honggao Liu, Krishnaswamy Nandakumar, Mayank Tyagi, James A. Lupo, Karsten Thompson. Discrete Particle Model for Porous Media Flow using OpenFOAM at Intel Xeon Phi Coprocessor. American Physical Society, Division of Fluid Dynamics 68th Annual Meeting, Boston, Massachusetts, November 22 - 24, 2015, USA.
- Zhi Shang, Mayank Tyagi, James A. Lupo, Honggao Liu, Krishnaswamy Nandakumar, Karsten Thompson. Large Scale CFD Simulations of Particulate Flows in Porous Media. 27th Supercomputing Conference (SC15), Austin, Texas, November 15 - 20, 2015, USA. (poster)
- Zhi Shang. High Performance Computing at Intel Xeon Phi Coprocessor for Discrete Particle Model of OpenFOAM. Society of Exploration Geophysicists (SEG), International Explosion and 85th Annual Meeting, New Orleans, Louisiana, October 18 - 23, 2015, USA. (presentation)
- Zhi Shang, Honggao Liu, Krishnaswamy Nandakumar, Mayank Tyagi, James A. Lupo, Karsten Thompson. High Performance Computing at Intel Xeon Phi Coprocessor Using Native and Symmetric Modes for Discrete Particle Model of OpenFOAM. 3rd Annual EPIC Workshop on Enabling Process Innovation through Computation, Louisiana State University, Baton Rouge, Louisiana, May 1, 2015, USA. (poster)
Media Reports for Research in Oil & Gas Applications
- Linda Barney. Transforming Raw Oil and Gas Reservoir Data into Actionable Insights. Wed, 10/21/2015 - 10:45am
Last modified: 20 Jun 2017