Institute of Aerodynamics, RWTH Aachen University (Germany)
Local Project ID:
HPC Platform used:
Hazel Hen of HLRS
Within the framework of a research project which aims at reducing the emission of CO2 by conventional coal-fired power plants through oxy-fuel combustion, scientists of the RWTH Aachen University simulated the heating processes of coal dust in order to gain a better understanding of the conditions causing carbon dust to ignite in an oxygen-carbon dioxide atmosphere. Since carbon particles are of irregular, non-spherical shape their motion is difficult to predict, thus simulations of large quantities of fully dissolved carbon particles moving freely in a turbulent flow require the availability of petascale HPC systems like Hazel Hen.
Turbulent particle-laden flows are encountered in many technical and natural environments such as pulverized coal and biomass combustors, during ice-crystal formation in atmospheric clouds, or the transport of pollutants through the human airways. For the large-scale numerical simulation of such problems often Lagrangian particle models are used, in which the particles are tracked assuming a point-mass behavior. To validate and improve those models and to enhance the understanding of the multiphase characteristics of these flows, fully-resolved simulations are required. In such simulations, the discrete approximation resolves each individual particle so that an accurate formulation of the fluid-particle momentum exchange driving the particle motion is obtained and their interaction with the turbulent flow field is inherently captured. For particle sizes comparable to or smaller than the smallest structures of the carrier flow, this results in a vast increase of the computational costs. Especially, solution schemes with dynamically changing meshes are required to efficiently resolve the particle motion.
Numerical methods based on hierarchical Cartesian meshes provide an ideal framework by facilitating dynamic mesh adaptation near the particle surfaces and dynamic load-balancing routines. In order to obtain representative statistical data, a large amount of particles on the order of O(1000) has to be simulated in a turbulent flow, which requires a high-performance computing system such as Hazel Hen installed at HLRS Stuttgart.
In this project, an efficient numerical method for fully-resolved particle simulations on adaptively-refined Cartesian meshes is developed and applied. During the simulation, the computational mesh is adaptively refined to automatically resolve the particle boundary layers and wakes along their trajectories. A parallel computation, however, requires that the constantly changing mesh is redistributed among the different processes during the simulation to balance the individual process loads.
Therefore, a dynamic load-balancing strategy has been developed to enable efficient simulations on high-performance computers. Fig. 1 shows the evolution of the time required to compute a single time step for a simulation with 4,000 particles moving in a turbulent flow. The simulation has been performed on Hazel Hen at HRLS Stuttgart using 16,000 cores and approximately 1 billion grid cells on average. It can be seen that the simulation with a static domain decomposition results in a steady increase of the computational costs due to the parallel imbalance caused by the motion of the particles, leading finally to an inevitable breakdown of the simulation due to a memory bottleneck.
In constrast, the simulation with dynamic load balancing shows a nearly constant computational load throughout the simulation and is up to 300% faster, proving the efficiency of the newly developed method. The turbulent flow field and particle locations obtained in the simulation is shown in Fig. 2. In more detail, it was possible for the first time to determine the influcnce of the particle shape on the turbulent flow statistics.
The results from these simulations are presently summarized and will be submitted for publication in an international journal.
Dr.-Ing. Matthias Meinke
Chair of Fluid Mechanics and Institute of Aerodynamics, RWTH Aachen University
Wüllnerstraße 5a, D-52062 Aachen/Germany
e-mail: m.meinke (at) aia.rwth-aachen.de