MFiX Software Quality Assurance

Integrated Verification, Validation and Uncertainty Quantification (VVUQ) Framework ​

The complexity of governing physics in multiphase flows necessitates various assumptions to be incorporated in the MFiX model development. In addition, the input conditions to run the MFiX models may have some uncertainties. The objective of the verification, validation and uncertainty quantification (VV&UQ) framework is to assess the credibility of MFiX simulation results for various flow configurations of interest by considering these modeling assumptions and sources of various uncertainties. An integrated approach is employed by leveraging the well-established Verification and Validation (V&V) Standards (e.g., ASME V&V 20) and the Laboratory Studies at NETL for carrying out physical experiments to support the validation studies. In addition to other open-source UQ toolkits (e.g., PSUADE, DAKOTA), Nodeworks developed at NETL is heavily employed for constructing and executing the scientific workflows for various UQ analysis.

The activities under the VVUQ framework are mainly comprised of the following:

  • Verification and Validation (V&V) Studies
  • Uncertainty Quantification (UQ) Assessment Studies
  • Calibration Studies

Verification, Validation and Uncertainty Quantification Workflow

The Verification, Validation and Uncertainty Quantification workflow developed and employed for multiphase flows consists of multiple phases and toll gates to ensure consensus among all stakeholders within each phase. An overview of the workflow is illustrated in the figures on the right. Additional details can be found in Gel et al. (2018).

Source: Gel, A., Vaidheeswaran, A., Musser, J., and Tong, C. H. (November 22, 2018). “Toward the Development of a Verification, Validation, and Uncertainty Quantification Framework for Granular and Multiphase Flows—Part 1: Screening Study and Sensitivity Analysis.” ASME. J. Verif. Valid. Uncert. September 2018; 3(3): 031001.

Verification and Validation (V&V) Studies

Verification attempts to determine whether the equations representing the governing physics are correctly implemented and solved by the computer. On the other hand, validation attempts to answer if the correct equations are being solved by comparing simulation results with a controlled experimental observation at specific validation points [Roache 1997].

In the recent years, there has been a growing emphasis on V&V of MFiX suite, especially with the introduction of MFiX-PIC and MFiX-CGDEM. Each module consists of three main modeling aspects, viz., hydrodynamics, heat and mass transfer and chemical reactions. The current activities involve adopting a comprehensive V&V exercise to cover all these topics. However, there is a shortage of published datasets which have been objectively assessed. Also, most of the experiments reported in the literature lack adequate particle characterization. We follow the roadmap above to design and perform lab-scale experiments after engaging the subject matter experts. We use in-house characterization techniques to assess particle properties before using them in actual physical experiments. This includes advanced instrumentation and optical techniques as discussed in laboratory studies.

The coordinated effort between the modeling and experimental groups has led to great advances in V&V efforts besides aiding our understanding of granular and particle-laden flows (Shahnam et al. 2016, Gel et al. 2017, Vaidheeswaran et al. 2017, Xu et al. 2018, Gel et al. 2018, Bakshi et al. 2018, Fullmer et al. 2018, Xu et al. 2019, Gao et al. 2020, Higham et al. 2020, Vaidheeswaran et al. 2021a, Vaidheeswaran et al. 2021b, Vaidheeswaran et al. 2021c, Higham et el. 2021, Gel et al. 2021a, Gel et al. 2021b, Adepu et al. 2021).

  • Roache, P. J., 1997, “Quantification of Uncertainty in Computational Fluid Dynamics,” Annu. Rev. Fluid Mech., 29(1), pp. 123–160.
  • Shahnam, M., Gel, A., Dietiker, J. F., Subramaniyan, A. K., & Musser, J. (2016). The effect of grid resolution and reaction models in simulation of a fluidized bed gasifier through nonintrusive uncertainty quantification techniques. Journal of Verification, Validation and Uncertainty Quantification, 1(4).
  • Gel, A., Shahnam, M., & Subramaniyan, A. K. (2017). Quantifying uncertainty of a reacting multiphase flow in a bench-scale fluidized bed gasifier: A Bayesian approach. Powder Technology, 311, 484-495.
  • Vaidheeswaran, A., Shaffer, F., & Gopalan, B. (2017). Statistics of velocity fluctuations of Geldart A particles in a circulating fluidized bed riser. Physical Review Fluids, 2(11), 112301.
  • Xu, Y., Musser, J., Li, T., Gopalan, B., Panday, R., Tucker, J., Breault, G., Clarke, M.A. & Rogers, W.A. (2018). Numerical simulation and experimental study of the gas–solid flow behavior inside a full-loop circulating fluidized bed: Evaluation of different drag models. Industrial & Engineering Chemistry Research, 57(2), 740-750.
  • Gel, A., Vaidheeswaran, A., Musser, J., & Tong, C. H. (2018). Toward the Development of a Verification, Validation, and Uncertainty Quantification Framework for Granular and Multiphase Flows—Part 1: Screening Study and Sensitivity Analysis. Journal of Verification, Validation and Uncertainty Quantification, 3(3).
  • Bakshi, A., Shahnam, M., Gel, A., Li, T., Altantzis, C., Rogers, W., & Ghoniem, A. F. (2018). Comprehensive multivariate sensitivity analysis of CFD-DEM simulations: Critical model parameters and their impact on fluidization hydrodynamics. Powder Technology, 338, 519-537.
  • Fullmer, W. D., Webber, J., & VanEssendelft, D. (2018). Parameter Calibration and Uncertainty Quantification via Surrogate Model Optimization for CFD-DEM Modelling of a Small-scale Slugging Bed (No. 134229e4-c444-410b-b82a-15fa5e404a56). National Energy Technology Laboratory (NETL), Pittsburgh, PA, Morgantown, WV, and Albany, OR (United States). Energy Data eXchange; National Energy Technology Laboratory (NETL), Pittsburgh, PA, Morgantown, WV (United States).
  • Xu, Y., Shahnam, M., Fullmer, W. D., & Rogers, W. A. (2019). CFD-Driven Optimization of a Bench-Scale Fluidized Bed Biomass Gasifier using MFiX-TFM and Nodeworks-OT (No. bec2f001-387f-4b70-9d84-b4b6c6bb198d). National Energy Technology Laboratory (NETL), Pittsburgh, PA, Morgantown, WV, and Albany, OR (United States). Energy Data eXchange; National Energy Technology Laboratory (NETL), Pittsburgh, PA, Morgantown, WV (United States).
  • Gao, X., Yu, J., Li, C., Panday, R., Xu, Y., Li, T., Ashfaq, H., Hughes, B. & Rogers, W. A. (2020). Comprehensive experimental investigation on biomass‐glass beads binary fluidization: A data set for CFD model validation. AIChE Journal, 66(2), e16843.
  • Higham, J. E., Shahnam, M., & Vaidheeswaran, A. (2020). Using a proper orthogonal decomposition to elucidate features in granular flows. Granular Matter, 22(4), 1-13.
  • Vaidheeswaran, A., & Rowan, S. (2021). Chaos and recurrence analyses of pressure signals from bubbling fluidized beds. Chaos, Solitons & Fractals, 142, 110354.
  • Vaidheeswaran, A., Li, C., Ashfaq, H., Wu, X., Rowan, S., & Rogers, W. A. (2021). Validation experiments on bubbling fluidization of Group B glass particles. Experimental and Computational Multiphase Flow, 1-10.
  • Vaidheeswaran, A., Gel, A., Clarke, M. A., & Rogers, W. A. (2021). Assessment of model parameters in MFiX particle-in-cell approach. Advanced Powder Technology, 32(8), 2962-2977.
  • Higham, J. E., Shahnam, M., & Vaidheeswaran, A. (2021). Anomalous diffusion in a bench-scale pulsed fluidized bed. Physical Review E, 103(4), 043103.
  • Gel, Aytekin, Weber, Justin, & Vaidheeswaran, Avinash. Sensitivity Analysis of MFiX-PIC Parameters Using Nodeworks, PSUADE, and DAKOTA. United States.
  • Gel, Aytekin, Vaidheeswaran, Avinash, & Clarke, Mary Ann. Deterministic Calibration of MFiX-PIC, Part 1: Settling Bed. United States.
  • Adepu, M., Chen, S., Jiao, Y., Gel, A., & Emady, H. (2021). Wall to particle bed contact conduction heat transfer in a rotary drum using DEM. Computational Particle Mechanics, 8(3), 589-599.

Uncertainty Quantification (UQ) Assessment Studies

UQ assessment aims to identify, characterize, and propagate various sources of uncertainties in MFiX simulations. One of the critical components of a UQ study is to identify and characterize these uncertainties, which could be due to model parameters, input variables or the inherent assumptions in the model. In addition, the quantities of interest (a.k.a. response variables) are identified to quantify the observed discrepancy. These steps are usually carried out with the feedback from all stakeholders (such as the engineer(s) concerned and subject matter experts). Once a consensus is reached, the uncertainties are characterized as aleatoric, epistemic or mixed. If the number of uncertain variables and parameters are high, sensitivity analysis is employed to assess the dominant ones to narrow down the scope and reduce computational burden. Sensitivity analysis can be also used to plan for follow-up studies. For example, additional physical experiments can be planned to reduce the uncertainties in the select set of model variables, which have been identified to have significant influence. Another UQ study is the forward propagation of the uncertainties, which enables the stakeholders to assess how the quantities of interest are affected and to what extent given the variability in input variables.

Due to the complex governing physics and the nature of computational models, a non-intrusive UQ analysis is preferable and have been adopted in our efforts. In this approach, MFiX solver is treated as a black-box and simulations are performed with the aid of well-established sampling methods (e.g., Optimal Latin Hypercube) in a systematic manner. The variability in the quantities of interest due to input uncertainties are captured using an adequate sampling size. For simulations where the computational cost per sample is high, surrogate model construction (a.k.a. response surface model) is employed to characterize the relationship between the selected set of uncertain input parameters and the quantities of interest. The surrogate models are subsequently used in lieu of the actual simulations in the UQ analysis which requires many function evaluations.

Calibration Studies

Calibration aims to improve the accuracy of MFiX simulations for a given flow configuration by tuning the uncertain model parameters (e.g., heat transfer coefficient in the transient temperature behavior of a fluidized bed simulation) under the guidance of available experimental data. It is important to note the difference between validation and calibration. Validation is a direct comparison of simulation results to experimental data without tuning and is typically employed to establish a baseline comparison between an experiment and a simulation. The identified discrepancy can then justify the need for model calibration. Both validation and calibration are always performed against a specific set of observable data, which makes the credibility of experimental results critical. Hence, careful consideration must be given when generalizing the insights gained from calibration studies, particularly when applying previously calibrated input parameters to new simulations.

Calibration performed for MFiX can be categorized under two methodologies: (i) deterministic, and (ii) statistical. The latter provides a distribution for the calibrated model parameters instead of single values, which is the outcome of deterministic calibration. Another major difference is the ability of statistical calibration to take into account model bias (a.k.a. model form uncertainty) while calibrating input model parameters.