.. _sma-ex3: Ex. 3: Sensitivity Analysis --------------------------- In this example, we will demonstrate global sensitivity analysis with the aid of a surrogate model. Here, the surrogate will represent a heat exchanger. Sampling of the input parameter space and collection of the response variable (a.k.a. quantity of interest) has already been conducted in a previous study, which we will use to construct a new surrogate model for global SA. Dataset is generated from the study of Qian et al. [Qian2006]_ where 64 orthogonal array-based Latin Hypercube design samples where generated for four input parameters, which the study aims to analyze the impact of these factors on heat transfer rates: .. tabularcolumns:: |c|c|c|c| +-----+---------------------+-----------+-----------+ | No | Variable Name | Symbol | Unit | +=====+=====================+===========+===========+ | x1 | Mass Flow Rate of | m_flowrate| kg/s | | | entry air | | | +-----+---------------------+-----------+-----------+ | x2 | Temp. of entry air | T_in | K | +-----+---------------------+-----------+-----------+ | x3 | Solid Material | | | | | Thermal Conductivity| k | W/mK | +-----+---------------------+-----------+-----------+ | x4 | Temperature of the | | | | | heat source | T_wall | K | +-----+---------------------+-----------+-----------+ For the details of the study and dataset itself reader is referred to [Qian2006]_ Table 2 on page 673. The dataset used for this example can be downloaded :download:`here <./data/LH_n64_i4_o1.csv>` in Comma-Separated Values (CSV) format to facilitate import directly to the ``Response Surface Node``. The CSV input file also contains a fifth column, which is the quantity of interest (or response variable), A copy of the input file has been provided at the end of this exercise in tabulated format to facilitate quick review of the imported dataset, which can be also reviewed in detail within the ``Response Surface Node``. This exercise will show how to construct a surrogate model from an external dataset imported to Nodeworks directly and perform sensitivity analysis to identify the most important input parameters among the four. A completed workflow showing the results of the sensitivity analysis graphically is presented in the following figure. .. figure:: ./images/ex3_SAworkflow.png :align: center .. _sma-ex3-build: Build the Surrogate +++++++++++++++++++ In this section, we'll build a surrogate model for the externally created design of experiments. The original DoE employed Latin Hypercube sampling based design for space filling property. The corresponding response variables were computed with finite difference simulations. The SA aims to determine which of the input parameters among the four has the most influence in the response variable. 1. Right click somewhere in the blank worksheet then click ``Add Node`` and then ``Surrogate Modeling Analysis`` option to select and add a ``Response Surface`` node as shown in the figure below. .. image:: ./images/ex3_AddNode_RSM.png :align: center :width: 350 2. In the ``Response Surface`` node click on the ``Import`` button in the first tab under ``Data`` This will open a new window titled ``Read CSV File`` as shown below. .. image:: ./images/ex3_RSMDataImportBlank.png :align: center 3. Select the datafile in CSV format, which contains the 64 sample Latin Hypercube for four parameters (m_flowrate, T_in, k, and T_wall) and one response variable (`y_a`) by clicking the Browse button. Once the correct file is located, the available dataset will be previewed in the window. Unclick the last entry `y_a` to remove from the ``Sample Columns`` list which is used to designate which columns in the file will be treated as input parameters. In the entry for ``Response Column``, use the scroll bar on the right side to scroll down and select `y_a` as the response variable. To import the dataset into Nodeworks click ``Import`` button after making the above changes in the window. .. image:: ./images/ex3_RSMDataImport_ReadCSVfile.png :align: center :width: 400 After successfully importing the dataset, the ``Response Surface`` node should show the imported dataset under Data tab as shown for the first 20 rows in following figure. .. image:: ./images/ex3_RSM_AfterData_Import.png :align: center :width: 400 4. Next step involves testing different surrogate model methods to assess the best data-fitted surrogate for the given dataset. In the ``Response Surface`` node, click on the ``Model`` tab and set the ``Cross validation points`` to 0. Next check and highlight several surrogate model options that is expected to give best fit. For this example, we will select ``gaussian process``, ``polynomial``, ``linear``, ``radial basis function``, and ``MARS``. Note that the wall clock time required to construct surrogate models with more options may noticeably increase as the sample size substantially increases (e.g., from 64 to 10000 samples). Each surrogate model option comes with its own set of parameters for tuning the process, which will not be changed for this example, and the defaults will be accepted. However, user is encouraged to understand these parameters and explore their effect while in pursuit of the best data-fitted surrogate model at this phase. The selection of the surrogate models before initiating the process for construction are shown below. .. image:: ./images/ex3_RSMModelSelectedHitRefit.png :align: center :width: 400 To initiate the surrogate model construction process hit ``Refit Model(s)`` button where the mouse cursor in the above figure is placed. After the process is completed user should see surrogate model quality metric calculated as shown below. Note that you may need to enlarge the node towards right side to be able to see the additional metrics shown in the figure. Clicking the header of any column triggers sort based on the numeric values in that column, which helps to better assess which method achieved the best fit based on the surrogate model quality metric such R :sup:`2`. .. image:: ./images/ex3_RSMModelFitted.png :align: center :width: 400 After reviewing the metrics to assess the quality of the fitted response and/or graphical visualizations showing the error for fitted response surface predictions at the actual sample locations, select the best suitable response surface in the ``Output Model`` field at the right bottom using the scroll bar as shown in the below illustration. .. image:: ./images/ex3_RSMModelFitted_ErrorHistogram_GPM.png :align: center :width: 400 5. After constructing an adequate response surface to characterize the relationship between input and outputs, add a ``Sensitivity Analysis`` node from Surrogate Modeling and Analysis collection of the node library. .. image:: ./images/ex3_RSMModelFitted_AddSA_Node.png :align: center Then, connect the ``Model`` terminal of the Response Surface node to the ``model`` terminal of the Sensitivity node, as shown in the illustration below. .. image:: ./images/ex3_RSMModelFitted_AddSA_Node_ClickCalcSensi.png :align: center To start the Sensitivity Analysis, click ``Calculate Sensitivities`` at the bottom of the ``Options`` tab. As shown below, it will retrieve the input parameter dataset from the Response Surface Node. The lower and upper bounds of the input parameters will be displayed in this tab after the completion of analysis. .. image:: ./images/ex3_RSMModelFitted_AddedSA_Node_Options.png :align: center The results of the Sensitivity Analysis can be visualized in the ``Plot`` tab. As shown in the following illustration, after clicking the ``Plot`` tab a bar chart is created. First order Sobols' indices show the main effects, i.e., the effect of the each input parameter considered standalone. The second order indices show the interaction effects such as interaction of mass flow rate with inlet temperature (labeled as ``m_flowrate:T_in`` in the x-axis of the bar chart). Total indices shown with blue color is an aggregate result for the overall effect of each input parameter. The bar chart shows that among the four input parameters, the wall temperature has the most influence on the system response. .. image:: ./images/ex3_RSMModelFitted_AddedSA_Node_Plot.png :align: center The remaining tabs in the Sensitivity Analysis node (i.e., Total, First Order and Second Order) show quantitatively the values of the sensitivity indices displayed graphically in ``Plot`` tab in a tabulated format as shown in the following figures. .. image:: ./images/ex3_RSMModelFitted_AddedSA_Node_Total.png :align: center In the tabulated data compiled, first and second columns show the computed Sobols' indices and confidence interval, respectively. .. image:: ./images/ex3_RSMModelFitted_AddedSA_Node_FO.png :align: center .. image:: ./images/ex3_RSMModelFitted_AddedSA_Node_SO.png :align: center References: +++++++++++ .. [Qian2006] Qian, Zhiguang, Carolyn Conner Seepersad, V. Roshan Joseph, Janet K. Allen, and CF Jeff Wu. "Building surrogate models based on detailed and approximate simulations." Journal of Mechanical Design 128, no. 4 (2006): 668-677. External Dataset: +++++++++++++++++ .. csv-table:: The contents of ``data/LH_n64_i4_o1.csv`` :widths: 10 10 10 10 10 :header: "x1", "x2", "x3", "x4", "y1" :file: ./data/LH_n64_i4_o1.csv