.. _ml-nn_regressor: Neural Network Regressor ======================== The ``Neural Network Regressor`` node has been designed to mimic the :ref:`sma-rsm` node, but tailored to construct neural networks. This node supports multiple continuous input variables and only one response variable. Currently, the node uses `PyTorch <https://pytorch.org/>`_ and will automatically use the GPU if the drivers are installed correctly. The resulting neural network can be used with the other from the ``surrogate modeling and analysis`` toolset such as the :ref:`sma-optimizer` node, :ref:`sma-fprop` node, :ref:`sma-sa` node, and the :ref:`sma-res_func` node. The main controls of the node persist at the bottom, consisting of ``Train``, ``Import``, and ``Export`` buttons. The ``Train`` button will start training the neural network. The ``Import`` button will open a browse dialog that can be used to select an exported neural network model to load. Finally, the ``Export`` button will open a browse dialog that can be used to export the trained model. .. note:: If the ``Neural Network Regressor`` node does not appear in your list of available nodes, please install PyTorch using PyTorch's install instructions located `here <https://pytorch.org/get-started/locally/>`_. .. _ml-nn_regressor-data: Data ---- In the ``Data`` tab the required parameter input and corresponding full model response values are entered into the node for the construction of the response surface. There are two ways in which the required data can be input. In a typical workflow, the input matrix (typically from a :ref:`sma-doe` node) and the corresponding model response (typically from a ``code`` node) are connected to the ``matrix/response`` terminal. Alternatively, if the data has been generated externally or exported from Nodeworks nodes and agglomerated into a single table, the ``Import`` button at the top of the table can be used to read in a ``csv`` file. Once the input and response data has been loaded into the ``Data`` tab, it can be sorted by individual input variables or the (full model) Response. The table can be returned to the original index order by right clicking and selecting ``clear sort``. The right click menu also offers the capability to ``Exclude`` or ``Include`` specific entries. .. figure:: ./images/nn_regressor_data.png :align: center .. _ml-nn_regressor-model: Model ----- The structure of the neural network is defined on the ``Model`` tab. Layers can be clicked and dragged from the ``Available layers`` list to the ``Model layers`` list. Layers can also be added by double clicking layers in the ``Available layers`` list. The ``Available layers`` list can also be filtered by entering text in the ``Search`` field. Individual layer options can be edited by selecting a layer in the ``Model layers`` list and changing the values in the widgets that appear under the ``Layer options``. .. figure:: ./images/nn_regressor_model.png :align: center .. _ml-nn_regressor-train: Train ----- Training parameters are selected on the ``Train`` tab. This includes the number of epochs and the percentage of samples to hold out for cross validation. During training, the plot will update, display the current epoch error metric for both the training data and the test data (if the cross validation percent is greater than 0). .. figure:: ./images/nn_regressor_train.png :align: center .. _ml-nn_regressor-error_metrics: Error Metrics ------------- The table on the ``Error Metrics`` tab shows several quantitative error metrics that can be used to assess the fitness of the trained neural network. :ref:`appx-smathry-rsmerr`: * ``MSE`` - :ref:`appx-smathry-rsmerr-mse` * ``SSE`` - :ref:`appx-smathry-rsmerr-sse` * ``R^2`` - :ref:`appx-smathry-rsmerr-rsq` * ``L_inf`` - :ref:`appx-smathry-rsmerr-linf` * ``L_1`` - :ref:`appx-smathry-rsmerr-l1` * ``L_2`` - :ref:`appx-smathry-rsmerr-l2` The error metrics operate on either the full dataset or the out-of-sample subset for `cross validation <https://en.wikipedia.org/wiki/Cross-validation_(statistics)>`_. In cross validation, a subset of the dataset is withheld from the training of the model. By default, 10 percent of the dataset is withheld as ``Cross validation points``. When cross validation is active, the :ref:`appx-smathry-rsmerr`: only apply to the withheld points. Without cross validation, the :ref:`appx-smathry-rsmerr`: apply to the whole dataset. .. figure:: ./images/nn_regressor_error_metrics.png :align: center .. _ml-nn_regressor-error: Error ----- The difference between the model and the data is visualized in the ``Error`` tab. The discrepancy for a given model can be viewed in three different forms selectable from the ``Plot`` dropdown menu: * ``parity`` plot * ``error`` plot * error ``histogram`` As in the ``Plot`` tab of this and the :ref:`sma-rsm` node, points can be highlighted by holding select and dragging the cursor. Highlighted points can be excluded (and included if previously excluded) from the fit of the model. .. figure:: ./images/nn_regressor_error.png :align: center .. _ml-nn_regressor-plot: Plot ---- The trained model can be visualized in either ``3D`` or as a ``contour`` plot, as shown below, in the ``Plot`` tab. If more than two input variables are used, the variables used for the ``X Axis`` and ``Y Axis`` become selectable from dropdown menus below the plot. For the variables not plotted on either the ``X Axis`` or ``Y Axis``, the surface will be evaluated at the center of the variable range. The user can change these values by selecting the ``variable values`` dropdown and adjusting either the slider or entering a new value in the line edit. Response data points are selectable and can be excluded from the fit or shown in the ``Data`` table from the right click menu. .. figure:: ./images/nn_regressor_plot.png :align: center