.. _sma-ex5:

Ex. 5: Deterministic Calibration
================================

Suppose we have a complex model with numerous unknown parameters that we want to
calibrate to experimental data (well not that complex):

.. math::
    f(x, y) = |x|^a + |y|^b

Where the model inputs are :math:`x` and :math:`y`, and :math:`a` and :math:`b`
are unknown model parameters. We can use nodes available in nodeworks to
explore a collection of :math:`a` and :math:`b` parameter values and fit a
surrogate model to the response. We can then evaluate the surrogate model at
experimental conditions and compare the response of the surrogate model to the
experimental response. Finally, we can calibrate the :math:`a` and :math:`b`
parameters to the experimental data using the optimization node.

Step 1: Sample the model
------------------------

We need to generate samples for all four variables. To do this, add a
:ref:`sma-doe` node. Add the four variables and adjust the ranges to match the
following:

+----------+------+-----+
| Variable | From | To  |
+----------+------+-----+
| x        | -1.0 | 1.0 |
+----------+------+-----+
| y        | -1.0 | 1.0 |
+----------+------+-----+
| a        | 1.0  | 4.0 |
+----------+------+-----+
| b        | 1.0  | 4.0 |
+----------+------+-----+

Next, go to the ``Design`` tab of the :ref:`sma-doe` node. Change the ``Method``
to ``latin hypercube`` and the ``samples`` to ``100``. Finally, press ``Build``
to generate the samples.

Add a ``Code`` node next to the :ref:`sma-doe` node. Create a new terminal by
entering "samples" in the ``arguments`` field. Copy and paste the following
python code into the editor. This code will evaluate the model for each sample.

.. code-block:: python

   import numpy as np
   s = np.abs(np.asarray(samples[1:]))
   returnOut = s[:,0]**s[:,2] + s[:,1]**s[:,3]

Finally, connect the ``DOE Matrix`` terminal of the :ref:`sma-doe` node to the
``samples`` terminal of the ``Code`` node. Your sheet should now look like the
following:

.. figure:: ./images/ex5_sample.png
   :align: center

Step 2: Build the surrogate
---------------------------

Now we will construct a surrogate model to represent the response of the model.
Add a :ref:`sma-rsm` node next to the ``Code`` node. Connect the ``returnOut``
terminal from the ``Code`` node to the ``matrix/response`` terminal on the
:ref:`sma-rsm` node. Next, connect the ``DOE Matrix`` terminal of the
:ref:`sma-doe` node to the ``matrix/response`` terminal on the :ref:`sma-rsm`
node.

On the ``Model`` tab of the :ref:`sma-rsm` node, check the check mark next to
the ``radial basis function`` model. Run the sheet by pressing the |play|
button. The sheet should now look like:

.. figure:: ./images/ex5_surrogate.png
   :align: center

Step 3: Generate experimental data
----------------------------------

For this particular example, we are going to generate experimental data. This
allows use to play with the experimental data and explore the effects of samples
and noise on the effectiveness of the calibration. You can also read data from
``*.csv`` files.

Add another ``Code`` node below the :ref:`sma-rsm` node. Copy and paste the
following python code into the editor.

.. code-block:: python

   import numpy as np

   n = 100        # samples
   a = 2          # model parameter
   b = 3          # model parameter
   sigma = 0.01   # std of the noise

   # generate random x, y samples
   s = np.random.uniform(-1, 1, size=(n, 2))

   # make some noise!
   noise = np.random.normal(0, sigma, n)

   # calculate the response
   r = np.abs(s[:,0])**a + np.abs(s[:,1])**b + noise

   # combine the samples (x, y) with the response (r)
   returnOut = np.hstack((s, np.atleast_2d(r).T))


Step 4: Calibrate
-----------------

We are now ready to compare the model with the experimental data and minimize
the error. Add a :ref:`sma-res_func` node next to the :ref:`sma-rsm` node.
Connect the ``Model`` terminal of the :ref:`sma-rsm` node to the ``model``
terminal of the :ref:`sma-res_func` node. Also, connect the ``returnOut`` from
the second ``Code`` node to the ``experimental data`` terminal of the
:ref:`sma-res_func` node.

Press |play| to run the sheet and populate the table of the :ref:`sma-res_func`
node. Now that the table is populated, we can map the experimental values to the
model inputs. You will see the red error indication around :ref:`sma-res_func`
node with Show Error message icon on the top right of the node, which is normal
as the Experimental dataset is not mapped to the surrogate model parameters.

To map the experimental values to the model parameters, ``Double-click`` the
cell on the ``x`` row, ``experiment`` column and select ``0`` from the
combo-box. This will map the first column of the experimental data to the
:math:`x` value of the model. Likewise, select ``1`` for the ``y`` row and ``2``
for the ``response`` row. Your sheet should now look like the following.

.. figure:: ./images/ex5_res_func.png
   :align: center

Finally, add a :ref:`sma-optimizer` node next to the :ref:`sma-res_func` node.
Connect the ``calibration model`` terminal of the :ref:`sma-res_func` node to
``model`` terminal of the :ref:`sma-optimizer` node. On the :ref:`sma-optimizer`
node, change the number of attempts to ``10``. Finally, press |play| to run the
sheet. The complete sheet should look like the following, note that the Latin
Hypercube samples in Response Surface node and optimization results shown in
General Optimizer might be different form your worksheet due to random sample
generation:

.. figure:: ./images/ex5_complete.png
   :align: center

Step 5: How close did it get?
-----------------------------

In the second code node, we specified :math:`a = 2` and :math:`b = 3` when
generating the experimental data. The :ref:`sma-optimizer` node minimized the
error between the model and the experiment data. In this particular instance,
out of the 10 optimization attempts, the minimum error resulted in
:math:`a = 1.63` and :math:`b = 2.59`. So why are these not exactly
:math:`a = 2` and :math:`b = 3`?

Well, there are random processes involved in generating the samples and noise
added to the experimental data. Play with the number of model samples
(:ref:`sma-doe` node), using other surogate models (:ref:`sma-rsm` node),
and the amount of samples and noise in the experimental data (``n`` and
``sigma`` in the second ``Code`` node). How close can you get?

For example, changing the number of samples of the model from ``100`` to ``500``
in the :ref:`sma-doe` node and re-running the sheet resulted in a more accurate
response surface. The :ref:`sma-optimizer` then found the optimal values to be
:math:`a = 1.97` and :math:`b = 3.18`.


.. |play| image:: ../../../nodeworks/images/play.svg