.. _sma-optimizer:

General Optimizer
=================

The ``General Optimizer`` is used to find the minimum, maximum, or root of the
model. This can be used to find the optimal solution (sample) of the quantity of
interest.

.. image:: images/optimizer.gif
    :align: center

.. warning::
    Using a black-box approach to optimization does not guarantee that the
    global minimum, maximum, or root has been found. To overcome this, see
    :ref:`sma-optimizer-attempts`

Minimize and Maximize
---------------------

To find the maximum or minimum of the model, select the operation from the
``Operation`` combo-box. Maximization is treated exactly like minimization
except the model is multiplied by a ``-1``.

There are two global methods and 8 local methods implemented in
`scipy.optimize <https://docs.scipy.org/doc/scipy/reference/optimize.html>`_
that can be used. Depending on the method selected, other options maybe be
exposed.

Global methods:

* `Basin Hopping <https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.basinhopping.html>`_
* `Differential Evolution <https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.differential_evolution.html>`_
* `Simplicial homology global optimization (SHGO) <https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.shgo.html>`_
* `Dual annealing <https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.dual_annealing.html>`_

Local methods (select ``local`` in the ``Method`` combo-box):

* `Nelder-Mean <https://docs.scipy.org/doc/scipy/reference/optimize.minimize-neldermead.html>`_
* `Powell <https://docs.scipy.org/doc/scipy/reference/optimize.minimize-powell.html>`_
* `Conjugate Gradient (CG) <https://docs.scipy.org/doc/scipy/reference/optimize.minimize-cg.html>`_
* `quasi-Newton method of Broyden, Fletcher, Goldfarb, and Shanno (BFGS) <https://docs.scipy.org/doc/scipy/reference/optimize.minimize-bfgs.html>`_
* `L-BFGS-B <https://docs.scipy.org/doc/scipy/reference/optimize.minimize-lbfgsb.html>`_
* `truncated Newton algorithm (TNC) <https://docs.scipy.org/doc/scipy/reference/optimize.minimize-tnc.html>`_
* `Constrained Optimization BY Linear Approximation (COBYLA) <https://docs.scipy.org/doc/scipy/reference/optimize.minimize-cobyla.html>`_
* `Sequential Least SQuares Programming (SLSQP) <https://docs.scipy.org/doc/scipy/reference/optimize.minimize-slsqp.html>`_

Root
----

To find the root of the model, select the ``find value (root)`` from the
``Operation`` combo-box. The method can be selected using the ``Method``
combo-box. The available root finding methods are:

* `Powell hybrid method (hybr) <https://docs.scipy.org/doc/scipy/reference/optimize.root-hybr.html>`_
* `Levenberg-Marquardt (lm) <https://docs.scipy.org/doc/scipy/reference/optimize.root-lm.html>`_
* `Broyden’s first Jacobian approximation (broyden1) <https://docs.scipy.org/doc/scipy/reference/optimize.root-broyden1.html>`_
* `Broyden’s second Jacobian approximation (broyden2) <https://docs.scipy.org/doc/scipy/reference/optimize.root-broyden2.html>`_
* `Anderson mixing (anderson) <https://docs.scipy.org/doc/scipy/reference/optimize.root-anderson.html>`_
* `Scalar Jacobian approximation (linearmixing) <https://docs.scipy.org/doc/scipy/reference/optimize.root-linearmixing.html>`_
* `Diagonal Broyden Jacobian approximation (diagbroyden) <https://docs.scipy.org/doc/scipy/reference/optimize.root-diagbroyden.html>`_
* `Tuned diagonal Jacobian approximation (exitingmixing) <https://docs.scipy.org/doc/scipy/reference/optimize.root-excitingmixing.html>`_
* `Krylov approximation for inverse Jacobian (krylov) <https://docs.scipy.org/doc/scipy/reference/optimize.root-krylov.html>`_
* `Derivative-free spectral method (df-sane) <https://docs.scipy.org/doc/scipy/reference/optimize.root-dfsane.html>`_

Enter the value to find in the ``Value`` line-edit.

.. _sma-optimizer-attempts:

Attempts
--------

As a tool to overcome the issue of the black-box approach to optimization not
guaranteeing that the global minimum, maximum, or root has been found, the
operation can be run multiple times with a random initial condition. The number
of attempts can be entered in the ``Attempts`` spin-box.
`numpy.random.random <https://www.numpy.org/devdocs/reference/generated/numpy.random.random.html>`_
is used to pick the random initial condition within the bounds of the fit model.
Each random initial condition is then used in the operation, with the solutions
being populated in the table. If the independently run solutions converge, then
the confidence that the converged solution is correct is increased.

Optimize
--------

Once the desired options have been selected, press the ``Optimize`` button.
This could take a significant amount of time to complete, especially if many
attempts are being performed. When finished, the sample and value for each
attempt will be displayed in the table. This table can be sorted by pressing
the column headers.

Plot
----

The plot tab will show the sampled model and all the optimization attempts in
either 3D or 2D. The plot type can be changed by selecting the plot from the
``Plot`` combo-box. If there are more than two variables, the currently x and y
axis can be changed by selecting the variables from ``X Axis`` and ``Y axis``
combo-boxes.

Parallel Plot
-------------

Sometimes the optimization routine will find different optimums. To help
visualize these optimums and look for groupings of values, the parallel plot
plots each optimization attempt. Options across the top provide tools to help
visualize the attempts.

.. image:: images/opt_parallel_plot.png
    :align: center
    :width: 500