petsc-3.7.3 2016-07-24
Report Typos and Errors

## TAO Hands On

TAO (as a component of the PETSc distribution) comes with a number of example programs to help demonstrate how to solve optimization problems with the TAO algorithms. These examples can be found in

- Unconstrained Optimization - src/tao/unconstrained/examples/tutorials
- Bound-constrained Optimization - src/tao/bound/examples/tutorials
- Complementarity Problems - src/tao/complementarity/examples/tutorials
- Derivative-free Nonlinear Least Squares - src/tao/leastsquares/examples/tutorials

## Example 1: Unconstrained optimization of the Rosenbrock function

WHAT THIS EXAMPLE DEMONSTRATES

- Using command line options
- Changing the default solver algorithm
- Using call-back functions
- Exploring derivative-free options

FURTHER DETAILS:

- Problem Description - Rosenbrock Function: F(x,y) = (1-x)2 + 100*(y-x2)2
- the source code

DO THE FOLLOWING:

- Compile petsc/src/tao/unconstrained/examples/tutorials/rosenbrock1.c

            cd petsc/src/tao/unconstrained/examples/tutorials
            make rosenbrock1

- Run the example with the default solver options, using the first order quasi-newton solver lmvm

            mpiexec -n 1 ./rosenbrock1 -tao_monitor -tao_view  [Expected output]

- Run the example again using an algorithm that requires hessian evaluations (Newton line search)

           mpiexec -n 1 ./rosenbrock1 -tao_monitor -tao_view -tao_type nls  [Expected output]

- Run the example again using no derivatives, both using a derivative-free algorithm (Nelder-Mead) and with a finite difference approximation. Finite difference approximations may be suitable for this example because the number of variables is small, but higher dimensional applications will find this approach much less practical.

  Notice in the output that (as should be expected) the derivative-free method (nm) takes many more iterations than methods that use derivatives, and that using finite differences has little affect on the iteration count but a large affect on the number of function evaluations.

            mpiexec -n 1 ./rosenbrock1 -tao_monitor -tao_view -tao_type nm [Expected output]

            mpiexec -n 1 ./rosenbrock1 -tao_monitor -tao_view -tao_type cg [Expected output]

            mpiexec -n 1 ./rosenbrock1 -tao_monitor -tao_view -tao_type cg -tao_fd_gradient [Expected output]

## Example 2: Bound-constrained example -- Pressure distribution in a journal bearing

WHAT THIS EXAMPLE DEMONSTRATES

- Setting bounds on variables
- Experiment with various bound-constrained algorithms
- Handling a 2d structured grid
- Using MPI for distributed applications

FURTHER DETAILS:

- Problem Description - pressure in a journal bearing MINPACK-2 (Page 33)

By discretizing the domain into triangular elements, the pressure surrounding the journal bearing is defined as the minimum of a quadratic function whose variables are bounded below by zero.

- the source code

DO THE FOLLOWING:

- Compile src/tao/bound/examples/tutorials/jbearing2.c

            cd petsc/src/tao/bound/examples/tutorials
            make jbearing2

- Run on 1 process on a 32x32 structured mesh

            mpiexec -n 1 ./jbearing2 -mx 32 -my 32 -tao_monitor -tao_view -tao_type blmvm [Expected output]

- Run on 16 processes using more grid points

            mpiexec -n 16 ./jbearing2 -mx 256 -my 256 -tao_monitor -tao_view -tao_type blmvm [Expected output]

            mpiexec -n 16 ./jbearing2 -mx 256 -my 256 -tao_monitor -tao_view -tao_type tron [Expected output]

            mpiexec -n 16 ./jbearing2 -mx 256 -my 256 -tao_monitor -tao_view -tao_type gpcg [Expected output]

  Notice that the gpcg algorithm assumes the application's objective function is quadratic, so the hessian does not need to be explicitly computed after the first iteration.

## Example 3: Unconstrained example -- Minimal surface

WHAT THIS EXAMPLE DEMONSTRATES

- Using PETSc utilities to profile applications
- Revisit derivative free options
- Checking the correctness of the gradient

FURTHER DETAILS:

- Problem description - Minimal surface area with boundary conditions MINPACK-2 (Page 34)
- the source code

DO THE FOLLOWING:

- Compile src/tao/unconstrained/examples/tutorials/minsurf2.c

            cd petsc/src/tao/unconstrained/examples/tutorials
            make minsurf2

- Run a 16-process on a 32x32 structured mesh using the quasi-Newton lmvm algorithm

            mpiexec -n 16 ./minsurf2 -mx 32 -my 32 -tao_type lmvm [Expected output]

- Now try running the same application using the derivative-free Nelder-Mead algorithm (nm). Notice that when we used this algorithm on the two-variable Rosenbrock example previously, the performance was worse than lmvm but not unusable. Increasing the number of variables past 100 makes this algorithm unusable for most purposes. This example fails to converge after a function evaluation budget of 4000 (This budget can be lifted using the command line option -tao_max_funcs n) compared to only around 20 function evaluations needed for lmvm.

            mpiexec -n 16 ./minsurf2 -mx 32 -my 32 -tao_type nm [Expected output]

- Likewise, using a finite difference approximation to the gradient will also become a problem with higher dimensional problems:

            mpiexec -n 16 ./minsurf2 -mx 32 -my 32 -tao_type lmvm -tao_fd_gradient [Expected output]

- However, there is still a good reason for using finite difference approximations in higher dimensional problems. By far the most common reason for a TAO algorithm to fail to converge is that there is an error in the user-provided gradient routine. Errors in the Hessian are less drastic and will usually only slow down the convergence rate. TAO (as well as PETSc) have devices for testing whether or not the user-provided gradients and hessians are "correct" by comparing the computations of the user-provided routines against results obtained through using finite difference approximations. Test the accuracy of the user-provided gradient routine in the lmvm function. This will not solve the minimization problem, but only compute the user-provided gradient and a finite-difference approximation to the gradient at several vectrs in the solution space.

            mpiexec -n 16 ./minsurf2 -mx 32 -my 32 -tao_type test [Expected output]