Optimization Lectures

Unconstrained problems: Newton's method as the paradigm for function minimization and the 
solution of systems of nonlinear equations. Kantorovich theory and the question of existence. 
How far can we go with least squares? Some implementation questions (role of transformation 
invariance, improving convergence properties). Singular Jacobians can cause chaos.

Constrained problems: Continuous and discrete problems. Convexity in mathematical programming. 
Necessary conditions. Fenchel and Lagrange duality. Penalty methods. Linear and quadratic 
programming. Active set and interior point methods. Sequential quadratic programming. 

References
Newton Methods for Nonlinear Problems, Deuflhard, Springer, 2005.
Numerical Optimization, Nocedal and Wright, Springer, 1999.
Simplicial Methods for Minimizing Polyhedral Functions, Osborne, Wiley, 2000. 
Lecture notes on least squares and maximum likelihood. 

Here are links to ps files on (1)convex analysis, (2)linear programming and (3)the simplex method.

A bibliography link is here. 

Unconstrained optimization:

Exercise 1            Exercise 2            Exercise 3 + additional information            Exercise 4        

Example 1           Example 2

Constrained problems (i) - convex analysis

 Exercise 1          Exercise 2

Constrained problems (ii) - multiplier conditions, duality

Exercise 1           Exercise 2

Linear  programming

Exercise 1

Mathematical programming

Exercise 1