Unit 2956: Optimization Theory and Applications

1. Introduction to Optimization Theory

• Overview of optimization theory
• Applications in engineering, economics, computer science, and operations research
• Fundamental concepts:
  • Objective functions
  • Decision variables
  • Constraints (equality and inequality)
  • Feasible solutions and feasible region

2. Unconstrained Optimization

• Definition and examples
• Local vs. global optima
• Methods:
  • Gradient descent
  • Newton's method
• Convexity:
  • Convex sets and convex functions
  • Importance of convexity in optimization

3. Constrained Optimization

• Types of constraints:
  • Equality constraints
  • Inequality constraints
• Lagrange multipliers method
• Karush-Kuhn-Tucker (KKT) conditions:
  • Necessary and sufficient conditions
  • Interpretation and applications

4. Linear Programming

• Introduction and problem formulation
• Graphical solution method (2-variable problems)
• The simplex method:
  • Basic feasible solutions
  • Pivot operations
  • Optimality conditions
• Duality theory:
  • Primal and dual problems
  • Duality theorems
• Sensitivity analysis:
  • Changes in coefficients and constraints
  • Impact on optimal solution

5. Nonlinear Programming

• Characteristics of nonlinear problems
• Nonlinear objective functions and constraints
• Optimization algorithms:
  • Gradient descent and variants
  • Newton's method for nonlinear problems
  • Interior point methods
• Convergence and complexity considerations

6. Integer Programming

• Definition and importance of discrete decision variables
• Integer linear programming formulation
• Solution methods:
  • Branch and bound algorithm
  • Cutting plane methods
• Applications:
  • Project scheduling
  • Network design

7. Dynamic Programming

• Concept and motivation
• Principle of optimality
• Bellman’s equations
• Applications:
  • Resource allocation problems
  • Shortest path problems

8. Multi-Objective Optimization

• Introduction to problems with multiple conflicting objectives
• Pareto optimality and Pareto front
• Trade-off analysis techniques
• Solution methods:
  • Weighted sum method
  • Goal programming

9. Heuristic Optimization

• Overview of heuristic and metaheuristic methods
• Genetic algorithms:
  • Representation, selection, crossover, mutation
• Simulated annealing
• Ant colony optimization
• Particle swarm optimization
• Applications in complex optimization problems

Summary and Integration

• Recap of major topics
• Discussion of how methods complement each other
• Case studies or real-world examples