Optimization Techniques
MATH:4820/CS:4720 (22M:174/22C:174)

The University of Iowa
The College of Liberal Arts and Sciences
Department of Mathematics
You can see me outside the office hours listed above provided it is mutually convenient.
Some of the policies relating to this course (such as the drop deadline) are governed by its administrative home, the College of Liberal Arts and Sciences, 120 Schaeffer Hall.

Description of Course:

This course covers the theory and practice of numerical optimization: numerically estimating the maximum or minimum of a function, possibly subject to a number of either equality or inequality constraints. On the theoretical side we will talk about necessary and sufficient condition for local minima and maxima with or without constraints, convex sets and functions, and the analysis of algorithms. On the practical side we will consider a wide range of algorithms and their implementation, including line search and trust region techniques.

Detailed list of topics

If there is time I will start looking at dynamic optimization problems and optimal control, or optimization problems related to “big data” tasks, depending on student interest.

Objectives and Goals of the Course:

The students will be introduced to the derivation, implementation, and analysis of numerical methods in the areas of numerical solution of ordinary differential equations and matrix computations, especially, solution of linear systems, solution of linear least-squares problems, and computation of eigenvalues and eigenvectors. Students are expected to become proficient in all aspects covered. This involves both theory (mathematial analysis) and practice (programming and implementation). While learning a particular programming language is not necessary, examples and demonstrations will be given in Matlab TM and students are expected to be able to read and understand code in this language.
There will be a more-or-less open-ended project for all students to complete which will involve work in all three aspects (derivation, implementation, and analysis) of numerical methods.


Required text: Numerical Optimization, 2nd Edition by S. Wright and J. Nocedal, which is published by Springer.

Grading System and the Use of +/–:

A +/– grading system will be used for this course. As noted on the CLAS teaching policy page, a score of A+ is a rare grade, assigned only in the case of exceptional work.

Assessment: Assignments and Percentage of Final Grade:

There will be an in-class exam and a final exam (20% each), homework assignments (40% total) which will include some programming, and a programming assignment (20%). There may also be ungraded homework assigned during the course. Familiarity with MATLAB will be useful during the course.
Please remember that final exams may only be given during finals week according to CLAS policy. Likewise, no major exams may be given the week before finals week.

A Word about the Date and Time of the Final Exam:

The date and time of every final examination is announced by the Registrar generally by the tenth day of classes. No exams of any kind are allowed during the last week of classes. All students should plan on being at the UI through the final examination period. Once the Registrar has announced the date, time, and location of each final exam, the complete schedule will be published on the Registrar’s web site and will be shared with instructors and students. It is the student's responsibility to know the date, time, and place of the final exam.

Course Policies:

Attendance and participation:

Students are expected to attend all classes unless there is an important and pressing reason otherwise. Attending students are expected to participate in classroom discussion and be prepared to give relevant comments on topics discussed in class in a civil and respectful manner.

Timely completion of assignments:

All assignments (homework and project) are expected to be completed and handed in on time. If you have questions about the homework or project, you should contact the TA or instructor. Unless prior arrangements have been made, late homework is subject to a 20% reduction in the grade.

Student Collaboration:

Students may discuss the homework and project. However, all work handed in must be the work of the student submitting the work and in their own words. All duplicated work will be considered plagiarized and graded as such. Any outside source of information must be cited in a way that any third person can obtain the cited work and check its veracity.
All exams are purely individual and there can be no discussion between students about the material in an exam during that exam whether in class or take-home.

Resources for Students:

Students may find the Writing Center and the Speaking Center very useful for this course:
Writing Center: http://writingcenter.uiowa.edu/
Speaking: Center: http://speakingcenter.uiowa.edu/

Calendar of Course Assignments and Exams:

Homework will be scheduled regularly, approximately every two weeks except near the end of semester. The programming project will take place during the last four or five weeks of semester. This will involve a significant amount of programming and interpretation of the results. Students should be prepared to write a substantial report describing what they did for the project and their results and conclusions.
The midterm exam will be held on Friday, March 4th (Friday of week 7).