Optimization Techniques
MATH:4820/CS:4720 (22M:174/22C:174)
The University of Iowa
The College of Liberal Arts and Sciences
Department of Mathematics
|
MWF 1:30–2:20pm in 210 MLH
|
|
|
|
|
Office location and hours:
|
325B MLH at MW 10:30–11:30am & Tu 10–11am
|
|
|
|
|
david-e-stewart@uiowa.edu
|
|
|
|
You can see me outside the office hours listed above provided it is mutually convenient.
Some of the policies relating to this course (such as the drop deadline) are governed by its administrative home, the College of Liberal Arts and Sciences, 120 Schaeffer Hall.
Description of Course:
This course covers the theory and practice of numerical optimization: numerically estimating the maximum or minimum of a function, possibly subject to a number of either equality or inequality constraints. On the theoretical side we will talk about necessary and sufficient condition for local minima and maxima with or without constraints, convex sets and functions, and the analysis of algorithms. On the practical side we will consider a wide range of algorithms and their implementation, including line search and trust region techniques.
Detailed list of topics
- Unconstrained optimization theory
- 1st order necessary conditions (gradients)
- 2nd order necessary and sufficient conditions (gradients and Hessian matrices)
- positive definite and semi-definite matrices
- Convex functions and convex sets
- Equivalent definitions
- Properties (scaling, adding, maxima)
- Local minima are global minima
- Algorithms for unconstrained optimization
- Steepest descent
- Newton and quasi-Newton methods
- line search and trust region “globablization” techniques
- Conjugate gradient methods
- Constrained optimization theory (Kuhn–Tucker conditions)
- 1st order necessary conditions
- Constraint qualifications
- 2nd order necessary and sufficient conditions
- Special problems:
- Linear programming
- Quadratic programming
- Algorithms for constrained optimization
If there is time I will start looking at dynamic optimization problems and optimal control, or optimization problems related to “big data” tasks, depending on student interest.
Objectives and Goals of the Course:
The students will be introduced to the derivation, implementation, and analysis of numerical methods in the areas of numerical solution of ordinary differential equations and matrix computations, especially, solution of linear systems, solution of linear least-squares problems, and computation of eigenvalues and eigenvectors. Students are expected to become proficient in
all aspects covered. This involves both theory (mathematial analysis) and practice (programming and implementation). While learning a particular programming language is not necessary, examples and demonstrations will be given in Matlab
and students are expected to be able to read and understand code in this language.
There will be a more-or-less open-ended project for all students to complete which will involve work in all three aspects (derivation, implementation, and analysis) of numerical methods.
Texts:
Required text:
Numerical Optimization, 2nd Edition by S. Wright and J. Nocedal, which is published by Springer.
Grading System and the Use of +/–:
A +/– grading system will be used for this course. As noted on the CLAS teaching policy page, a score of A+ is a rare grade, assigned only in the case of exceptional work.
Assessment: Assignments and Percentage of Final Grade:
There will be an in-class exam and a final exam (20% each), homework assignments (40% total) which will include some programming, and a programming assignment (20%). There may also be ungraded homework assigned during the course. Familiarity with MATLAB will be useful during the course.
Please remember that final exams may only be given during finals week according to CLAS policy. Likewise, no major exams may be given the week before finals week.
A Word about the Date and Time of the Final Exam:
The date and time of every final examination is announced by the Registrar generally by the tenth day of classes. No exams of any kind are allowed during the last week of classes. All students should plan on being at the UI through the final examination period. Once the Registrar has announced the date, time, and location of each final exam, the complete schedule will be published on the Registrar’s web site and will be shared with instructors and students. It is the student's responsibility to know the date, time, and place of the final exam.
Course Policies:
Attendance and participation:
Students are expected to attend all classes unless there is an important and pressing reason otherwise. Attending students are expected to participate in classroom discussion and be prepared to give relevant comments on topics discussed in class in a civil and respectful manner.
Timely completion of assignments:
All assignments (homework and project) are expected to be completed and handed in on time. If you have questions about the homework or project, you should contact the TA or instructor. Unless prior arrangements have been made, late homework is subject to a 20% reduction in the grade.
Student Collaboration:
Students may discuss the homework and project. However, all work handed in must be the work of the student submitting the work and in their own words. All duplicated work will be considered plagiarized and graded as such. Any outside source of information must be cited in a way that any third person can obtain the cited work and check its veracity.
All exams are purely individual and there can be no discussion between students about the material in an exam during that exam whether in class or take-home.
Resources for Students:
Students may find the Writing Center and the Speaking Center very useful for this course:
Calendar of Course Assignments and Exams:
Homework will be scheduled regularly, approximately every two weeks except near the end of semester. The programming project will take place during the last four or five weeks of semester. This will involve a significant amount of programming and interpretation of the results. Students should be prepared to write a substantial report describing what they did for the project and their results and conclusions.
The midterm exam will be held on Friday, March 4th (Friday of week 7).
Notes
- Course plan: The course plan may be modified during the semester. Such modifications will be announced in advance during class periods; the student has responsibility for keeping up with such changes. You should also make a habit of reviewing the ICON web page for this course, which is accessible via: ICON http://icon.uiowa.edu/ This page will have homework details and other information posted to it as the class progresses.
- Administrative Home
The Department of Mathematics in The College of Liberal Arts and Sciences is the administrative home of this course and governs matters such as the add/drop deadlines, the second-grade-only option, and other related issues. Different colleges may have different policies. Questions may be addressed to 120 Schaeffer Hall, or see the CLAS Academic Policies Handbook at
http://clas.uiowa.edu/students/handbook.
- Electronic Communication
University policy specifies that students are responsible for all official correspondences sent to their University of Iowa e-mail address (@uiowa.edu). Faculty and students should use this account for correspondences (
Operations Manual, III.15.2, k.11).
- Accommodations for Disabilities
A student seeking academic accommodations should first register with Student Disability Services and then meet with the course instructor privately in the instructor's office to make particular arrangements. See
https://sds.studentlife.uiowa.edu/ for more information.
- Academic Honesty
All CLAS students or students taking classes offered by CLAS have, in essence, agreed to the College's
Code of Academic Honesty: "I pledge to do my own academic work and to excel to the best of my abilities, upholding the
IOWA Challenge. I promise not to lie about my academic work, to cheat, or to steal the words or ideas of others; nor will I help fellow students to violate the Code of Academic Honesty." Any student committing academic misconduct is reported to the College and placed on disciplinary probation or may be suspended or expelled (
CLAS Academic Policies Handbook).
- CLAS Final Examination Policies
The final examination schedule for each class is announced by the Registrar generally by the fifth week of classes. Final exams are offered only during the official final examination period. No exams of any kind are allowed during the last week of classes. All students should plan on being at the UI through the final examination period. Once the Registrar has announced the date, time, and location of each final exam, the complete schedule will be published on the Registrar's web site and will be shared with instructors and students. It is the student's responsibility to know the date, time, and place of a final exam.
- Making a Suggestion or a Complaint
Students with a suggestion or complaint should first visit with the instructor (and the course supervisor), and then with the departmental DEO (Dan Anderson, ph: 335-0714). Complaints must be made within six months of the incident (
CLAS Academic Policies Handbook).
- Understanding Sexual Harassment
Sexual harassment subverts the mission of the University and threatens the well-being of students, faculty, and staff. All members of the UI community have a responsibility to uphold this mission and to contribute to a safe environment that enhances learning. Incidents of sexual harassment should be reported immediately. See the UI
Comprehensive Guide on Sexual Harassment for assistance, definitions, and the full University policy.
- Reacting Safely to Severe Weather
In severe weather, class members should seek appropriate shelter immediately, leaving the classroom if necessary. The class will continue if possible when the event is over. For more information on Hawk Alert and the siren warning system, visit the
Department of Public Safety website.