ECE 6710 Optimal Control Systems

Fall 2020
version 9 December 2020


Dr. Damon A. Miller, Associate Professor of Electrical and Computer Engineering, Western Michigan University, College of Engineering and Applied Sciences, Floyd Hall, Room A-240, 269.276.3158, 269.276.3151 (fax),,

Course Format Summary
Lectures will be conducted synchronous online at the scheduled class meeting time.

Office Hours
Dr. Miller is available for online office hours W 4:00PM-5:00PM at Note that you may be online with other students; discussions related to confidential issues are by appointment as requested by email to

Catalog Description
ECE 6710 Optimal Control Systems (3-0), 3 hrs.  Optimal control dynamic programming, Portryagin’s principle, linear optimal regulator, system identification. Stochastic and adaptive control. Prerequisite: ECE 6700 Modern Control Theory.



Dr. Miller thanks Dr. Raghe Gejji for his support in preparing course materials. He also thanks the Educational Technology Department for contributions to this syllabus. Adapted/adopted in part from syllabi by J. Gesink and J. Kelemen.


Copyright Information

Materials prepared by Dr. Miller are © 2020 Damon A. Miller. Other copyrights apply to materials such as text and images from books, datasheets, etc. Consult source documents for copyright information.
Due to the unusual circumstances, I am providing my handwritten lecture notes for your personal use in ECE 6710 only. These must never be distributed or posted in any way. Since the notes are meant only as a supplement to the text, they may contain verbatim material from the text without quotes. The notes must not be considered as a primary source and never referenced. The text is the primary source, unless otherwise noted.


Textbook and Materials



1.      You must have access to a computer less than 5-years old with a webcam and microphone and high-speed internet access. This is required for online lectures and Dr. Miller’s office hours.

2.      Text: F. L. Lewis, D. L. Vrable, V. L. Symore, Optimal Control, John Wiley & Sons, Inc, 3rd edition, 2012.

3.      The MathWorks, MATLAB®, any reasonably recent version will suffice.  The CAE center provides access to this software; however, students are strongly encouraged to have access on their personal computer*.

*You may use a different programming language at your own risk if they include access to comparable optimization routines.

References (also see course schedule):


1.      T. W. Colthurst (founding editor) et al., “The Hessian,” World Web Math, available here. 

2.      M. J. Maron, Numerical Analysis: A Practical Approach, Macmillan, New York, 1982.

3.      J. Wilde et al., Unconstrained Optimization, available here.

4.      J. Belk, Some Linear Algebra, available here.

5.      A. Ali Ahmadi, review of linear algebra and multivariable calculus, available here.

6.      D. Khoshnevisan, Some Linear Algebra, available here.

7.      A. Christian, Definiteness of Quadratic Forms, available here.

8.      S. Webb, T. Croft, L. Mustoe, J. Ward, z-Transforms and Difference Equations, part of Engineering Mathematics Open Learning Project, available here.

9.      G. L. Plett, ECE4520/5520: Multivariable Control Systems I Lecture Notes: CH 6, available here.

10.   M. de Oliveria, Solution to Linear Time-Invariant Systems, available here.

11.   Weisstein, Eric W. "Leibniz Integral Rule." From MathWorld--A Wolfram Web Resource.

12.   P. Haile, Differentiating an Integral, available at

13.   G. B. Thomas, Jr. and R. L. Finney, Calculus and Analytical Geometry, 5th ed., Addison-Wesley, 1982.

14.   Wolfram Language & System Documentation Center, Numerical Solution of Boundary Value Problems (BVP), available at

15.   L. F. Shampine, J. Kierzenka, and M. W. Reichelt, Solving Boundary Value Problems for Ordinary Di erential Equations in MATLAB with bvp4c, available at

16.   K. B. Petersen and M. S. Pederson, The Matrix Cookbook, available at


Course Policies

Academic Honesty


Students are responsible for making themselves aware of and understanding the University policies and procedures that pertain to Academic Honesty. These policies include cheating, fabrication, falsification and forgery, multiple submission, plagiarism, complicity and computer misuse. The academic policies addressing Student Rights and Responsibilities can be found in the Undergraduate Catalog at and the Graduate Catalog at If there is reason to believe you have been involved in academic dishonesty, you will be referred to the Office of Student Conduct. You will be given the opportunity to review the charge(s) and if you believe you are not responsible, you will have the opportunity for a hearing. You should consult with your instructor if you are uncertain about an issue of academic honesty prior to the submission of an assignment or test.

Students and instructors are responsible for making themselves aware of and abiding by the “Western Michigan University Sexual and Gender-Based Harassment and Violence, Intimate Partner Violence, and Stalking Policy and Procedures” related to prohibited sexual misconduct under Title IX, the Clery Act and the Violence Against Women Act (VAWA) and Campus Safe. Under this policy, responsible employees (including instructors) are required to report claims of sexual misconduct to the Title IX Coordinator or designee (located in the Office of Institutional Equity). Responsible employees are not confidential resources. For a complete list of resources and more information about the policy see


In addition, students are encouraged to access the Code of Conduct, as well as resources and general academic policies on such issues as diversity, religious observance, and student disabilities:

·        Office of Student Conduct

·        Division of Student Affairs

·        Registrar’s Office

·        Disability Services for Students

— section provided by the WMU Faculty Senate with minor link reformatting



For an in-depth exploration of plagiarism, see

COVID-19 Statement

Due to the current COVID-19 Pandemic, and consistent with the State of Michigan* requirements and the WMU Safe Return plan (, safety requirements are in place to minimize exposure to the Western Michigan University community. These guidelines apply to all in-person or hybrid classes held either inside or outside a WMU building.


Facial coverings (masks), over both the nose and mouth, are required for all students while in class, no matter the size of the space. This includes outdoor class settings where social distancing is not possible (i.e., at least six feet of space between individuals). Following this recommendation can minimize the transmission of the virus, which is spread between people interacting in close proximity through speaking, coughing, or sneezing. During specified classes in which facial coverings (masks) would prevent required class elements, students may remove facial coverings (masks) with instructor permission, in accordance with the exceptions in the Facial Covering (mask) Policy** ("such as playing an instrument, acting, singing, etc.").


Facial coverings (masks) must remain in place throughout the class. Any student who removes the mandatory facial covering (mask) during class will be required to leave the classroom immediately.


Facial coverings (masks) are not a substitute for social distancing. Students shall observe current social distancing guidelines in all instructional spaces, both indoors and outdoors. Students should avoid congregating around instructional space entrances before and after class sessions. Students should exit the instructional space immediately after the end of class to help ensure social distancing and to allow for those attending the next scheduled class session to enter.


Students who are unable to wear a facial covering (mask) for medical reasons must contact Disability Services for Students ( before they attend class.


These guidelines are in place to ensure the safety of all students, faculty, and staff during the pandemic. Noncompliance is a violation of the class requirements and the Student Code of Honor (https:/


*For current State of Michigan Executive orders, see:,9309,7-387-90499_90705---,00.html


**For the WMU Facial Covering (Mask) Policy, see:

— statement provided by the WMU Faculty Senate

If you have a documented disability and verification from the Disability Services for Students (DSS), and wish to discuss academic accommodations, please contact your instructor as soon as possible. It is the student’s responsibility to provide documentation of disability to DSS and meet with a DSS counselor to request special accommodation before classes start.

Grading Basis

Projects (100%) will be assigned on a regular basis.  LATE PROJECTS WILL NOT BE ACCEPTED AND ARE DUE AS INDICATED VIA ELEARNING DROPBOX. All projects are to be completed individually.  Projects may include/consist of a series of homework style problems. Use the prescribed homework format for those problems. Be sure to follow the guidelines for computer assignments.


OUTSTANDING WORK might earn extra credit. The first student to report an error in any material prepared by the instructor(s) will earn extra credit. The course grading scale is:


Scale: 0-59 E | 60-64 D | 65-69 DC | 70-74 C | 75-79 CB | 80-84 B | 85-89 BA | 90-100 A |
Numeric scores are rounded to the nearest integer.

Midterm grades are not assigned.

Grade Appeals
If you have a question regarding a graded assignment, contact Dr. Miller within TWO business days of receiving the grade for the assignment in question.

Late Assignments will not be accepted without a documented excuse. If an emergency prevents you from submitting an assignment on-time, contact your instructor PRIOR to the assignment due date or as soon as you can, via email.  Failure to adhere to this policy will result in zero credit for the assignment.

HOMEWORK contributes to the project grade category. Each homework problem must be worked on separate page(s).  LATE HOMEWORK will not be accepted, except under extraordinary circumstances. Homework is to be completed individually.

Homework should normally be done on 8 1/2'' by 11'' sheets and scanned for submission. “Engineer's Pad” sheets are preferred.  Solutions must be done in a neat, structured, logical, and orderly manner with frequent brief notations enabling the grader to readily verify the author's source of information, steps taken, sources of formulas, equations, and methods used. USE THE PARTIAL CHECK LIST FOR SUBMITTED HOMEWORK BELOW.  Papers failing to meet these guidelines may not be graded and may be returned, with or without an opportunity for resubmission with a penalty.


  1. Each problem must include: (a) author's name, (b) name/title of the assignment, and (c) date of completion.
  2. Use only one side of the paper and include a brief and concise statement of the problem prior to its solution. Begin each problem on a new page.
  3. Number the pages and DOUBLE SPACE the text.
  4. Staple each problem in the upper left corner as needed.
  5. Entitle graphs, label and include axes, include key symbols for multiple curve graphs, and give brief notes of explanation where appropriate.
  6. Briefly but clearly annotate your document in a way which will provide the document reader with information such as
    1. which part of the assignment is this?
    2. what is being done and why?
    3. how was it done and what are the results?
    4. how was this equation obtained and how was it used?
    5. sample calculations and definitions of symbols/parameters where appropriate; and

COMPUTER ASSIGNMENTS must be implemented via MATLAB®.  Computer assignments must include

1.      a problem statement;

2.      description of techniques utilized including pseudo-code (as in “listings” in the text);

3.      results;

4.      discussion of results; and

5.      computer code listing(s) attached as an appendix. Computer code must include explanatory comments. Some of those comments should relate computer code to the pseudo-code of item 2 above.  Use modular programming.

Project Submission

Submit your files as a compressed folder (.zip format) attachment to your instructor’s Elearning Dropbox by the indicated due date. The folder must contain ONLY the report in pdf format. No other files will be considered. The folder must be named in this format: “LastNameFirstName_Project#”; for example, “DoeJane_Project1” is Jane Doe’s Project 1 submission.


Submissions deviating from these instructions will not be accepted.


Course Schedule
The schedule will be frequently updated as the semester progresses.









1.      Static Optimization (CH 1)


Read CH 1




1.      Static Optimization (CH 1)


Read CH 2.1


Project 1 (CH 1) DUE by 9/16 5PM
Submit to ELearning Dropbox


1.      TEXT PROBLEM 1-1.1
Also: use MATLAB® to plot L; the contours of L; and gradient Lu

2.      TEXT PROBLEM 1.1-2
Also: use MATLAB® to plot L; the contours of L; and gradient Lu

Also: use MATLAB® to numerically find the minimum using: fminsearch(); compare results.

3.      TEXT PROBLEM 1.1-3
Also: use MATLAB® to plot f; the contours of f; and gradient of f

4.      TEXT PROBLEM 1.2-1

5.      TEXT PROBLEM 1.2-5.
Use MATLAB® to plot contours and constraint for both cases.

6.      TEXT PROBLEM 1.2-7

7.      EXAMPLE 1.2-2. Select values for Q, R, B, c for m=n=2. Find the optimal control u by hand. Then use the algorithm of 1.3 Numerical Solution Methods to find u. Also find u using MATLAB fmincon(). Compare the three results.

8.      Apply what you learned in CH 1 to minimization of the Rosenbrock function [ref]
f(x,y) = (1-x)2+100(y-x2)2
with constraint x-y=0.




2.      Solution of the General Discrete-Time Optimization Problem (CH 2.1)

Read CH 2.2

Project 2 (CH 2.1) DUE by 9/23 5PM

Submit to ELearning Dropbox


1.      Duplicate the results of the computer simulations on page 30 of the text (fixed and free final state).

2.      TEXT PROBLEM 2.1-1

3.      Complete TEXT PROBLEM 2.1-2

Then set Q=SN=0. Find the optimal control for a fixed and free final state. Verify your results via simulation. Investigate the effect of varying r in both cases. Show state trajectories on a contour plot of J.









3.      Discrete-Time Linear-Quadratic Regulator (CH 2.2)

Read CH 2.3 and CH 2.4


Project 3 (CH 2.2) DUE BY10/7 by 5PM. Submit to ELearning Dropbox


1.      See if you can show that the coefficient matrix of (2.2-9) is Hamiltonian. I could not L.

2.      TEXT PROBLEM 2.2-4.
Plot the state, control input, feedback gain, and Jk

3.      TEXT PROBLEM 2.2-5.
Verify results with simulation.

4.      TEXT PROBLEM 2.2-8(a).
Verify results with simulation.

5.      TEXT PROBLEM 2.2-9




4.      Digital Control of Continuous Time Systems (CH 2.3)

5.      Steady-State Closed-Loop Control and Suboptimal Feedback (CH 2.4)

Read CH 3.1-3.3


Project 4 (CH 2.3 and CH 2.4)
DUE BY 11/4 by 5PM.
Submit to ELearning Dropbox


1.      Duplicate the results of Example 2.3-1

2.      TEXT PROBLEM 2.3-2
You do not have to use lsim.m.

3.      Duplicate the results of Example 2.3-2 and 2.4-1. Verify that the suboptimal cost Jk is greater than the optimal cost J*k for all k.

4.      Duplicate results of Example 2.4-3 via simulation.

5.      Duplicate results of Example 2.4-4.

6.      Duplicate results of Example 2.4-6.

7.      TEXT PROBLEM 2.4-2




Steady-State Closed-Loop Control and Suboptimal Feedback (CH 2.4)





6.      The Calculus of Variations (CH 3.1)

7.      Solution of the General Continuous-Time Optimization Problem (CH 3.2)

Project 5 (CH 3.2)
DUE 11/4 by 5PM
Submit to ELearning Dropbox


1.      TEXT PROBLEM 3.2-1

2.      TEXT PROBLEM 3.2-2; simulate your solution and plot lambda*, u*, x*, and J as a function of time.

3.      EXTRA CREDIT: Improve Dr. Miller’s derivation of Table 3.2-1

4.      EXTRA CREDIT: Derive 3.2-11




8.      Q/A Session





Solution of the General Continuous-Time Optimization Problem (CH 3.2)

9.      Solution of Two-point Boundary-value Problems (CH 3.2 subsection)

10.   Continuous-Time Linear Quadratic Regulator (CH 3.3)

Read CH 4.1

Project 6 (CH 3.2)
DUE 11/11 by 5PM
Submit to ELearning Dropbox


1.      TEXT PROBLEM 3.2-8
See if you can use the unit solution method. At a minimum, solve using any mathematics computation package, e.g. MATLAB®, Mathematica®, etc. Assume T=2 and x1(T)=1. Note that (1b) is x2 DOT.

2.      TEXT PROBLEM 3.3-1

3.      TEXT PROBLEM 3.3-3




11.   The Tracking Problem (CH 4.1)

12.   Regulator with Function of Final State Fixed (CH 4.2)

Read 5.2

Project 7 (CH 4.1 and 4.2)
DUE 12/9 by 5PM

1.      TEXT PROBLEM 4.1-3.
Note u is the unit step.

2.      TEXT PROBLEM 4.2-2(a)
Plot the states and control input.




13.   Constrained Input Problems w/ Bang-Bang Control
 (CH 5.2)

Read CH 6.1-6.2

Project 8 (CH 5.2)
DUE 12/9 by 5PM

1.      TEXT PROBLEM 5.2-2
Do not “derive a minimum-time feedback control law.”




Thanksgiving break





14.   Review Project 1






15.   Bellman’s Principle of Optimality (CH 6.1)

16.   Discrete-Time Systems (Ch 6.2)

Project 9 (CH 6)

1.      Example 6.1-2, pg. 263

2.      Example 6.1-3, pg. 263

3.      Example 6.1-4(a), pg. 263

4.      Text Problem 6.2-1

DUE 12/16 BY 7PM









Final Exam: 5-7PM
(Project 9 due)


Return to Dr. Miller’s homepage