ECE 5730 Foundations of Neural Networks

Fall 2018
version 20 September 2018


Dr. Damon A. Miller, Associate Professor of Electrical and Computer Engineering, Western Michigan University, College of Engineering and Applied Sciences, Parkview Campus, Room A-240, 269.276.3158, 269.276.3151 (fax),,

Office Hours

Guaranteed office hours are posted on Dr. Miller’s door and at Please respect my office hours.  Other times are available by appointment.

WMU Catalog Description

ECE 5730 Foundations of Neural Networks, 3 hrs.  Biological and artificial neural networks from an electrical and computer engineering perspective. Neuron anatomy. Electrical signaling, learning, and memory in biological neural networks. Development of neural network circuit models. Artificial neural systems including multilayer feedforward neural networks, Hopfield networks, and associative memories. Electronic implementations and engineering applications of neural networks.


Prerequisite Abilities


You must be able to work independently on research projects and to write professional quality written reports describing your results. You must also be able to write computer programs using a language such as C, MATLAB®, Mathematica®, etc.


Fall 2018 Semester


Dr. Miller will tailor a course plan for each individual student based on the catalog description and individual research objectives.


Textbook and Materials

1.  Jacek M. Zurada, Artificial Neural Systems, PWS Publishing, Boston, 1992 (ISBN 0-314-93391-3).  Available from the author, instructions for securing a copy to be provided in class.

2.  W. Otto Friesen and J. A. Friesen, NeuroDynamix II:  Concepts of Neurophysiology Illustrated by Computer Simulations, Oxford University Press, 2010 (ISBN 978-0-19-537183-3).

3.  Scott Freeman, Biological Science, Prentice Hall, 2nd edition, 2005 (ISBN 0-13-140941-7):  chapters 6 (“Lipids, Membranes, and the First Cells”), 45 (“Electrical Signals in Animals”), and 46 (“Animal Sensory Systems and Movement”) only.  Any version of this text is acceptable provided these chapters are present.



1.  Simon Haykin, Neural Networks: A Comprehensive Foundation, Prentice Hall, 1st ed., 1994. On reserve in ECE Office.

Course Policies

Academic Honesty


“Students are responsible for making themselves aware of and understanding the University policies and procedures that pertain to Academic Honesty. These policies include cheating, fabrication, falsification and forgery, multiple submission, plagiarism, complicity and computer misuse. The academic policies addressing Student Rights and Responsibilities can be found in the Undergraduate Catalog at [] and the Graduate Catalog at []. If there is reason to believe you have been involved in academic dishonesty, you will be referred to the Office of Student Conduct. You will be given the opportunity to review the charge(s) and if you believe you are not responsible, you will have the opportunity for a hearing. You should consult with your instructor if you are uncertain about an issue of academic honesty prior to the submission of an assignment or test.

Students and instructors are responsible for making themselves aware of and abiding by the “Western Michigan University Sexual and Gender-Based Harassment and Violence, Intimate Partner Violence, and Stalking Policy and Procedures” related to prohibited sexual misconduct under Title IX, the Clery Act and the Violence Against Women Act (VAWA) and Campus Safe. Under this policy, responsible employees (including instructors) are required to report claims of sexual misconduct to the Title IX Coordinator or designee (located in the Office of Institutional Equity). Responsible employees are not confidential resources. For a complete list of resources and more information about the policy see

In addition, students are encouraged to access the Code of Conduct, as well as resources and general academic policies on such issues as diversity, religious observance, and student disabilities:

·        Office of Student Conduct

·        Division of Student Affairs

·        University Relations Office

·        Disability Services for Students

— provided by the WMU Faculty Senate Professional Concerns Committee


Plagiarism WILL NOT BE TOLERATED.  For an in-depth exploration of plagiarism, see

Grading Basis

1.      Projects (70%) will be assigned on a regular basis.  Some project results will be reported using the IEEE journal paper format; see for details. You may not use any sources other than those provided in class or in this syllabus when preparing your project report without prior approval from the course instructor.  You may be asked to demonstrate your project. LATE PROJECTS WILL NOT BE ACCEPTED AND ARE DUE AT THE BEGINNING OF CLASS. All projects are to be completed individually.

2.      Homework (30%)

Note: OUTSTANDING WORK might earn extra credit.


Scale: 0-59 E | 60-64 D | 65-69 DC | 70-74 C | 75-79 CB | 80-84 B | 85-89 BA | 90-100 A |
Numeric scores are rounded to the nearest integer.

Grade Appeals:  If you have a question regarding grading of any course materials, see Dr. Miller within FIVE business days of receiving the grade for the assignment in question.  If you disagree with the assessment of that assignment at that meeting, you must submit a written description of your concern to Dr. Miller via his ECE Department mailbox (not email) within five business days of that meeting.


HOMEWORK is assigned in class with a due date. Each problem must be worked on separate page(s).  LATE HOMEWORK will not be accepted, except under extraordinary circumstances. Homework is to be completed individually.


Homework should normally be done on 8 1/2'' by 11'' sheets. “Engineer's Pad” sheets are preferred.  Solutions must be done in a neat, structured, logical, and orderly manner with frequent brief notations enabling the grader to readily verify the author's source of information, steps taken, sources of formula, equations, and methods used. USE THE PARTIAL CHECK LIST FOR SUBMITTED HOMEWORK BELOW.  Papers failing to meet these guidelines may not be graded and may be returned, with or without an opportunity for resubmission with a penalty.




1.      Each problem must include: (a) author's name, (b) name/title of the assignment, and (c) date of completion.

2.      Use only one side of the paper and include a brief and concise statement of the problem prior to its solution. Begin each problem on a new page.

3.      Number the pages and DOUBLE SPACE the text.

4.      Staple each problem in the upper left corner as needed.

5.      Entitle graphs, label and include axes, include key symbols for multiple curve graphs, and give brief notes of explanation where appropriate.

6.      Briefly but clearly annotate your document in a way which will provide the document reader with information such as

a.      which part of the assignment is this?

b.      what is being done and why?

c.      how was it done and what are the results?

d.      how was this equation obtained and how was it used?

e.      sample calculations and definitions of symbols/parameters where appropriate; and



In case of conflict, information in this syllabus supersedes all other course documents.



Students are expected to attend all lectures and to be on time (homework is collected at the beginning of class). Electronic devices are to be turned off (unless there is a safety issue) during lecture unless arrangements have been made with the instructor.

Course Schedule
Will be updated as the semester progresses


Class #



[some topics may be verbatim from course references]





Course introduction

Secure a copy of the Zurada, Friesen and Friesen, and Freeman texts listed above.


Secure a copy of the Zurada text.



Read Zurada: Preface, CH 1 Artificial Neural System: Preliminaries,
CH 2 Fundamental Concepts and Models of Artificial Neural Systems,
A1 Vectors and Matrices,
A6 Analytic Geometry in Euclidian Space in Cartesian Coordinates




Introduction to Artificial Neural Systems
Biological Neurons and Their Artificial Models [Zurada 2.1]

Models of Artificial Neural Networks

[Zurada 2.2]

HW 1:  Zurada:  CH 2:  1, 4, 14.
Verify the results of Figure 2.15 using LTspice and MATLAB® (numerically solve the differential equations that describe the circuit)

(Use homework format)
HW #1 Example MATLAB® Code
DUE 9/21 in Dr. Miller’s department mailbox



Models of Artificial Neural Networks

[Zurada 2.2]


Learning and Adaptation

[Zurada 2.4]


Neural Network Learning Rules

[Zurada 2.5]

LE: 6730 Presentation (12:25PM)
ZR: 6730 Presentation






ZR: 6730 Presentation (cont’d)

Neural Network Learning Rules

[Zurada 2.5]


Classification Model, Features, and Feature Recognition
[Zurada 3.1]


Discriminant Functions

[Zurada 3.2]


Machine Learning and Minimum Distance Classification
[Zurada 3.3]

Read Zurada CH 3 Single Layer Perceptron Classifiers
CH 4 Multilayer Feedforward Networks
A2 Quadratic Forms and Definite Matrices

A3 Time-Varying and Gradient Vectors, Jacobian, and Hessian Matrices



Nonparametric Training Concept
[Zurada 3.4]


Training and Classification Using the Discrete Perceptron…

[Zurada 3.5]


Single-Layer Continuous Perceptron Networks for Linearly Separable Classifications

[Zurada 3.6]


Multicategory Single-Layer Perceptron Networks

[Zurada 3.7]

HW 2:  Zurada:  CH 3:  3, 5, 6, 7, 8, 13 (use MATLAB® to plot the error surface in 3D and to prepare a contour plot as in Fig. P3.13 of [Zurada]).

(Use homework format)

DUE 9/25




GC: 6730 Presentation (11:30AM)

LE: 6730 Presentation


ZR: 6730 Presentation



Linearly Nonseparable Patter Classification
[Zurada 4.1]


Delta Learning Rule for Multiperceptron Layer

[Zurada 4.2]




Zurada CH 4 cont’d

Discuss Project 1

EBPT training example

Project 1:  Design of a Multilayer Feedforward Neural Network Classifier and Approximator
(use IEEE report format)
DUE 9/27

project files:



DUE 10/4








ZR: 6730 Presentation
LE: 6730 Presentation









GC: 6730 Presentation









Parts adapted/adopted from syllabi by J. Gesink and J. Kelemen.

© 2018 Damon A. Miller

Return to Dr. Miller’s homepage