ECE 6730 Advanced Neural Networks

Fall 2018
version 4 December 2018

Instructor

Dr. Damon A. Miller, Associate Professor of Electrical and Computer Engineering, Western Michigan University, College of Engineering and Applied Sciences, Parkview Campus, Room A-240, 269.276.3158, 269.276.3151 (fax), damon.miller@wmich.edu, www.homepages.wmich.edu/~miller/.

Office Hours

Guaranteed office hours are posted on Dr. Miller’s door and at http://homepages.wmich.edu/~miller/. Please respect my office hours.  Other times are available by appointment.

WMU Catalog Description

ECE 6730 Advanced Neural Networks, 3 hrs.  Advanced topics in biological and artificial neural networks from an electrical and computer engineering perspective. Modeling, simulation, and implementation of neural networks. Information theory and knowledge representation. Adaptation and learning. Review of current research.

 

Prerequisite Abilities

 

You must be able to work independently on research projects and to write professional quality written reports describing your results. You must also be able to write computer programs using a language such as C, MATLAB®, Mathematica®, etc.

 

Fall 2018 Semester

 

Dr. Miller will tailor a course plan for each individual student based on the catalog description and individual research objectives.

 

Textbook and Materials

1. GC: Practical Convolutional Neural Networks by M. Sewak, Md. Karim, and P. Pujar, Packt Publishing, 2018.

Course Policies

Academic Honesty

General:

“Students are responsible for making themselves aware of and understanding the University policies and procedures that pertain to Academic Honesty. These policies include cheating, fabrication, falsification and forgery, multiple submission, plagiarism, complicity and computer misuse. The academic policies addressing Student Rights and Responsibilities can be found in the Undergraduate Catalog at [http://catalog.wmich.edu/index.php] and the Graduate Catalog at [http://catalog.wmich.edu/index.php]. If there is reason to believe you have been involved in academic dishonesty, you will be referred to the Office of Student Conduct. You will be given the opportunity to review the charge(s) and if you believe you are not responsible, you will have the opportunity for a hearing. You should consult with your instructor if you are uncertain about an issue of academic honesty prior to the submission of an assignment or test.

Students and instructors are responsible for making themselves aware of and abiding by the “Western Michigan University Sexual and Gender-Based Harassment and Violence, Intimate Partner Violence, and Stalking Policy and Procedures” related to prohibited sexual misconduct under Title IX, the Clery Act and the Violence Against Women Act (VAWA) and Campus Safe. Under this policy, responsible employees (including instructors) are required to report claims of sexual misconduct to the Title IX Coordinator or designee (located in the Office of Institutional Equity). Responsible employees are not confidential resources. For a complete list of resources and more information about the policy see www.wmich.edu/sexualmisconduct.

In addition, students are encouraged to access the Code of Conduct, as well as resources and general academic policies on such issues as diversity, religious observance, and student disabilities:

·        Office of Student Conduct www.wmich.edu/conduct

·        Division of Student Affairs www.wmich.edu/students/diversity

·        University Relations Office http://www.wmich.edu/registrar/calendars/interfaith

·        Disability Services for Students www.wmich.edu/disabilityservices

— provided by the WMU Faculty Senate Professional Concerns Committee

Plagiarism:

Plagiarism WILL NOT BE TOLERATED.  For an in-depth exploration of plagiarism, see http://lib.usm.edu/plagiarism_tutorial.html

Grading Basis

1.      Projects (70%) will be assigned on a regular basis.  Some project results will be reported using the IEEE journal paper format; see http://ieeeauthorcenter.ieee.org/wp-content/uploads/Transactions-instructions-only.pdf for details. You may not use any sources other than those provided in class or in this syllabus when preparing your project report without prior approval from the course instructor.  You may be asked to demonstrate your project. LATE PROJECTS WILL NOT BE ACCEPTED AND ARE DUE AT THE BEGINNING OF CLASS. All projects are to be completed individually.

2.      Homework (30%)

Note: OUTSTANDING WORK might earn extra credit.

 

Scale: 0-59 E | 60-64 D | 65-69 DC | 70-74 C | 75-79 CB | 80-84 B | 85-89 BA | 90-100 A |
Numeric scores are rounded to the nearest integer.

Grade Appeals:  If you have a question regarding grading of any course materials, see Dr. Miller within FIVE business days of receiving the grade for the assignment in question.  If you disagree with the assessment of that assignment at that meeting, you must submit a written description of your concern to Dr. Miller via his ECE Department mailbox (not email) within five business days of that meeting.

 

HOMEWORK is assigned in class with a due date. Each problem must be worked on separate page(s).  LATE HOMEWORK will not be accepted, except under extraordinary circumstances. Homework is to be completed individually.

 

Homework should normally be done on 8 1/2'' by 11'' sheets. “Engineer's Pad” sheets are preferred.  Solutions must be done in a neat, structured, logical, and orderly manner with frequent brief notations enabling the grader to readily verify the author's source of information, steps taken, sources of formula, equations, and methods used. USE THE PARTIAL CHECK LIST FOR SUBMITTED HOMEWORK BELOW.  Papers failing to meet these guidelines may not be graded and may be returned, with or without an opportunity for resubmission with a penalty.

 

PARTIAL CHECK LIST FOR SUBMITTED HOMEWORK

 

1.      Each problem must include: (a) author's name, (b) name/title of the assignment, and (c) date of completion.

2.      Use only one side of the paper and include a brief and concise statement of the problem prior to its solution. Begin each problem on a new page.

3.      Number the pages and DOUBLE SPACE the text.

4.      Staple each problem in the upper left corner as needed.

5.      Entitle graphs, label and include axes, include key symbols for multiple curve graphs, and give brief notes of explanation where appropriate.

6.      Briefly but clearly annotate your document in a way which will provide the document reader with information such as

a.      which part of the assignment is this?

b.      what is being done and why?

c.      how was it done and what are the results?

d.      how was this equation obtained and how was it used?

e.      sample calculations and definitions of symbols/parameters where appropriate; and

f.       BOX AND LABEL ANSWERS.

 

In case of conflict, information in this syllabus supersedes all other course documents.

 

Other

Students are expected to attend all lectures and to be on time (homework is collected at the beginning of class). Electronic devices are to be turned off (unless there is a safety issue) during lecture unless arrangements have been made with the instructor.


 

Course Schedule
Will be updated as the semester progresses.

 

Class #

Date

Topic

[some topics may be verbatim from course references]

Assignment

WEEK 1

1                  

8/31

course introduction

GC: Consider the suitability of Practical Convolutional Neural Networks by M. Sewak, Md. Karim, and P. Pujari as your course text. Provide feedback to Dr. Miller via email by 9/4

 

LE: Perform a literature search on the topic of applications of multilayer feedforward neural networks to electric utility power load forecasting. The goal is to identify an approach for implementation. Present results in class 9/6.

 

ZR: Develop a concise verbal/mathematical problem description for your modeling problem. Consider suitability of the text Nonlinear System Identification by O. Nelles as your course text. Present description in class 9/6.

WEEK 2

 

9/4

ATTENDANCE OPTIONAL

 

2                  

9/6

LE: Presentation (12:20PM)
ZR: Presentation (12:30PM)

 

WEEK 3

3                  

9/11

ZR: Presentation
(cont’d)
(11:30AM)

GC: Use TensorFlow to implement the classifier and approximator project from ECE 5730. Present in class 9/18.

 

LE: Create training set data for electricity load forecasting. Train neural network. Present in class 9/18.

 

ZR: Use TensorFlow to implement an example neural network classification problem. Present in class 9/18.

4                  

9/13

ATTENDANCE OPTIONAL

 

 WEEK 4

5                  

9/18

GC Presentation (11:30AM)
LE Presentation
(11:40AM)
ZR Presentation
(11:50AM)

GC: Make improvements on use of TensorFlow to implement the classifier and approximator project from ECE 5730. Present in class 9/25.

 

LE: Expand training set data for electricity load forecasting and train neural network. Present in class 10/2.

 

ZR: Use TensorFlow to implement an example neural network curve fitting problem. Present in class 9/25

6                  

9/20

ATTENDANCE OPTIONAL

 

WEEK 5

7                  

9/25

ZR Presentation
(11:30AM)

GC Presentation
(11:40AM)

Review the perspective of Michael Jordan as described in “Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Other Huge Engineering Efforts” at http://spectrum.ieee.org/robotics/artificial-intelligence/machinelearning-maestro-michael-jordan-on-the-delusions-of-big-data-and-other-huge-engineering-efforts

 

Read the online article How a Pioneer of Machine Learning Became One of Its Sharpest Critics by Kevin Hartnett about the perspective of Judea Pearl.
Note: This link is provided to share this article only and does not imply endorsement of any views expressed on that webpage.

 

Read Deep Learning Reinvents the Hearing Aid by DeLiang Wang

 

GC:  Apply TensorFlow to an image classification problem. Present in class 10/2.

 

ZR: Provide overview of TensorFlow optimization techniques. Update on selection of dynamical system for neuron modeling. Present in class 10/9.

8                  

9/27

ATTENDANCE OPTIONAL

 

WEEK 6

9                  

10/2

LE: Presentation

(11:30AM)

ZR: Presentation

(11:40AM)

GC: Presentation

(11:50AM)

LE:  Investigate alternative minimization techniques. Consider adding noise during training to avoid local minima. Also explore training an ensemble of neural networks. Present in class 10/9.

 

GC: Provide update on work to use a convolutional neural network for a real-world classification problem. Present in class 10/16.

10               

10/4

ATTENDANCE OPTIONAL

 

WEEK 7

11               

10/9

ZR: 6730 Presentation

(11:30AM)

LE: 6730 Presentation

(11:40AM)

ZR: Train a network using data from simulation of dynamical system.
Present in class 10/23.

 

LE: Implement feedback on training and error characterization. Investigate using subsets of data for training.
Present in class 10/16.

12               

10/11

ATTENDANCE OPTIONAL

 

WEEK 8

13               

10/16

GC: Presentation

(11:30AM)

LE: Presentation

(11:40AM)

GC: Train a convolutional neural network for a simple classification problem. Present in class 10/30.

 

LE: Perform a cost-benefit analysis of using the electric load prediction neural network. Present in class 10/30.

14               

10/18

FALL BREAK

 

WEEK 9

15               

10/23

ZR: Presentation

(11:30AM)

 

ZR: Add derivative information to training set. Investigate applying SVD to training set. Present results in class 11/6.

 

10/25

NO CLASS

 

WEEK 10

16               

10/30

LE: Presentation
(11:30AM)
GC: Presentation
(11:40AM)

LE: Investigate circuits perform intracellular neuron stimulation and recording. Present in class 11/13.
GC: Develop classifier for handwritten digits. Present in class 11/13.

17               

11/1

ATTENDANCE OPTIONAL

 

WEEK 11

18               

11/6

ZR: Presentation

(11:30AM)

ZR:  Continue investigation of methods to reduce input space size and neural network size. Present results in class 11/20

19               

11/8

ATTENDANCE OPTIONAL

 

WEEK 12

20               

11/13

LE: Presentation
(11:30AM)
GC: 6730 Presentation

(11:50AM)

LE: Provide detailed schematics of circuit diagrams presented in class. Use IEEE database to find more recent references. Present in class 11/27.
GC: Implement paper current classifier. Present in class 11/27.

 

21               

11/15

ATTENDANCE OPTIONAL

 

WEEK 13

22               

11/20

ZR: Presentation

(11:30AM)

ZR:LE:GC: Prepare draft of final report for your project. DUE 12/4

 

11/22

 

NO CLASS: THANKSGIVING

WEEK 14

23               

11/27

LE: Presentation
(11:30AM)
GC: Presentation

(11:40AM)

 

24               

11/29

ATTENDANCE OPTIONAL

 

WEEK 15

25               

12/4

review final reports

ZR:LE:GC: Final draft report for your project due.

 

12/6

NO CLASS

ZR:LE:GC: Final report for your project DUE 12/12 at final exam. Present results during final exam period.

WEEK 16

 

12/12WED.

10:15AM-12:15PM

FINAL EXAM

(verify this day/time on your own)

ZR:LE:GC: Final report due. Present results.
ECE 5730 presentations

 

Credits

 

Parts adapted/adopted from syllabi by J. Gesink and J. Kelemen.

© 2018 Damon A. Miller

Return to Dr. Miller’s homepage