ECEG335 Detection and Estimation Theory

(reload this page each time you visit)

Table of Contents
Suggested Reading Material
Meeting Times, Place
Instructor
Teaching Assistant
Course Description
Course Administration and Workload
Homework Assignments/Due Dates
Handouts

Suggested Reading Material
Elements of Signal Detection and Estimation, Helstrom, PTR-PH, ISBN: 0-13-808940-X

An Introduction to Signal Detection and Estimation, Poor, Springer-Verlag, ISBN: 0-387-94173-8

Detection and Estimation, Kazakos&Kazakos, Computer Science Press, ISBN: 0-7167-8181-6

Statistical Inference, Silvey, Chapman&Hall, ISBN: 0-412-13820-4

Probability, Random Variables, and Stochastic Processes, Papoulis&Pillai, McGraw-Hill, ISBN: 0-07-36611-6

Testing Statistical Hypotheses, Lehman, Wiley-Interscience, ISBN: 0-471-84083-1

Theory of Point Estimation, Lehman, Wiley-Interscience, ISBN: 0-471-05849-1

-> Table of Contents

Course Credit: 4QH
Live Class Times: TTh 1:30-310 Eastern
Class Location: 408 El
Textbook: Statistical Signal Processing- Detection, Estimation, and Time-Series Analysis, Louis Scharf, Addison Wesley, ISBN: 0-201-19038-9
-> Table of Contents

Instructor: Professor David Brady
Office Hours: Wed. 1:35-3:25pm Eastern
Instructor's Office: 416 Dana
Email: brady@ece.neu.edu
Phone: 617.373.5400
-> Table of Contents

Teaching Assistant: TBD
Office Hours: TBD
TA's Office: TBD
Email: TBD -> Table of Contents

Course Topics:

Estimation concepts include: review of vector space and stochastic concepts,
sufficiency, unbiased estimation, Cramer-Rao bound, Rao-Blackwell theorem,
Pitman efficiency, maximum likelihood estimation, Bayesian estimation, minimum
mean squared error estimation, least squared estimation, Gauss-Markov theorem.

Detection concepts include: simple and composite hypotheses, Neyman-Pearson
tests, uniformly most powerful tests, invariant tests, CFAR detection,
Bayesian detection, minimax detection, nonparametric testing, sequential
testing, quickest detection.
-> Table of Contents

Course Administration and Student Workload

Weekly Homework Policy. Students are expected to complete each homework assignment independently of other students or solution sets, in a synchronous manner with the lectures. Homeworks should be handwritten (not typeset). Calculations supporting answers should be clearly and concisely presented. Late or copied homeworks will receive no credit.
Midterm and Final Examination Policy. All examinations are in-class (or monitored), open book and open notes. All devices with communication capabilities, including cell phones, Blackberries, and pagers, must be turned off during the examination. Stand-alone calculators may be used during the examination (do not use your cell phone calculator). Laptops or palmtops are not permitted.
midterm exam: TBD
final exam: TBD
Grading Policy. 30% Homeworks 40% Midterm, 30% Final Examination.

-> Table of Contents

Homework Assignments
Assignment 1
        Due (in-class students): Th 16 Sep'04 at 3:10pm
        Read Ch 2, Scharf, pp.1-50
        Do problems 2.2a, 2.2b, 2.6, 2.8, 2.10.a, 2.10.b, 2.17. Optional problems 2.25, 2.26 will not be graded.
   Assignment 2
        Due (in-class students): Th 23 Sep'04 at 3:10pm
        Read Ch 3, Scharf
        Do problems 3.11, 3.12, 3.13d-g, 3.14d-e, 3.15. In 3.13f-g, you need not find the distributions of the sufficient statistics.     In 3.15e find the characteristic function of the random variable. Optional problems 3.14f-g.
   Assignment 3
        Due (in-class students): Tu 12 Oct’04 at 3:10pm
        Read Ch 4, Scharf
        Do problems 4.1, 4.4, 4.5, 4.6.

   Assignment 4
        Due (in-class students): Th 21 Oct’04 at 3:10pm
        Read Ch 5, Scharf
        Do problems 4.9, 4.10, 4.20, 4.23, 4.24.

HOMEWORKS 5 and 6 (+ midterm): Author Identification Project

ECEG335 Credit

2 homework assignments, plus midterm.

Introduction

This project will demonstrate that it is relatively easy to distinguish between two authors' writing styles when very little information is provided. You will be provided with a word length record for each of two opinion articles from a newspaper. A word length record is the sequential listing of word lengths (in ASCII characters) of the corresponding article. The two articles represent the work of two different writers at the Globe, say author 0 and author 1. The word length record of a third article is presented as the observation. Your tasks are to form statistical hypothesis pair(s), testing some attribute(s) of the observation. A hypothesis pair will take on the form:

H_i : author i wrote the article, i=0,1.

All three articles are opinion articles which appeared in the Boston Globe within the last 4 years, and exactly 2 were written by the same author. Once the hypothesis pair is constructed, then all the detection rules discussed in class may be applied. Some notes on forming the hypothesis pair follow.

Forming Hypothesis Pairs

Let W_i denote the i^th word length in the observed article. Clearly, W_i is a positive integer. Since the articles came from a newspaper, each W_i can't be too large. Note that I have included 0's in the data. An isolated 0 denotes end of sentence, and two successive 0's denote end of paragraph. In this way, you can test for more than just word length, e.g., sentence length, paragraph length. Make sure that you do not include length-0 words in your calculations.

Some helpful hints:

	It is to your advantage to be inventive in this endeavor. Creativity of methodology will be equally weighted with the rigor of analysis in the grading of this project.
	Safe bets. {W_k} are i.i.d., with common probability mass function p_i(w) under H_i. However, see previous note.
	Do not assume that W_i is Gaussian-distributed. Prob[W_i less than 0] =0, unlike the Gaussian of similar variance. Similarly, do not use any distribution which is based more on convenience than on realism.
	You may assume successive word lengths are independent and identically distributed. This will enable easier calculations for distributions under each hypothesis (see histograms in MATLAB). However, some student solutions from previous years have demonstrated very powerful testing of the joint distribution of {W_i,W_i₊₁}. Be careful about estimating joint distributions in this case. Are you forming statistical models which test for dependence, yet form distributions using an implicit assumption of independence?
	Consider sequential testing (see lecture notes).
	Optional. Read ahead, and consider nonparametric testing. Nonparametric tests are helpful when the distributions are poorly known under each hypothesis.

Project Proposal Format

One page, typeset. Include team members' names. For each of 3 detection strategies, concisely describe what you are testing, and what assumptions you will make about the data. Please put a lot of your effort in this phase of the work. Try out your ideas on the data (below). Proposal Grade: 15% of overall Project Grade.

Project Report Guidelines

Maximum of 2 sides of 8.5x11" paper per detection rule. Include:

	Clear statement of hypothesis pair.
	Present method and result of statistical model under each hypothesis.
	Simplfy detection rule as much as possible.
	Present analysis of either: Bayesian risk, min-max risk, probability of error, or power vs. false alarm level.
	Present the results of your detector on the data in the variable. (Below.)
	Final Report Grade: 85% of overall Project Grade.
	Page Limits. Reports exceding 6 total pages will lose 10% of Final Report Grade per excess page.

Project Timeline

	(Thu, Oct. 28.) Submit your team's proposal for 3 detection strategies.
	(Tue., Nov. 2.) Receive graded proposals, and begin work. About proposal format.
	(Tue, Nov 9.) Submit final report. Breathe. About final report.