Theory

Theory 1 - Binary testing, MAP and ML

Binary hypothesis test

Ingredients of a binary hypothesis test:

• and — Complementary hypotheses
  • Maybe also know the prior probabilities and
  • Goal: determine which case we are in, or
• and — Complementary events of the Decision Rule
  • Directionality: given , is likely; given , is likely
  • Decision Rule: outcome , accept ; outcome , accept
  • Usually: written in terms of decision statistic using a design
  • We cover three designs:
    • MAP and ML (minimize ‘error probability’)
    • MC (minimizes ‘error cost’)
  • Designs use and (or , ) to construct and

MAP design

Suppose we know:

• and
  • Both prior probabilities
• and (or and )
  • Both conditional distributions

The maximum a posteriori probability (MAP) design for a decision statistic :

Discrete case:

Continuous case:

And .

The MAP design minimizes the total probability of error.

ML design

Suppose we don’t know the priors, we know only:

• and (or and )
  • Both conditional distributions

The maximum likelihood (ML) design for :

ML is a simplified version of MAP. (Set and to .)

---

The probability of a false alarm, a Type I error, is called . The probability of a miss, a Type II error, is called .

Total probability of error:

Wrong meanings of

Suppose sets off a smoke alarm, and is ‘no fire’ and is ‘yes fire’.

Then is the odds that we get an alarm assuming there is no fire.

This is not the odds of experiencing a false alarm (no context). That would be .

This is not the odds of a given alarm being a false one. That would be .

Theory 2 - MAP criterion proof

Explanation of MAP criterion - discrete case

First, we show that the MAP design selects for all those which render more likely than . This will be used in the next step to show that MAP minimizes probability of error.

Observe this calculation:

Recall the MAP criterion:

Divide both sides by and apply the above Calculation in reverse:

This is what we sought to prove.

---

Next, we verify that the MAP design minimizes the total probability of error.

The total probability of error is:

Expand this with summation notation (assuming the discrete case):

Now, how do we choose the set (and thus ) in such a way that this sum is minimized?

Since all terms are positive, and any may be placed in or in freely and independently of all other choices, the total sum is minimized when we minimize the impact of placing each .

So, for each , we place it in if:

That is equivalent to the MAP criterion.

Theory 3 - MC design

• Write for cost of false alarm, i.e. cost when is true but decided .
  • Probability of incurring cost is .
• Write for cost of miss, i.e. cost when is true but decided .
  • Probability of incurring cost is .

Expected value of cost incurred

MC design

Suppose we know:

• Both prior probabilities and
• Both conditional distributions and (or and )

The minimum cost (MC) design for a decision statistic :

Discrete case:

Continuous case:

Then .

The MC design minimizes the expected value of the cost of error.

MC minimizes expected cost

Inside the argument that MAP minimizes total probability of error, we have this summation:

The expected value of the cost has a similar summation:

Following the same reasoning, we see that the cost is minimized if each is placed into precisely when the MC design condition is satisfied, and otherwise it is placed into .