Eroxl's Notes
Hypothesis Testing

Hypothesis testing is a strategy of inference for determining whether a statement about the value of a population parameter should or should not be rejected. It involves stating a null hypothesis (), which assumes no effect or difference, and an alternative hypothesis (), which suggests there is an effect or difference.

The null and alternative hypothesis typically take on one of the three following forms for a parameter and a hypothesized value

  • Lower-tailed:  vs
  • Upper-tailed:  vs
  • Two-tailed:   vs

The hypotheses should always be formulated before viewing or analyzing the data.

Test Procedures

A test procedure consists of several key components to evaluate the hypotheses.

Properties

-value

The -value is a measure of how "unusual" the data would be if was true. Specifically it is defined as the probability of observing a result as extreme or more extreme than the one obtained, assuming the null hypothesis is true. A small -value provides evidence against , suggesting the observed data is unlikely to occur under the null hypothesis.

Among the 3 forms of null and alternative hypothesizes the -value is calculated as follows:

  • Lower-tailed:  
  • Upper-tailed: 
  • Two-tailed:  

Significance Level ()

The significance level, denoted as , is the predetermined threshold for the -value that determines whether to reject the null hypothesis. It represents the maximum probability of committing a type I error. Common choices for are 0.05, 0.01, or 0.10. If the -value , the result is considered statistically significant, and is rejected.

Errors

Type I Error

Type I error is a type of error in hypothesis testing in which the null hypothesis is incorrectly rejected even though it is correct. In terms of the courtroom example, a type I error corresponds to convicting an innocent defendant.

Where is the probability of rejecting when it's actually true and is the event of committing a type I error.

Type II Error

Type II error is a type of error in hypothesis testing in which the null hypothesis is incorrectly accepted even though it is incorrect. In terms of the courtroom example, a type II error corresponds to acquitting a criminal.

Where is the probability of rejecting when it's actually true and is the event of committing a type II error.

Table of Error Types

is true is false
Not Rejected Correct inference (true negative) Type II error (false negative)
Rejected Type I error (false positive) Correct inference (true positive)

Test Statistic

The test statistic is a function of the data whose sampling distribution under the null hypothesis is known. It measures the discrepancy between the observed data and what would be expected if were true.

Z-Statistic

A Z-statistic is a test statistic for the population mean used when the population standard deviation is known or the sample size is large.

Where:

t-Statistic

A t-statistic is a test statistic for the population mean used when the population standard deviation is unknown and the sample size is small.

Where:

  • - Sample mean. The average value of your observations.
  • ​ - Hypothesized population mean under the null hypothesis ​.
  • - Sample standard deviation. An estimate of the population standard deviation when it is unknown.
  • - Sample size (number of observations).

Chi-Squared Statistic

A chi-squared statistic is a test statistic used to test hypotheses about the population variance.

Where: