For the computer science notion of a “critical section”, sometimes called a “critical region”, see critical section. A statistical hypothesis, sometimes called confirmatory data analysis, is a hypothesis that is testable on the basis of observing a process that is modeled via a set of random variables. An alternative null hypothesis vs alternative hypothesis pdf for statistical hypothesis testing is to specify a set of statistical models, one for each candidate hypothesis, and then use model selection techniques to choose the most appropriate model. Confirmatory data analysis can be contrasted with exploratory data analysis, which may not have pre-specified hypotheses.

Statistical hypothesis testing is a key technique of both frequentist inference and Bayesian inference, although the two types of inference have notable differences. One naïve Bayesian approach to hypothesis testing is to base decisions on the posterior probability, but this fails when comparing point and continuous hypotheses. In the statistics literature, statistical hypothesis testing plays a fundamental role. There is an initial research hypothesis of which the truth is unknown. The first step is to state the relevant null and alternative hypotheses. This is important, as mis-stating the hypotheses will muddy the rest of the process. Decide which test is appropriate, and state the relevant test statistic T.

Derive the distribution of the test statistic under the null hypothesis from the assumptions. In standard cases this will be a well-known result. For example, the test statistic might follow a Student’s t distribution or a normal distribution. The distribution of the test statistic under the null hypothesis partitions the possible values of T into those for which the null hypothesis is rejected—the so-called critical region—and those for which it is not. The probability of the critical region is α.

Compute from the observations the observed value tobs of the test statistic T. Decide to either reject the null hypothesis in favor of the alternative or not reject it. The decision rule is to reject the null hypothesis H0 if the observed value tobs is in the critical region, and to accept or “fail to reject” the hypothesis otherwise. This is the probability, under the null hypothesis, of sampling a test statistic at least as extreme as that which was observed. The former process was advantageous in the past when only tables of test statistics at common probability thresholds were available.

It allowed a decision to be made without the calculation of a probability. It was adequate for classwork and for operational use, but it was deficient for reporting results. The latter process relied on extensive tables or on computational support not always available. The explicit calculation of a probability is useful for reporting. The calculations are now trivially performed with appropriate software. The former report is adequate, the latter gives a more detailed explanation of the data and the reason why the suitcase is being checked.