This applet shows the relationship between the P-value produced by statistical software and the critical values for a chosen probability of a Type I statistical error (α).
Here, α is the chosen probability of a Type I statistical error. It determines the critical value of the test statistic for deciding whether or not to reject the null hypothesis, H0. Traditionally, for statistical significance one uses α = 0.05.
The P value is the probability, assuming the null hypothesis H0 is true, of obtaining a test statistic with a magnitude as large or larger than what was computed from the data.
This applet shows that the decision rule “reject H0 if the magnitude of the test statistic is greater than the critical value” is exactly equivalent to the decision rule “reject H0 if P < α”.