About p-values
A p-value is used in null hypothesis (H0) significance testing. We first define H0 and a threshold value α for p, typically 5% or 1%, where α is referred to as the confidence level of the test. If p is calculated properly, the test guarantees that the Type I error rate is at most α (Wikipedia, 2017a).
For instance, for a typical two-tailed test adopting α = 0.05,
- H0 is rejected when p < α = 0.05
-
H0 is not rejected when p > α = 0.05.
A similar reasoning applies for a one-tailed test, for instance, adopting a confidence level α = 0.05/2 = 0.0250.
If H0 is rejected, the experiment was conclusive at the confidence level selected, meaning the data is sufficiently inconsistent with H0.
By contrast, if H0 is not rejected, the experiment is inconclusive at the chosen confidence level, so the data is sufficiently consistent with H0.
Evidently, a p-value is only a tool for addressing whether to reject H0. Consider this: p < α suggests that the observed data is so sufficiently inconsistent with H0 that it may be rejected. On the other hand, p > α does not prove that H0 can be accepted. It only means that it cannot be rejected with the sample data examined. We can do better by formulating an alternative hypothesis known to be true, H1 so if H0 is rejected, we could accept H1.
However, to accept H1, we must consistently reject H0. This can be done by increasing the statistical power of the test, by increasing α; for instance, from 0.01 to 0.05 to 0.10. Thus, if one end rejecting H0 more often, there is a greater chance of safely accepting H1 (Trochim, 2006).
Calculating p-values for t-testing Hypotheses
Given t and υ,
p-values can be computed from the Cumulative Distribution Function (CDF) of the Student's t Distribution by integrating its Probability Distribution Function (PDF). These functions are shown below (Wikipedia, 2017b).
(1)
(2)
where Γ stands for a gamma function and 2F1 for a hypergeometric function with two parameters in its numerator and one in its denominator.
Numerical approximation methods are required to solve these functions, particularly considering that (2) is valid for t2 < υ. One of such approximations are given as equations 26.7.3 and 26.7.4 in Abramowitz and Stegun (1972).
Computing t from a p-value for a given υ is also possible by inverting the CDF to compute the Quantile Function (QF), also known as the inverse CDF. (Wikipedia, 2017c).
For the reverse calculations, computing t-values from p-values, you may want to use our t-values Calculator.
In addition,
our t,p & Effect Size Estimator tool allows you to easily move back-and-forth between calculations, then between CDF and ICDF transforms.
Abramowitz and Stegun (1972), Shaw (2006), and Culham (2004) describe these functions extensively. Introductory discussions on p-values are given elsewhere (Bowles, 2013; Willet, 2016).
