

As Neyman wrote: “The error that a practising statistician would consider the more important to avoid (which is a subjective judgment) is called the error of the first kind. If we set the significance level alpha to 0.05, and only reject the null hypothesis if the p-value is less than or equal to 0.05, then our hypothesis test will indeed have significance level (maximal type 1 error rate) 0.05. This definition ensures the complementarity of p-values and alpha-levels. In these circumstances (the case of a so-called composite null hypothesis) the p-value is defined by taking the least favourable null-hypothesis case, which is typically on the border between null and alternative. For each possible value of the theoretical mean, the Z-test statistic has a different probability distribution. In the just mentioned example that would be the Z-statistic belonging to the one-sided one-sample Z-test. For example, when testing the null hypothesis that a distribution is normal with a mean less than or equal to zero against the alternative that the mean is greater than zero (variance known), the null hypothesis does not specify the probability distribution of the appropriate test statistic. In contrast, in a composite hypothesis the parameter's value is given by a set of numbers. In parametric hypothesis testing problems, a simple or point hypothesis refers to a hypothesis where the parameter's value is assumed to be a single number. A p-curve can be used to assess the reliability of scientific literature, such as by detecting publication bias or p-hacking. The distribution of p-values for a group of studies is sometimes called a p-curve. By contrast, if the alternative hypothesis is true, the distribution is dependent on sample size and the true value of the parameter being studied.

For hypothesis tests about a single population mean, visit the Hypothesis Testing Calculator.In statistics, every conjecture concerning the unknown probability distribution of a collection of random variables representing the observed data X, and the underlying random variable is continuous, then the probability distribution of the p-value is uniform on the interval. For confidence intervals about a single population mean, visit the Confidence Interval Calculator. The calculator above computes confidence intervals and hypothesis tests for two population mean. The point estimate of the difference between two population means is simply the difference between two sample means ($ \bar $ A confidence interval is made up of two parts, the point estimate and the margin of error. When computing confidence intervals for two population means, we are interested in the difference between the population means ($ \mu_1 - \mu_2 $).
