4.4 Non-directional hypotheses

Non-directional hypothesis testing involves testing the null hypothesis that the population mean is equal to a certain value. The alternative hypothesis asserts that the population mean is not equal to that value. As mentioned previously, the test of such hypothesis is two-tailed, in the sense that the null hypothesis gets rejected whenever there is a sizable difference between the sample mean and the hypothesised mean in either direction.

But how sizable this difference needs to be in order for us to reject the hypothesis? This is where the concept of critical values comes in. Critical values play a vital role in statistical hypothesis testing and confidence interval construction. A critical value is a threshold that defines the boundary between the fail to reject and rejection regions in a statistical hypothesis test. It marks the point where a test statistic becomes statistically significant, allowing decision-makers to make decisions about rejecting or failing to reject the null hypothesis.

There are several types of critical values, depending on the statistical test being performed. For example:

• Z critical value: Used for z-tests and based on the standard normal distribution.

• T critical value: Used for t-tests and based on the t-distribution.

The two-tailed nature of a hypothesis becomes evident when we consider the rejection region, which includes outcomes from both the upper and lower tails of the sample distribution.

Hypothesis testing is fundamentally rooted in probability theory. When we conduct a hypothesis test, we essentially ask: "What is the probability of observing our sample results if the null hypothesis were true?"

In our coffee foam example, we are examining the probability distribution of sample means. Under the null hypothesis (H₀: μ = 1 cm), we assume that if we were to repeatedly draw samples and calculate their means, these sample means would follow a normal distribution centred around 1 cm.

 

We establish two competing hypotheses:

 

• H₀: μ = 1 cm
• H₁: μ ≠ 1 cm

Here, μ represents the true population mean foam height. The null hypothesis assumes no difference from our target value (1 cm), while the alternative hypothesis suggests a significant deviation.

Our chosen significance level, α = 0.05, has a direct probabilistic interpretation. It means we are willing to accept a 5% chance of rejecting the null hypothesis when it is actually true. In a two-tailed test, we divide this 5% probability equally between the two tails of the distribution. This results in:

• 2.5% probability in the left tail (area below z = -1.96)
• 2.5% probability in the right tail (area above z = +1.96)

It is worth noting that the orange area in the figure is equivalent to 5%. The ‘fail to reject region’ needs to amount to the remaining 95%, because probabilites of all possible outcomes sum to 100%.

The central 95% of the distribution (between z = -1.96 and z = +1.96) represents the range of sample means that we would consider "not unusual" if the null hypothesis were true. The rejection region in our two-tailed test corresponds to the areas of low probability under the null hypothesis. Specifically:

• If our test statistic falls in either tail (z < -1.96 or z > 1.96), it means we have observed a result that had less than a 2.5% chance of occurring if H₀ were true.
• The total probability of falling in the rejection region is 5% (2.5% + 2.5%).

Interpreting Results Probabilistically:

• If we reject H₀: We conclude that we have observed a result that had a less than 5% chance of occurring if the true population mean were 1 cm. This low probability leads us to doubt the null hypothesis.
• If we fail to reject H₀: We have observed a result that had a greater than 5% chance of occurring under H₀. This does not prove H₀ is true, but suggests we lack strong evidence against it.

It is crucial to understand that these probabilities relate to the long-run frequency of results if we were to repeat our sampling process many times. They do not tell us the probability that H₀ is true or false in any single test.

We will explore how to calculate the actual probability of our observed results using the z-test later. This will involve determining where our sample mean falls in this probability distribution and calculating the associated p-value.