2 Testing with data

Once hypotheses have been developed, the next step is to find existing data or design a study (to generate some primary data) to test them. The results of the testing will have two possible outcomes.

1. Reject null hypothesis (H₀) – this also means that the alternative hypothesis (H_a) is accepted.
2. Fail to reject null hypothesis (H₀) – this means that you accept H₀ and reject alternative hypothesis (H_a).

You can test hypotheses empirically using test statistics that involve calculating sample data and using the results to decide whether you need to reject the null hypothesis or fail to reject the null hypothesis.

A ‘test statistic’ is a number calculated by a statistical test, which provides information about how much the relationship between variables in the test differs from the null hypothesis. There are different types of test statistics and the choice of test statistic depends on the type of hypothesis being tested and the nature of the data. For example, if the data is continuous and normally distributed, the t-test or z-test may be used as a test statistic. If the data is categorical, the chi-squared test may be used. Some of these will be discussed later in this course.

To begin, you will first obtain an overview of the general guidelines for using the test statistic to determine whether to reject or fail to reject the null hypothesis. Consider the coffee example from Activity 1. In order to test the null hypothesis (H₀: µ = 1cm foam), you might sample 60 cups of caffè latte made by the coffee machine – i.e. select 60 cups of caffè latte at random (random sampling) from all the cups of caffè latte that the coffee machine made throughout the day. Then, you would measure the height of the foam to obtain the average height and calculate the test statistic.

You may be wondering why here the null hypothesis and not the alternative hypothesis (H₀: µ ≠ 1cm foam) is being exclusively discussed. In the realm of hypothesis testing, you commence by assuming that the null hypothesis is true and then employ sample data to determine whether or not to reject it. If you reject the null hypothesis, you can infer that the alternative hypothesis is true. As you may recall, the null and alternative hypotheses are complementary and mutually exclusive. The null hypothesis denotes the prevailing belief, whereas the alternative hypothesis represents what you aim to demonstrate. Therefore, rejecting the null hypothesis implies that you have evidence to support the alternative hypothesis. It is noteworthy that you cannot directly prove the alternative hypothesis. Instead, you can only reject or fail to reject the null hypothesis. The decision to reject or fail to reject the null hypothesis is grounded in the test statistics.

Figure 4 Caffè latte

Show description|Hide description

A cup of caffè latte with milk foam.

Figure 4 Caffè latte

For the study:

• Researcher 1 samples 60 cups of caffè latte and gets the average foam height equal to 1.1 cm.
• Researcher 2 samples 60 cups of caffè latte and gets the average foam height equal to 1.5 cm.
• Researcher 3 samples 60 cups of caffè latte and gets the average foam height equal to 2.6 cm.

The question now is how to make your decision. Using only the data collected by Researcher 1, the average height of the foam is 1.1 cm. It is not exactly equal to 1 cm, but it is very close. In this case, you may say that you cannot reject the null hypothesis. However, if you look at the data collected by Researcher 3 (the average height of the foam is equal to 2.6 cm), this is far beyond the commonly accepted idea of 1 cm foam for a caffè latte. Therefore, you can reject the null hypothesis and accept the alternative hypothesis (µ ≠ 1 cm foam). Considering the data collected by Researchers 1 and 3, it is easy to decide whether to reject the null hypothesis or not. However, based on the data collected by Researcher 2, it is extremely difficult to make a decision. Although the average foam height of 1.5 cm is not far away from 1 cm, does it go far enough to be considered sufficient?

In order to answer this question, you would need to introduce the concept of ‘statistical significance’, which you will look at in more detail in the next section.