8 Independent t-tests
In previous activities, you learned how to enter data into Jamovi, explore datasets using descriptive statistics, and investigate relationships between variables using correlations. In this activity, you will learn how to carry out an independent samples t-test in Jamovi.
Like correlations, t-tests are used to test research hypotheses. A hypothesis is a prediction about what researchers expect to happen in a study. In experimental research, hypotheses often predict that there will be a difference between groups or conditions.
While correlations investigate relationships between variables, t-tests investigate differences between groups or conditions. For example, researchers may wish to compare whether one group scores higher or lower than another, or whether participants perform differently before and after an intervention.
An independent samples t-test is used when researchers want to compare the average scores of two separate groups or conditions. This type of test is commonly used in experiments and other studies where different participants take part in each condition.
Activity 7: Independent samples t-tests
Timing: Allow 20 minutes
The video uses a fictional research example based on a study investigating how people respond when witnessing a conflict between two individuals. Participants are shown one of two versions of a video depicting an argument and are then asked to rate how willing they would be to intervene. The study explores whether participants respond differently depending on whether they believe the individuals are strangers or in a relationship.
As you work through the activity, you will learn how to:
- identify the independent and dependent variables in an independent groups design
- run an independent samples t-test in Jamovi
- check assumptions such as homogeneity of variance
- interpret descriptive and inferential statistics from the output
- report and interpret the results of an independent samples t-test appropriately.
The video will guide you through selecting variables for analysis, choosing the correct t-test options, and interpreting the output produced by Jamovi, including means, standard deviations, t-values, p-values, and effect sizes.
Download this video clip.Video player: Video 6: Independent t-test
Transcript: Video 6: Independent t-test
SPEAKER
Independent t-test. The t-test assesses whether the means of two groups or conditions are statistically different from one another. These are reasonably powerful tests used on data that is parametric and normally distributed. T-tests are useful for analyzing simple experiments or when making simple comparisons between levels of your independent variable.
The two most common variants of the t-test are the independent t-test, which is used when you have two separate groups of individuals or cases in a between participants design, for example male versus female experimental versus control group. The repeated measures t-test, also known as the paired samples or related t-test, is used when participants provide data for each level or condition of the independent variable within participant's design, for example, before and after an intervention.
For this tutorial, we will focus on the independent t-test. There is another tutorial for the repeated measures t-test. The reason for separating them is that the data files are set up differently, and they produce different types of output. The following example demonstrates what happens after you have created the data file. See the earlier tutorial on adding variables to see how to create your own file.
Doing an independent t-test in Jamovi. This tutorial will walk you through how to run and interpret an independent t-test. The example is based on a study by Schottland and Straw, 1976, who were interested in how the perceived relationship between a couple fighting may affect the likelihood of someone intervening.
To test this, participants were split into two groups and asked to watch a video of two people fighting. The videos were identical except for one crucial line. One group sees the victim shouting, "leave me alone, I don't know you," while the other group sees her saying, "leave me alone. I should never have married you." Participants were then asked to rate how willing they would be to intervene on a scale of 1 to 5.
This is what the data looks like in Jamovi, and this can be found in the file below. For an independent t-test, the data file should have at least two columns, one for the independent variable and one for the dependent variable. Each column represents a different variable, and each row contains the data from one participant.
The different columns display the following data. The ID_No variable refers to the ID number assigned to each participant. We use numbers as identifiers instead of participant names as this allows us to collect data whilst keeping the participants anonymous. This is good practice in psychology, especially when collecting potentially sensitive data.
The variable group tells you which experimental condition participants are in. This is our independent variable. In this example, participants were split into two groups, 1, perceived relationship, 2, perceived strangers. Willingness score is a self-report rating of how willing participants would be to intervene in the fight they witnessed. This is our dependent variable.
In our example, the score is rated using a 5 point Likert scale ranging from strongly disagree to strongly agree, with strongly agree indicating the likelihood of intervention. As mentioned at the beginning of this tutorial, the independent t-test compares the scores of two groups on a certain variable. In this case, we want to compare the willingness scores of the two experimental groups.
Running the independent t-test. To start the independent t-test analysis, click on analysis, select t-tests, and click independent samples t-test. This brings up the independent samples t-test dialog box. Here, we can see all our variables from the data file displayed in the box on the left. To tell Jamovi what we want to analyze, we need to move our variables to the correct boxes on the right.
First, select the willingness score in the left hand box and click on the arrow to move it to the right hand box titled Dependent Variables. Then select the group variable in the left hand box and click on the arrow to move it to the right hand box titled Grouping Variables. Before running the t-test, we will want to check homogeneity using the Levene's test.
To do this, go to Assumption Checks, and select Homogeneity Test. The output on the right-hand side will update depending on which boxes we select. The second table in the output window, titled "Assumptions" shows us the result of Levene's test for equality of variances. This test is significant because the p value is 0.031, which is less than 0.05, meaning that equal variances cannot be assumed.
Because of this, when we run the independent t-test, we should use Welch's correction for unequal variances. To run Welch's t-test, unselect students under tests and select Welch's. Then make sure that under Additional Statistics, you've selected mean difference, effect size, and descriptives. Lastly, we need to go to the hypothesis section and specify whether we have a two tailed, non-directional or one tailed directional hypothesis.
If your hypothesis is that there will be a difference between the groups but you do not specify which group you expect to have higher scores than the other, then your hypothesis is two tailed as either group scoring significantly higher than the other would support your hypothesis. That is, the difference can be in two directions.
For a two tailed hypothesis, you would choose group 1 will not be equal to group 2. If your hypothesis is that there will be a particular difference between the groups, for example, that those who perceive the couple as strangers would be more likely to intervene, then your hypothesis is one tailed, as only significantly higher scores from that one group over the other group would support your hypothesis.
For a one tailed hypothesis, you need to select one of the options specifying the direction of your prediction. Group 1 will have higher scores than group 2, or group 1 will have lower scores than group 2. In the current example, our hypothesis is that the perceived relationship between the people fighting will have an effect on participants willingness to intervene.
As there is no specific direction to this hypothesis, this is two-tailed. Therefore, we select the box that group 1 will not be equal to group 2. Once this has all been done, your selection should mirror the image shown, and the output will update on the right-hand side of the window. The output, so what does the output show you? Let's look at the tables one at a time.
Firstly, group descriptives. We're going to look at the third and final box in our output first as it provides us with descriptive statistics for the groups. It is always useful to inspect this box before you look at anything else as it allows you to gain an initial insight into the pattern of your data. In this table, you can see that the mean willingness score for participants in the perceived relationship condition is 1.60 and 2.35 in the perceived strangers condition.
In addition, you can see from the standard deviations that the variation in the data that is the spread of the scores is a little wider for the strangers group with an SD of 1.23 than the relationship group with an SD of 0.75. It is standard practice to report these descriptive statistics when reporting your results.
So by looking at the means, you can see that on average, participants who thought the fighting couple were in a relationship with one another were less likely to be willing to intervene than those who thought the couple were strangers. But how should you interpret the difference between the means? To find out whether this observed difference between the scores is statistically significant, you next need to look at the first table in the output titled Independent Samples T-Test.
The first table in our output gives us our t-test statistics. We will go through all of the columns and statistics that you need to understand and to write up your independent samples t-test. Statistic, t-statistic, this is the value of t-test statistic that Jamovi has calculated. The t-value may be negative or positive depending on our groups.
In this case, it is negative because one group, the perceived relationship group, had a lower mean than our other group, the perceived strangers group. If our perceived relationship group had a higher mean than our perceived stranger group, than the t-value would be positive but would still have the same numeric value of 2.33. Whilst you should report whether the value is positive or negative, the important thing here is the size or magnitude of t. The larger the value of t, the smaller the probability that the results occurred by chance.
DF, Degrees of Freedom. You will come across degrees of freedom in most statistical tests. It is a value we use to represent the size of the sample or samples used in the statistical test, and it needs to be reported. The way that degrees of freedom are calculated varies for different statistical tests, but they must be calculated correctly before a test result can be checked for significance.
Don't worry too much about this as Jamovi automatically calculates the value for you. But it might be useful to note that with independent t-tests, the degrees of freedom, DF, is always close to the total number of participants. P, significance, the significance columns give us the significance value or p value for our test. If the value is smaller than 0.05, then the result is statistically significant. If it is larger, it's statistically non-significant, and we reject our hypothesis in favor of the null hypothesis, which is that there is no difference between the two groups.
Effect size, Cohen's d. To measure effect size for the independent t-test, we generally use Cohen's d. This is calculated by taking the difference between the group means and dividing it by the average standard deviation of the groups. The magnitude of the effect size can tell you how much of an effect your experimental manipulation has had.
Cohen, 1988, suggested that values of 0.8 or greater can be seen as large effect size. Values of 0.5 to 0.79 are medium effect sizes. And values of 0.2 to 0.49 are small effect sizes. Values smaller than 0.2 are very small. When reporting Cohen's d, ignore whether the value is positive or negative and simply report the number itself after the t-statistic. In this case, d is equal to 0.074.
Writing up your t-test results. When writing up the results of your t-test, you need to report whether the test was significant following this formula, t, bracket, df, equals t value, comma, p equals p value, comma, d equals Cohen's d value. Here you insert the relevant numbers from the table into the formula.
For this example, we have found that the t-test is significant as the p value is less than 0.05. This is reported as t, bracket, 31.58, close bracket, equals minus 2.33, comma, p equals 0.026, comma, d equals 0.074. Alternatively, Cohen's d can be commented on separately. So t, bracket, 31.58, close bracket, equals minus 2.33, comma, p equals 0.026, full stop. There was a medium effect size, bracket d equals 0.074, closed bracket.
Putting it all together, when interpreting and writing up your findings need to use information from the different tables provided in your output and finish by clearly interpreting your results. For example, step 1, describe the pattern of your data using the means and standard deviations. In this case, you could say something like, results showed participants who saw a relationship between the couple had lower willingness scores, bracket, m 1.60, SD equals 0.75, close bracket. Then those who did not, bracket, m equals 2.35, SD equals 1.23, close bracket.
Step two, use both words and numbers to formally report whether this difference is significant, and comment on the effect size. An independent t-test found this pattern to be significant, t, bracket, 31.58, close bracket, equals minus 2.33, comma, b is less than 0.05. There was a medium effect size, bracket, d equals 0.074, close bracket.
Step three, finally, you need to put this information together to interpret and summarize what you found in terms of your hypothesis. This should be written in plain English. For example, "Together this suggests the perceived relationship between the victim and perpetrator affects participants willingness to intervene, supporting our hypothesis." What next? Now, you've learned how to carry out an independent t-test using Jamovi. Why not try downloading the data file from this tutorial to see if you can produce the same output on your own? Remember, practice makes perfect.
Video 6: Independent t-test
Interactive feature not available in single page view (see it in standard view).
File: Independent t-test [Tip: hold Ctrl and click a link to open it in a new tab. (Hide tip)]
Now you have seen how to carry out an independent samples t-test in Jamovi, you can practise these skills yourself using the example dataset provided alongside this activity. Try reproducing the same analysis and explore how changing different options within the t-test menu affects the output. You can download a PDF version of this tutorial, and the dataset used in it below: