9 Repeated measures t-tests

The video uses a fictional research example inspired by studies examining the so-called ‘Hungry Judges’ effect – the idea that hunger may influence decision-making. In this example, participants rate the severity of fictional crimes before and after a lunch break, allowing researchers to investigate whether sentencing decisions differ depending on hunger levels.

As you work through the activity, you will learn how to:

• recognise when a repeated measures design is being used
• understand how repeated measures datasets are organised in Jamovi
• run a paired samples t-test in Jamovi
• interpret descriptive and inferential statistics from the output
• report and interpret repeated measures t-test findings appropriately.

The video explains how repeated measures data differs from independent groups data, demonstrates how to select paired variables for analysis, and guides you through interpreting the resulting output, including the means, standard deviations, t-values, degrees of freedom, and significance values produced by Jamovi.

Download this video clip.Video player: Video 7: Repeated measures t-test

Show transcript | Hide transcript

Transcript: Video 7: Repeated measures t-test

SPEAKER

Repeated measures t-test. The t-test assesses whether the mean scores from two experimental conditions are statistically different from one another. A repeated measure t-test, also known by other names, such as the paired samples or related t-test, is what you should use in situations when your design is within participants.

In a within-participants design, participants contribute data for the dependent variable in all of the experimental conditions. If you have a study design where different participants take part in your different experimental conditions, then you have a between-participants design. In this case, you would need to use the independent t-test. There is a separate tutorial for this type of test.

The following example demonstrates what happens after you have created the data file. See the tutorial on adding variables for more information on how to create your own file. Doing a repeated measures t-test in Jamovi. This tutorial will walk you through how to run and interpret a repeated measures t-test in Jamovi. In this tutorial, we will examine fictional data based on a study by Muncurlkin et al. in 2021 that was inspired by the Hungry Judges effect.

This is the finding that hunger can influence judges sentencing decisions, with studies showing that judges may be more lenient when sentencing after a meal break and more severe before a meal break. To test this effect, the researchers arranged a day of testing in which participants were given 10 vignettes, or short descriptions, which depicted fictional crimes committed by a fictional individual.

For each vignette, participants scored how severe a sentence they would give to that crime if they were judged from 1, least severe, to 10, most severe. Crucially, participants saw five of the vignettes in the morning before a lunch break and five of the vignettes after a lunch break. The seriousness of the crimes depicted in the vignettes before and after lunch would need to be matched, and the order they were presented should be counterbalanced to prevent any order effects occurring.

As we are interested in the effect that a lunch break might have on sentencing severity, the lunch break is our independent variable. It has two conditions, before the break and after the break. As participants see all the vignettes delivered both before and after lunch, this is a within-participants, also known as a paired samples or repeated measures design. As the variable we are measuring is the severity of the sentence the participants give to the crimes described in the vignettes, our dependent variable is sentence severity.

This is measured by taking the mathematical average or mean of the severity scores for the five vignettes given before and after the lunch break. In this example, higher scores indicate more severe judgments. This is what the data looks like in Jamovi, and this can be found in the file below. The different columns display the following data. Participant ID, this refers to the ID number assigned to the participants.

We use these numbers as identifiers instead of participant names, as this allows us to keep participants' data together whilst also keeping the participants anonymous. This is good practice in psychology, especially when collecting potentially sensitive data. Before severity. This column contains the mean severity scores for the five different vignettes seen before lunch, e.g. if a participant scored the five vignettes before lunch as 2, 2, 4, 4, 3, then the score shown for that participant in this column would be the mean of that data, which is 3. These are the DV scores for the first experimental condition.

After severity. This column contains the mean severity scores for the five different vignettes seen after lunch. These are the DV scores for the second experimental condition. Have you spotted the difference between this data file and the one we used for the independent t-test? While the data file for an independent t-test contains a specific column that defines and codes the conditions of the independent variable, this is not the case for a repeated measures t-test.

Instead, within-participants IV is encoded by the columns representing the two conditions under which the dependent variable was measured. In this case, before severity and after severity. Using this approach ensures that all the data from a single participant is in just one row in the data file. Running the repeated measures t-test in Jamovi. To start the repeated measures t-test analysis, click on analyses, select t-tests, and clicked paired samples t-test.

This brings up the paired samples t-test dialog box. Here we can see all our variables from the data file displayed in the box on the left. To tell Jamovi what we want to analyze, we need to move our two variables that represent DV to the correct box on the right. First, select before severity in the left hand box and click on the arrow to move it to the right hand box titled paired variables.

Then select after severity in the left hand box and click on the arrow to move it to the right hand box titled paired variables. In the paired variables box, which you can see here, both conditions of your experiment now create one pair. Pair one. In other words, they provide a pair of data points for Jamovi to compare. Hence the paired samples terminology used by Jamovi to describe a repeated measures design.

If you wanted to conduct more than one t-test at this point, you could insert more pairs of conditions into this box. Before running the t-test in Jamovi, go to the hypothesis section and make sure that you confirm whether you have a two-tailed non-directional, or one tailed directional hypothesis. If your hypothesis is that there will be a difference between the conditions, but you do not specify which conditions you expect to have higher scores, then your hypothesis is two-tailed as either condition resulting in significantly higher scores than the other would support your hypothesis.

If this is the case, you would select the box measure 1 is not equal to measure 2. If your hypothesis is that there will be a particular difference between the conditions, for example, that participants would be more severe in sentencing before lunch than after, then your hypothesis is one-tailed, as only significantly higher scores from that condition over the other would support your hypothesis.

For a one-tailed hypothesis, you need to predict one of the options shown specifying the direction of your prediction. Measure 1 will have higher scores than measure 2, or measure 1 will have lower scores than measure 2. In the current example, our hypothesis is that sentences will be more severe before lunch. This is a one-tailed hypothesis, as we have predicted that scores will be higher before lunch, which is measure 1 than after lunch, measure 2. We therefore select the option, measure 1 is greater than measure 2.

Before running the t-test, we also want to make sure that we've selected descriptives. Once all this has been done, your selection should mirror the image shown and the output will update on the right hand side of the window. The output. So what does the output show you? Let's look at the tables one at a time.

Group descriptives. We're going to look at the second and final box in our output first as it provides us with descriptive statistics for our two conditions. It is always useful to inspect this box before you look at anything else, as it allows you to gain insight into the pattern of your data. We are mainly interested in the mean and the standard deviation here.

We can see from the two means that the participants made more severe sentencing decisions before the lunch break, where the mean was 6.12, than after the lunch break, where the mean was 4.39. We can also see from the standard deviations, before severity, standard deviation was 2.31. After severity, standard deviation was 2.30. That the scores in both conditions are similarly dispersed. Note the value under N refers to the number of participants in each condition.

Don't forget, though, that the same participants take part in both conditions. So in this case, we had 107 participants participate in both conditions. How should you interpret the difference between the means? To find out whether this observed difference between the scores is statistically significant, we next need to look at the first table in the output titled paired samples t-test.

The first table in the output gives us our t-test statistics. We will go through all the columns and statistics that you will need to understand and write up your paired samples t-test output. Statistic, t-test statistic. This is the T value calculated by the repeated measures t-test. This is an important statistic that you will need to report when writing up your findings.

It is an expression of the difference between the scores in your two experimental conditions. The larger the value of T, the more pronounced the difference between the conditions and the smaller the probability that this difference occurred by chance. df, degrees of freedom. This is an important statistic that needs to be reported, as its value directly impacts the significance of the T-statistic.

In a repeated measures t-test, the value of df will be one less than the number of participants in the study. In this case, there are 107 participants, so the degrees of freedom or df is 106. p, significance. The significance columns give us a significance value or p-value for our test. If the value is smaller than 0.05, then the result is statistically significant. If it is larger so statistically non-significant, then we reject our hypothesis in favor of the null hypothesis, which that there is no difference between the two groups.

Writing up your t-test results. When writing up the results of your t-test, you need to report whether the test was significant following this formula. t bracket degrees of freedom or df close bracket equals t value comma p equals p value. You insert the relevant numbers from our table into the formula. For this example, we have found that the t-test is significant as the p-value is less than 0.05. This is reported as t bracket 106 Close bracket equals 5.36 comma p is less than 0.001.

Putting it all together. When interpreting and writing up your findings, you need to use information from both the descriptive and inferential statistics in your output. It doesn't matter which order you report these two types of statistics, but always finish by interpreting your results meaningfully. Step one, describe the pattern of your data using the means and standard deviations from the first output table.

In this case, could say something like, results showed participants made harsher sentencing decisions before lunch, bracket M 6.12, SD equals 2.31 close bracket than after lunch bracket M equals 4.39, SD equals 2.30 close bracket. Step two, report the test you used if it was one-tailed or two-tailed, and whether or not the finding was significant. For example, a one-tailed repeated measures t-test found this difference to be significant, t bracket 106 close bracket equals 5.36 comma p is less than 0.001.

Step three, finally, you need to put this information together to interpret and summarize what you found in terms of your hypothesis. This should be written in plain English. For example, this finding supported our hypothesis that sentencing severity would be affected by hunger. So what next? Now you've learned how to carry out a repeated measures t-test using Jamovi, why not try downloading the data file and see if you can produce the same output on your own?

 

End transcript: Video 7: Repeated measures t-test

Download

Video 7: Repeated measures t-test

File: Repeated measures t-test [Tip: hold Ctrl and click a link to open it in a new tab. (Hide tip)]

Now you have seen how to carry out a repeated measures t-test in Jamovi, you can practise these skills yourself using the example dataset provided alongside this activity. Try reproducing the same analysis and explore how the output changes when different options are selected.

You can download a PDF version of this tutorial, and the dataset used in it below:

File: Repeated measures t-test tutorial

File: Paired t-test dataset