4.3.5 Choosing statistical tests (OPTIONAL)

This section is optional in that it provides more detail on the statistical tests that can be applied to analyse the data. If you have time and/or such analysis is important to the type of work you do, you should study this section.

Many different statistical tests have been developed, and we will only look at the most common ones in this module. Firstly, statistical tests can be classified as parametric or non-parametric. Parametric tests assume a specific distribution of data (e.g. a normal distribution for numeric data). Examples of parametric tests include the chi-squared test, linear regression and logistic regression. Non-parametric tests do not assume a specific distribution of the data. A commonly used non-parametric test is the Mann-Whitney U test for ordinal data. Non-parametric tests are generally not as statistically powerful as parametric tests, but have the advantage of needing fewer assumptions.

We will look at two categories of tests in this section. First, let us consider the tests to use when the exposure variable is a categorical variable. In this case, statistical tests allow us to assess differences between counts or means of the outcome variable at different levels of the exposure variable (e.g. assess the differences in average healing time (outcome variable) for different treatments A, B and C). The choice of such a statistical test depends on the type and characteristics of the data being analysed. One way to determine what test is most appropriate is to identify the type of outcome variable, the number of groups being compared (i.e. in our example – there are three levels of the variable ‘treatment’: A, B and C) and whether the assumptions for the test are met (this step may be informed by the descriptive analysis). Table 6 lists the most important assumptions for each test. For example, a non-parametric Mann-Whitney test might be used to test whether there are differences in the presence of AMR genes in bacterial isolates sampled from patients in hospitals, compared to patients in the community.

Statistical tests in a second category are those used when the exposure variable is numeric. In this case, tests allow users to assess the existence of a significant relationship between the (numeric) outcome and exposure variables. When there are no other variables to consider, such a test is called a correlation or collinearity test. The most commonly used correlation tests are Pearson’s r (parametric, normally distributed variables) and Spearman’s ρ rank correlation (non-parametric). For example, researchers conducting a multi-country study have data on the prevalence of resistance to a particular antimicrobial for two target organisms for 20 countries. If they want to know whether resistance to a particular antimicrobial in one target organism is related (correlated) to resistance in the second target organism, they could use a test for correlation. If additional variables need to be considered in the analysis, multivariable regression models can be constructed, though this is beyond the scope of this module.

Table 6 provides an overview of the types of statistical tests you can conduct according to the type of data you have. If you’re just starting out in learning statistics, some of this information might seem very complex, but you don’t need to understand it all now. Keep a copy of this table for future reference for when you plan and conduct your own analyses of AMR data. An important point to note for now is that results obtained from statistical tests are only valid when all the assumptions for the test are met. This important point is not always easy to assess in practice. Two important assumptions common to the seven tests presented in Table 6 are that the observations are based on a random sample and that they are independent from each other. When observations are not independent, for example, when several observations are obtained from the same individual at different points in time (repeated observations) or from a matched study design, then other tests should be used, such as paired sample t-test or mixed-effect regression models. Such tests are out of the scope of this course.

Table 6 Commonly used statistical tests to assess differences between counts or means.
Parametric or non-parametric?Outcome variableNumber of groups Footnotes   1Statistical testKey assumptions
ParametricCategorical: nominal with two levels (dichotomous)Two or moreChi-squared testExpected frequency in any cell of a contingency table is not
Non-parametricCategorical: ordinal, or numeric when assumptions for a t-test are not metTwo groupsMann-Whitney U test (Wilcoxon rank-sum test)
  • Row and column totals are fixed
  • Outcome can be ranked
Non-parametricCategorical: ordinal, or numeric when ANOVA test assumptions are not metThree or more groups Kruskal-Wallis testOutcome can be ranked
ParametricNumericTwo groupsStudent’s t-test
  • Normal distribution of outcome variable
  • Residuals have normal distribution
  • Variance is the same in both groups (otherwise use modified t-test)
ParametricNumericTwo or more groupsOne-way ANOVA
  • Normal distribution of outcome variable
  • Variance is the same in all groups
ParametricNumericTwo or more groupsSimple linear regression with one exposure variable
  • Normal distribution of outcome variable for a given exposure value
  • Linear relationship (roughly) between exposure and outcome (check with scatterplot)
  • Homoscedasticity: the variance of residuals is the same for any value of the exposure variable
ParametricCategorical: nominal with two levels (dichotomous)Two groupsBinomial logistic regressionLinear relationship between the exposure and log odds

Footnotes  

Footnotes   1This is the number of groups being compared i.e. the number of levels in the exposure variable. Back to main text
Described image
Figure 5 Example of hypothesis testing applied to a study of the resistance to cefazolin in urinary E. coli isolated in Australia between 2009 and 2013 (Fasugba et al., 2016)

Activity 8: Conducting a statistical test

Timing: Allow about 30 minutes
By signing in and enrolling on this course you can view and complete all activities within the course, track your progress in My OpenLearn Create. and when you have completed a course, you can download and print a free Statement of Participation - which you can use to demonstrate your learning.
  1. Draw a simplified two-by-two contingency table with only the count values that are of interest for the first part of this activity (comparing 2009 with 2010) from the data presented in Figure 5.

Hint: the contingency table should show the number of isolates with were identified as resistant and the number identified as susceptible in each year, in all settings.

Answer

Isolate statusYear 2009Year 2010
Resistant74120
Susceptible876944

The number of susceptible isolates each year was obtained by removing the number of resistant isolates from the total number tested, i.e. 950 and 1064 isolates in 2009 and 2010, respectively.

  1. Using an online calculator for carrying out Chi-squared tests (such as this one), use the data above to perform a Chi-squared test.

Hint: if you made your table in Excel, just copy and paste into the calculator. Alternatively, you can enter the data by copying and pasting the text below:

Isolate statusYear_2009Year_2010
Resistant74120
Susceptible876944

Answer

The p-value obtained from the test is 0.010, indicating that there is a statistically significant difference between the proportion of resistant isolates in 2009 and in 2010. Note that the result also presents the Chi-squared statistic (which is 6.62) and degree of freedom (which is one) – these parameters are used to calculate the p-value, but apart from recognising that Chi-squared tests are suitable for data where the outcome variable is categorical (Table 6), you don’t need to understand more about these parameters in this module.

  1. Do the same test to compare the proportion of resistant isolates between 2010 and 2011.

Answer

The Chi-squared statistic for this second comparison is 0.003, with one degree of freedom. The p-value obtained from the test is 0.95, indicating that there is no evidence for a statistically significant difference between the proportion of resistant isolates in 2010 and in 2011.

4.3.4 Limitations of null hypothesis testing

5 Recap: bias and error in AMR data