6 P-value

In our previous discussion, we examined a case study involving a digital marketing agency’s new advertising campaign. We used a z-test to evaluate the claim that the campaign has not increased the average number of purchases per day. Let us recap our findings and delve deeper into their interpretation, focusing on the crucial concept of p-values and their role in statistical analysis.

We calculated a z-statistic of 2.58, which exceeded our z-critical value of 1.65 (at a 95% confidence level). This led us to reject the null hypothesis, concluding that there was sufficient evidence to support the marketing executive’s claim of increased purchases. This is illustrated in the figure below.

Figure 18 Z critical value and z-statistic

Show description|Hide description

The image depicts a bell-shaped curve, representing a standard normal distribution, with areas marked for the ‘fail to reject’ region and the ‘rejection’ region. The curve shows the z critical value at 1.65, which marks the boundary between these regions. A z-statistic of 2.58 is located in the shaded rejection region, indicating that any test statistic beyond this point would lead to rejecting the null hypothesis.

Figure 18 Z critical value and z-statistic

Nevertheless, the z-statistic of 2.58 provides more information than simply allowing us to make a binary decision about rejecting or failing to reject the null hypothesis. It tells us how many standard deviations away from the mean our sample result lies, assuming the null hypothesis is true.

While the z-critical value approach is useful, it does not tell us the exact probability of obtaining our result if the null hypothesis were true. This is where the concept of p-values becomes invaluable.

The p-value is a fundamental concept in statistical analysis used to quantify the statistical significance of observed results. It represents the probability of obtaining an effect equal to or more extreme than the one observed, assuming that the null hypothesis is true.

In our context, the p-value represents the probability of obtaining a test statistic as extreme as, or more extreme than, our observed z-statistic of 2.58, assuming the null hypothesis (the advertising campaign had no or negative effect) is true.

Key Aspects of P-Values:

1. Null Hypothesis: The p-value is based on two hypotheses:

• H₀: Typically assumes no difference or effect. In our case, H₀ stated that the digital marketing campaign would produce equal or less than average monthly purchase results (µ ≤ 50.20).

• H₁: Assumes the null hypothesis is untrue. Our H₁ stated that the campaign would produce greater than average monthly purchase results (µ > 50.20).

2. Significance Level: Often denoted as alpha (α), the significance level is the threshold below which a p-value is considered statistically significant. In our original analysis, we used a 95% confidence level, which corresponds to a significance level (α) of 5% or 0.05.

3. Interpretation: A smaller p-value suggests stronger evidence against the null hypothesis. However, it is crucial to note that the p-value does not indicate the size or importance of an observed effect.