Statistical Analysis

Once you have completed your experiment, you will have collected data on twenty participants.  Just looking at the raw data, however, won't tell you very much.  You will have to carry out a number of statistical procedures in order to make sense of the data, and determine if the independent variable is really having a real effect.  Although the thought of statistical analysis is enough to keep many Psychology students awake late at night, in truth there are a number of simple steps to carry out a statistical analysis that anyone can follow.


Try it Out

Ali and Sarah carry out a replication of Craik and Lockhart's experiment on the levels of processing.  They have two groups of participants.  The first group of participants, the shallow processing group, is asked to identify whether a series of words are written in lower or upper case.  The second group of participants, the deep processing group, is asked whether each word fits in a given sentence.  At the end of the experiment, all participants are asked to recall as many words as they can.  Here are the raw results:

Group 1 (Shallow) - Words recalled (out of 20):

4, 5, 9, 6, 3, 7, 5, 4, 2, 5

Group 2 (Deep) - Words recalled (out of 20):

7, 10, 9, 6, 7, 5, 12, 8, 6, 5

It seems as if participants in Group 2 have remembered more words, but how can we actually calculate this? 

  • What statistical calculations could you apply to this data?  

  • What sort of graph could you create to summarize these results?
​​
Descriptive Statistics

Descriptive statistics simply describe and summarize the data you have collected.  There are three items to include in your descriptive statistics:  a calculation of central tendency (such as the mean) for each group, a calculation of spread (such as standard deviation) for each group, and a graph displaying the results.

  • Calculation of central tendency (mean) - The mean is simply the sum of all the values, divided by the number of values.  The mean is often referred to as the "average" of the data.  You should calculate the mean of the dependent variable for each group of participants.

  • Calculation of spread (standard deviation) - The standard deviation indicates how "spread out" the data is within each group.  If all the values are relatively close to each other, the standard deviation will be low.  However, if the data varies a great deal, the standard deviation will be large.  You can use a graphical display calculator or online calculator to easily calculate the standard deviation.

You should write a paragraph in your IA stating the mean and standard deviation for each group, as well as summarizing the calculations in a table, as shown below.


In addition, you should include a bar chart displaying the mean of the two experimental conditions.  You can create a bar chart using Excel or any similar online tool.  Make sure that your bar chart is properly labelled, as seen in the example below:
Think Critically

A team of researchers decides to carry out replications of Loftus & Palmer's famous car accident experiment in a number of different countries.  In each country, participants are shown a video of a car accident, and either asked "How fast were the cars going when they bumped into each other?", or "How fast were the cars going when the smashed into each other?".  The raw data for three countries is shown below.

Country 1 - Germany

Group A (bumped into):  45 km/hr, 50 km/hr, 60 km/hr, 40 km/hr, 50 km/hr, 60 km/hr, 80 km/hr, 55 km/hr, 50 km/hr, 55 km/hr

Group B (smashed): 50 km/hr, 60 km/hr, 40 km/hr, 45 km/hr, 55 km/hr, 55 km/hr, 75 km/hr, 50 km/hr, 60 km/hr, 60 km/hr

Country 2 - Spain

Group A (bumped into): 10 km/hr, 60 km/hr, 65 km/hr, 20 km/hr, 90 km/hr, 85 km/hr, 15 km/hr, 30 km/hr, 75 km/hr, 65 km/hr

Group B (smashed): 20 km/hr, 70 km/hr, 80 km/hr, 30 km/hr, 105 km/hr, 95 km/hr, 25 km/hr, 40 km/hr, 85 km/hr, 75 km/hr

Country 3 - Turkey

Group A (bumped into):  35 km/hr, 40 km/hr, 35 km/hr, 50 km/hr, 45 km/hr, 60 km/hr, 30 km/hr, 30 km/hr, 25 km/hr, 45 km/hr

Group B (smashed):  50 km/hr, 65 km/hr, 80 km/hr, 85 km/hr, 75 km/hr, 95 km/hr, 80 km/hr, 45 km/hr, 65 km/hr, 70 km/hr
​​


  • Calculate the mean and standard deviation for each of the six groups of participants

  • Of the three countries above, only one country has results which indicate a significant difference between the "bumped into" group and the "smashed" group.  Which country is that?  Explain why
​​
  • What is meant by a significant difference?  Use the examples above to explain your answer

Understanding significance

Whenever you carry out an experiment with two participant groups, there will always be some difference in the mean of the dependent variable between the two groups.  However, we shouldn't immediately assume that this difference has been caused by the independent variable.  There could be other variables which might have influenced the results.  Even when randomly assigning participants to different groups, it is possible that the participants in one group will be somewhat different than the other group - perhaps participants in one group happen to have a slightly better ability to remember words.

Carrying out a test of statistical significance can help us interpret the results.  If the test indicates that the result is non-significant, then there is a good chance that the result was obtained by sampling errors, participant variability, or other random factors.  If, on the other hand, the test indicates that the result is significant, then there is good reason to believe that the independent variable was, in fact, the cause of the difference between the two groups.

Looking at the results from the three countries above, you could probably guess which results were significant and which were not.  In Germany, the mean speed estimates for Group A and Group B are fairly similar, and so the difference is not significant.  In Spain, the mean for Group B is higher than for Group A, but the results are all over the place - participants in both groups made both very high and very low estimates.  When results are highly dispersed (indicated by a large standard deviation), it is more likely that other variables are playing a considerable role, and so the result is also non-significant.  Finally, in Turkey, the mean speed estimates for Group B is considerably higher than for Group A, and the results within each group are fairly consistent.  As you might have guessed, this is the only country that has produced significant results.
Video Activity - P values

Statistical tests utilize p-values to report significance.  Unfortunately, p-values are often misunderstood, not only by Psychology students, but even by many professionals!

Briefly put, the p-value is the probability of obtaining your results given that the null hypothesis is true.  Remember that the null hypothesis states that the independent variable has no effect.  Therefore, the lower the p-value, the more unlikely it is that your results can be attributed to random factors alone.  When the p-value is lower than a particular cutoff point (usually 0.05), it is quite unlikely (less than a 5% chance) that random factors alone have produced your results.  Therefore, if p is less thean 0.05, the result is considered to be significant - the independent variable is most likely having a significant effect on the dependent variable.

Watch the video "What is a P-value" below and answer the following:

  • What is the null hypothesis and alternate hypothesis about the coin?

  • After how many coin tosses of only "Tails" is the p value less than 0.05?

  • If the p-value is very low, what decision should be made regarding the null hypothesis?






Inferential Stats - In practice

Now that you understand some of the theory behind inferential statistics, it is time to actually carry out your inferential statistical test.  First, the good news - you are not required to carry out complicated calculations by hand.  Instead, you can use an online statistics calculator to carry out the test.  Just make sure that you include a screenshot of the online calculator in your Appendix.

If you have carried out an experiment using an independent samples design - meaning that there are two groups of participants - you can use the Mann-Whitney U Test to determine statistical significance.  This is a non-parametic test, meaning that it is not important whether the data in each group is normally distributed.  

To carry out a Man-Whitney Test, you can click on the link below to view a useful online calculator.  Below is a screenshot of the online calculator, followed by an explanation of each of the options

Mann Whitney U Test Online Calculator


Below is an explanation of how to use the online calculator:

  • Enter in your data in the "Sample 1" and "Sample 2" box, seperated by commas.  In the screenshot above, the data has been entered from the replication of Loftus & Palmer carried out in Turkey.  "Sample 1" are the participants who were asked how fast the cars were going when they "bumped into" each other, and "Sample 2" are the participants who were asked how fast the cars were going when they "smashed" into each other

  • Significance level is usually set to 0.05 in Psychology and other social sciences

  • A one-tailed hypothesis is used when you are making a specific prediction regarding which sample will be greater than the other.  It is recommended to use a two-tailed hypothesis, in which your alternative hypothesis is simply that the independent variable will have a significant effect, without specifying what kind of effect you think that will be

Once all your data and settings have been entered, press "Calculate U" to see the results of the test
Here is how to interpret the result:

  • The U-value is 4.5, which is lower than the critical value of 23.  If the U-value is lower than the critical value, as in this case, the result is significant - meaning that we reject the null hypothesis

  • We can also see this by looking at the p-value, which is 0.00068.  Since this p-value is less than the significance level of 0.05, the result is significant - meaning that we reject the null hypothesis
Quiz Yourself!

1.  Which of the following is not an example of descriptive statistics?

(a) Calculation of central tendency

(b) Calculation of spread

(c) Graphical display of data

(d) Mann-Whitney U-test


2.  Complete the blanks:  "It is more likely that the results are significant if the mean difference between the two groups is ________ and the spread of data within each group is ___________"

(a) Large / high

(b) Large / low

(c) Small / high

(d) Small / low


3.  The p-value tells you the probability,,,,

(a) That the null hypothesis is true, given the data you have obtained

(b) That the alternative hypothesis is true, given the data you have obtained

(c) That the results are significant

(d) That the results are non-significant


4.  Which of the following is not a requirement for carrying out a Mann-Whitney U-Test?

(a) Two independent groups of participants

(b) Data is measured quantitatively

(c) Data within each group is normally distributed



5.  If the Mann-Whitney U-Test returns a p-value of greather than 0.05, what decision should be made?

(a) Reject the null hypothesis, as the result is significant

(b) Reject the null hypothesis, as the result is not significant

(c) Retain the null hypothesis, as the result is significant

(d) Retain the null hypothesis, as the result is not significant


Answers

​1 - D, 2 - B, 3 - A, 4 - C, 5 - D