Beware Generalisation
Continuing from my musing yesterday about supporting evidence, I started thinking about another potential trap when writing: generalisation.
Having done your due diligence, gone to the primary sources and been sceptical in your reading, two key issues still remain.
Applying the Particular to the General
Data from studies generally only covers a very small sub-set of a population. For example, it is well near impossible to survey all the inhabitants of a country (though census data tries to come close). Thus surveys which claim X per cent of people in Australia think this or Y percent of people in America do that, is generally an extrapolation of data based on a sub-set of the whole population. Perhaps a sample of 1,000 people was used, from which a data model was built to make a broad claim about the population as a whole.
While this is reasonable, it does mean a writer can fall into the trap of imagining because 67% of respondent to a survey support X, there is genuinely wide spread support. When in reality, the people chosen and the methodology of the questions asked has in fact skewed the sample.
Skewed Samples
A wonderful example skewing a sample though leading questions comes from Yes Prime Minister. In a memorable scene, Sir Humphrey demonstrates to Bernard how leading questions can sway a respondent to be both for and against National Service:
Sir Humphrey Appleby: Mr. Woolley, are you worried about the number of young people without jobs?
Bernard Woolley: Yes
Sir Humphrey Appleby: Are you worried about the rise in crime among teenagers?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think there is lack of discipline in our Comprehensive Schools?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think young people welcome some authority and leadership in their lives?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think they respond to a challenge?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Would you be in favour of reintroducing National Service?
Bernard Woolley: Oh, well I suppose I might.
Sir Humphrey Appleby: Yes or no?
Bernard Woolley: Yes.
Sir Humphrey: Of course you would, Bernard. After all, you told you you can’t say no to that. So they don’t mention the first five questions and they publish the last one.
Conversely, the wily civil servant shows his junior how the opposite response can be achieved:
Sir Humphrey Appleby: Mr. Woolley, are you worried about the danger of war?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Are you worried about the growth of armaments?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think there's a danger in giving young people guns and teaching them how to kill?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think it's wrong to force people to take arms against their will?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Would you oppose the reintroduction of National Service?
Bernard Woolley: Yes.
Sir Humphrey Appleby: There you are, you see, Bernard. The perfect balanced sample.
With that example of wisdom...
Good night and good luck.
Image by Sydney Rae is liscensed under Unsplash.
This post is day 085 of my #100DaysToOffload challenge. If you want to get involved, you can get more info from 100daystooffload.com.