Power vs separation: How the language of polls can influence results and public opinion

New data published by the Scottish Social Attitudes survey shows stronger support for devolution than independence - but is everything as it seems?

POLLS are essential in providing a backdrop into public opinion. They can cover many different topics such as independence, welfare and foreign affairs. But how much does the language used in polls and the way questions are worded influence someone's opinion?

A particular topic of debate in Scotland is the issue of independence versus devolution. ScotCen Social Research, the Scottish branch of NatCen Social Research, is a non-profit organisation specialising in surveys. Its 2015 Scottish Social Attitudes (SSA) survey polled a sample of 1,288 people between July 2015 and January 2016. It asked questions about constitutional preference and about who should make governmental decisions in Scotland, among other things.

Its key findings showed that support for independence is at a 15-year high but also showed quite divided opinion between the people who took part. A total of 39 per cent of people affiliate their constitutional preference with independence. Despite this, a total of 51 per cent of people think that the Scottish Parliament should make all decisions for Scotland.

The survey showed a total of 39 per cent of people affiliate their constitutional preference with independence. Despite this, a total of 51 per cent of people think that the Scottish Parliament should make all decisions for Scotland.

The results are intriguing because of the similarity of the statements. While only 39 per cent of people want full independence, 51 per cent of people want all decision-making for Scotland to be given to the Scottish Parliament. If one group of people wants independence for Scotland and another wants the Scottish Parliament to make all decisions for Scotland, how are they much different?

John Curtice, senior research fellow at ScotCen Social Research, says that the phrase "separate from the UK" in the independence question is an important detail.

"One obvious distinction between the two questions is that the question that focuses on the powers of the parliament doesn't explicitly say this, therefore, means that Scotland would not be part of the United Kingdom," he says.

"Whereas the constitutional preference question not only says that Scotland should [become] independent but it also [says] separate from the UK. It drives the point home."

On the other hand, Stewart Kirkpatrick, digital consultant and head of Scotland at activism organisation 38 Degrees , argues that "the two statements are identical".

"My personal view is that the two statements are identical. If the Scottish Parliament is in control of all matters then you are independent." Stewart Kirkpatrick, digital consultant

He says: "People have their own personal view of what independence means. Maybe they view some kind of devo max or fully federal vision between independence and the Scottish Parliament having control of everything.

"My personal view is that the two statements are identical. If the Scottish Parliament is in control of all matters then you are independent. There might be people who view that kind of scenario happening where there is still a second chamber that's UK-wide or some government structures that are UK-wide that encompass that."

On the question of how the language of surveys can contribute to mixed results, Kirkpatrick says that the structure of a question is very important.

"When you are constructing opinion polls you need to be very careful about making sure your question is neutral, that your question isn't leading but also that your question makes clear what it is people are being asked so that they are very clear about what the terms mean.

"When you are dealing with a term like independence, it sounds like it is incredibly obvious but actually as we saw during the independence referendum there are lots of different visions of independence and lots of different understandings of how things like, say, currency might work.

"It's not a perfect measure, but it's the only long-term measure we've got, it's the only handle we've got on whether or not Scotland really is changing." Professor John Curtice

"It's not surprising that members of the public, when they've been asked questions, give answers that reflect that complexity."

Poll results are important for active campaigners as an indicator of how successful or otherwise their efforts are with the public, and can offer hints on where resources might be best placed. Kirkpatrick was previously head of digital at Yes Scotland, and although he is no longer involved in any active campaigning for independence, he says if he was, that's where his focus would be drawn.

"Let's interrogate what's being meant there, where people are saying that they don't want independence but they do want the Scottish Parliament to have full powers," he says. "What does that mean? How do we get people to join up those two statements in a way that means that the public view them as identical in the way that campaigner[s] might view them as identical?

"I'm certain as part of that process that there will be people within the SNP and other pro-independence parties looking at those kind of results and working out how to bridge that gap."

The process of deciding which questions to ask in a poll is important to the way that the results will be interpreted. ScotCen has asked the independence question in every one of its annual SSA surveys since 1999, which, according to Curtice, helps provide a useful overview of how public opinion in that area is changing.

"One of the things we are crucially trying to do on SSA is to understand attitudinal change over time," he says. "Indeed, the long-running five-point question from everything from independence outside of Europe to no parliament at all, that is the only question that has been asked regularly. It's the only measure we have of how public opinion in Scotland is shifting.

"I don't think anybody, if they were starting this poll now, would phrase it like this. There is such a strong feeling now that the word 'separate' is pejorative so you wouldn't use that now." James Kelly, Scot Goes Pop

"It's not a perfect measure, but it's the only long-term measure we've got, it's the only handle we've got on whether or not Scotland really is changing."

James Kelly, who runs survey analysis website Scot Goes Pop , agrees, adding that the wording in the independence question in the SSA survey is outdated, although it does offer consistency.

"I don't think anybody, if they were starting this poll now, would phrase it like this," he says.

"There is such a strong feeling now that the word 'separate' is pejorative so you wouldn't use that now. But in a sense they're doing it for a good reason which is to maintain consistency so that you can see how support for each of the options has changed over time. The wording is not ideal because the debate has moved on since 1999."

If the Scottish public had voted Yes for independence in 2014, 24 March 2016 would have been the official day for independence. In terms of how much public opinion has changed since then, Kirkpatrick looks at the different elements to consider.

He says: "I think there are two dimensions to that: one is the political dimension which is Scotland once again has a [UK] government for which it did not vote, so the political strand of the independence debate has not moved forward. But [in terms of] the economic strand, as has been acknowledged by SNP politicians, more work needs to be done to make the case on the economic side for Scotland to be independent and that work's not been done yet.

"There will be people within the SNP and other pro-independence parties looking at those kind of results and working out how to bridge that gap." Stewart Kirkpatrick, digital consultant

"I don't think public opinion has shifted enormously from where it was because newer cases and arguments around the economic side have not been made since the vote."

The SSA survey results are at least notable because they show support for independence at its highest point in 15 years, even if there is some disparity in the detail. Ultimately, it will be up to both pro-independence campaigners and pro-union campaigners to analyse those results to best help them strengthen their cases in the battle for Scotland's future.

CommonSpace journalism is completely free from the influence of advertisers and is only possible with your continued support. Please contribute a monthly amount towards our costs . Build the Scotland you want to live in - support our new media.

Picture courtesy of Mark Ferbert