Are the Weston Lib Dems telling fibs?

You may have seen a few graphs doing the rounds on social media, used as evidence by both Labour and Lib Dems that if you want the Tories out, only a vote for them can achieve that.

They can’t both be right, can they?

One graph, shared by Weston Labour, shows the results from the 2017 general election in Weston-super-Mare. I know the intention of this graph is to illustrate the number of votes cast for all parties in 2017, providing you with accurate data to make your own conclusions, because I made it.

Another, shared by Weston Lib Dems, is a little bit more complicated, and a lot more instructive. Rather than giving the numbers of what actually happened in the most recent 2017 general election, it uses polling data and tells you that “only we can beat the tories here”.

Assuming that your primary motivation is to ensure that the Tories don’t win again in Weston, perhaps you’re wondering which information is accurate, and which is most likely to help you decide who to cast your vote for.

Following the response below from the Lib Dem parliamentary candidate, Patrick Keating I am going to dive into the data from the three sources they have used to create this graphic.

It’s all going to get a bit technical, so I’ll state clearly my conclusions here and leave it to you to decide if you want to carry on reading to dig down with me.

The Lib Dem graph is misleading.

The data used is inaccurate, old and/or biased.

Flavible – can’t even label a table correctly and clearly states data is national and should not conflated with constituency polling.

Electoral Calculus – old data that assumes votes in the 2015 local election can predict votes in the 2019 general election (ignoring actual results from the 2017 general election and 2019 local election in favour of their own estimates).

Best of Britain – explicitly about tactically voting for a pro-Remain MP, not about who people are most likely to vote for in the 2019 general election.

Which leads to one of two conclusions:

Either, the local Lib Dems don’t understand how to accurately use data (in which case how can we trust them to effectively improve our constituency?), or

The local Lib Dems are purposefully misleading voters.

Now let’s go on a little tour of the data sources.

The small print

The small print in the Lib Dem graphic above says the figures presented are “Based on national polling by Ipsos Mori 31/10/2019, and local projection model by Electoral Calculus Ltd.”

Here are the figures provided by the Electoral Calculus Ltd seat explorer (in graph form for easier comparison to the Lib Dem literature above).

The differences may be small, but comparing with the figures on the Lib Dem graphic above, the figures do not match source cited in the small print.

Problem 1: The small print claims that the figures presented in the Lib Dem literature are provided by Electoral Calculus Ltd, so why are the figures in the Lib Dem graph different to those provided on the Electoral Calculus Ltd website?

Perhaps it’s because as the message from Patrick Keating states, information from three sources was used. As a social scientist, I can tell you that combining multiple data sets of differing indicators is very bad science, unless you can standardise the measures. I’ve looked at this data. You can’t.

Let’s explore them one by one.

Electoral Calculus Ltd.

The website states that predictions are calculated by looking

“at the district council wards which make up the new seat, and estimate how they voted in 2017, and also to predict how they will vote at the next election. These estimates are based both on the recent local election results and the Census 2011 demography in those wards, with adjustments made to allow for different turnouts and different voting patterns for local and general elections.”

This means the first mental leap we have to make to believe in this data analysis is that who you voted for in the 2015 local election, plus estimates from census and demography data, indicates who you will vote for in the upcoming 2019 national election.

Problem 2: It is using local election data from 2015, rather than the most recent local election data in 2019.

Problem 3: It assumes that how you voted in the 2015 local election indicates how you will vote in the 2019 general election. That’s a big assumption.

Old data

Table one tells us “the district council election results for all the wards in the seat of Weston-Super-Mare. The first block shows the actual results.” with no explanation of why 2015 rather than 2019 local election results are used.

Interestingly, the total votes for the three most popular parties are not provided in this table. They are as follows:

Conservative – 33,106

Labour – 19,325

Lib Dem – 12, 451

Why is local election data from 2015, rather than 2019 being used? I don’t know, but for the sake of accuracy and transparency while digging through this murky world of figures, 2019 local election results are as follows:

Conservative – 15,393

Labour – 12,510

Lib Dem – 9,238

Or, we can look at the results from both elections in graph form:

More assumptions

The next table shows us “the raw results and adjusted for turnout and combined with the implied results from the Census Demography calculation, and then adjusted to match the general election totals. The results of these calculations are shown in the table below.” (Are you lost yet?)

That’s a few more leaps and assumptions for us to make, most importantly that the implied results from the Census Demography calculation accurately describes how people voted in 2017.

The Census Demography is used is based on two sources of data.

  • 2011 census data
  • Three waves of BES [British Electoral Survey] polls (waves 7, 10 and 13) which were conducted between April 2016 and June 2017

Problem 4: Old data from before the most recent 2017 general election is being used.

Problem 5: Where is the evidence and justification that 2011 census data and three waves of BES data can be combined to estimate how people voted in the 2017 general election?

Which brings us to the final table which apparently “shows the predicted general election result broken down over each ward in the seat of Weston-Super-Mare.”

How is this predicted? “The basic idea of the calculation is to look at the district council wards which make up the new seat, and estimate how they voted in 2017, and also to predict how they will vote at the next election.”

Problem 6: The assumption being made about how you will vote in the 2019 general election is based on how you voted in the 2015 local election, not the 2019 local election.

Problem 7: The assumption being made about how you will vote in the 2019 general election is based on estimates of how you voted in the 2017 general election, using 2015 local election results plus estimates from the Census Demography, rather than actual 2017 general election results.

It’s up to you to decide if this ‘basic idea of calculation’ paints an accurate picture. Asking these questions may help you decide:

  1. Why is local election data from 2015, rather than the most recent local election data in 2019 being used?
  2. Is it correct to assume that how you voted in the 2015 local election indicates how you voted in the 2017 general election?
  3. Why is data from before the 2017 general election being used, rather than the results of the 2017 general election?
  4. Why are estimates about how people voted in the 2017 general election based on 2015 local election results, rather than the actual results from the 2017 general election?
  5. Why are estimates of voting intention being calculated from 2011 census data and three waves of BES data rather than 2017 general election results?

Flavible

It gets worse.

Looking at the Weston-super-Mare prediction, we are given the following information. A table that gives us a prediction for the 2019 general election, and a table that gives us the percentage of votes at the previous 2017 general election.

Problem 8: The ‘Prev GE’ (previous general election 2017 results) look incorrect, until you realise this organisation can’t even label their tables correctly (the ‘Prediction’ table are the figures from the ‘Prev GE’ and so I assume the ‘Prev GE’ figures are their predictions).

BBC 2017 general election results.

That however, is not the worst of the problems with this data. Perhaps you already spotted it underneath the flavible tables?

Problem 9: Flavible make it clear that this data cannot be used to predict local results at the constituency level.

The print is small, but clear:

“The prediction table is a projection by flavible based on these national percentages, do not conflate this with constituency polling”

Best for Britain

Problem 10: This site is explicitly about using your vote to stop Brexit, not to predict the likelihood of how people will vote.

Misleading, or inept?

Our tour through the data, described by the Lib Dem parliamentary candidate Patrick Keating as “not being “misleading” – we are using the best data available in relation to what’s happening now” has highlighted ten problems, recapped below.

Electoral Calculus Ltd

Problem 1: The small print claims that the figures presented in the Lib Dem literature are provided by Electoral Calculus Ltd, so why are the figures in the Lib Dem graph different to those provided on the Electoral Calculus Ltd website?

Problem 2: It is using local election data from 2015, rather than the most recent local election data in 2019.

Problem 3: It assumes that how you voted in the 2015 local election indicates how you will vote in the 2019 general election. That’s a big assumption.

Problem 4 Old data from before the most recent 2017 general election is being used.

Problem 5: Where is the evidence and justification that 2011 census data and three waves of BES data can be combined to describe how people voted in the 2017 general election?

Problem 6: The assumption being made about how you will vote in the 2019 general election is based on how you voted in the 2015 local election, not the 2019 local election.

Problem 7: The assumption being made about how you will vote in the 2019 general election is based on estimates of how you voted in the 2017 general election, using 2015 local election results plus estimates from the Census Demography, rather than actual 2017 general election results.

Flavible

Problem 8: The ‘Prev GE’ (previous general election 2017 results) look incorrect, until you realise this organisation can’t even label their tables correctly (the ‘Prediction’ table are the figures from the ‘Prev GE’ and so I assume the ‘Prev GE’ figures are their predictions).

Problem 9: Flavible make it clear that this data cannot be used to predict local results at the constituency level.

Best for Britain

Problem 10: This site is explicitly about using your vote to stop Brexit, not to predict the likelihood of how people will vote.

Which leads to one of two conclusions:

  • The local Lib Dems don’t understand how to accurately use data (in which case how can we trust them to effectively improve our constituency?), or
  • The local Lib Dems are purposefully misleading voters.

Is this a national issue?

Is this more than a case of a local candidate using (or if I’m being kind, misunderstanding) highly questionable data to produce misleading literature?

This video of the Liberal Democrat leader Jo Swinson being interviewed by Sophie Ridge yesterday suggests misleading data is not confined to Weston-super-Mare.

Just like Jo Swinson, the Lib Dem candidate Patrick Keating is claiming that they are using up to date data to describe what’s happening now. I hope this tour of the data has shown that to be false. Is this an attempt to discredit the most recent, accurate data we have about voting intention that considers the 2017 general election results, and here in Weston at least, shows that it is Labour leading the charge?

Inspect all data

I shall finish by asking (begging!) that you inspect all the data from any source that comes at you over the next 6 weeks running up to the general election on 12th December.

Good data will be based on the most up to date information, it will thoroughly explain any assumptions so you can decide if you agree, and it will not tell you how to think but provide clear information for you to make up your own mind.

  • Check the where the data comes from
  • Ask the candidate sharing it questions
  • Ask yourself what the assumptions in the data are
  • Ask yourself if the assumptions make sense
  • Don’t share something just because it aligns with your preference.

And most of all – do more than look at data! While I understand the desire to ensure the Tories don’t win here, I can’t help feeling that tactical voting is a sorry state of affairs. Look at the policies from all parties and think about what aligns with your values.

Wishing us all an honest election,

Holly

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s