How Many Times Do We Have To Teach You This Lesson, Old Man? - Or, Why Do Politicians Get Things Wrong?

© UK Parliament / Maria Unger, CC BY 3.0 <https://creativecommons.org/licenses/by/3.0>, via Wikimedia Commons

We’ve all done it. Watched some election results come in, or a news story break, and sat there tearing our hair out. Our political party or politician of choice got something wrong, and badly wrong at that. We’re not just talking about bad circumstances here, but actual mistakes and blunders. See, it seems obvious. Starmer should just have done something about immigration!, we cry to ourselves. This sort of mistake is different from a gaffe, which really refers to slips of the tongue or misjudged comments made by politicians who thought they were off the record. This is more like a strategic misjudgement, or some kind of inflexibility. I’m talking about clearly wrongheaded but very much deliberate and considered strategy. In my view, it’s surprisingly hard to explain why this kind of mistake happens as often as it seems to.

See, there’s definitely a lot of this sort of thing. It’s hard to see Labour’s recent drubbing in the local elections in any other light. Voters are furious about Labour’s removal of the winter fuel allowance, and angry that nothing seems to have been fixed yet almost a year into the new government’s tenure. In many respects, Labour seems to have chosen the worst possible route. Sir Keir Starmer has made none of the radical changes that seem necessary to many voters to fix Britain - be that in terms of wide planning reform, systematic tax reform, or administrative restructuring - and has instead delivered a platter of small yet thoroughly foul-tasting nibbles for voters: saving barely a billion on winter fuel, less than a billion on inheritance tax, and so on. None of this is making a dent in Britain’s fiscal situation, but it is making the Labour Party desperately unpopular. This is all ignoring a key issue that Reform have made a killing on - immigration. The UK’s net migration figures did increase significantly between 2020 and 2023 and remains relatively high, and voters concerned about this don’t feel heard. 

So why is this hard to explain, as I claim? After all, it would be easy enough to argue that Keir Starmer is just a fallible human, like everyone else. All politicians are. Being elected doesn’t guarantee much intelligence or competence, in theory. However, we aren’t just talking about individuals here. To take Starmer as the “extreme” case, the Prime Minister doesn’t come up with policy ideas on his own. He has a swarm of both Party and Civil Service assistants who draft proposals for him and research ideas. He has Cabinet members, to whom he delegates a great many tasks. The same goes not only for every other Cabinet member, but pretty much every MP in the country. Politicians exist in a party ecosystem with both specialized politician friends and non-politician assistants available to help them come up with ideas. They spend all of their waking hours on politics, and are surely constantly thinking about how to improve their position. So how is it that even with an army of aides, politicians get things wrong that seem obvious to us, non-professionals who occasionally browse social media?

One could argue that it’s all just a matter of retrospect. It’s a lot easier to say that a different strategy is better when that proposition can no longer be tested easily - when “would be” becomes “would have been”. Furthermore, there’s selection bias at play: we’re more likely to remember when we make a correct judgement that a politician does not than vice versa! However, it’s also not the whole story. There are some interesting systematic biases affecting politicians, and, I would suggest, there pretty much always will be. As a consequence, we should think about politicians - and their errors - in a new way.

Without further ado…

Retrospect and the Inscrutable Voter

Of course, it really is easier to be right in retrospect than in advance. This is especially true in an electoral democracy. Judgements are better made when we can see the results, and voters are notoriously unreliable until they vote. One aspect of this is honesty and self-knowledge. YouGov’s internal data show that people frequently vote one way in a given election, but then change who they said they voted for when polled on it a few years later. Similarly, they will happily say one thing in a poll and then do another in an election. Though pollsters try to correct for this, it’s impossible to get it perfectly right. There’s a reason newspapers are always publishing stories about how rubbish their own in-house polling was immediately after the election, prompting even-more-clueless-still pundits to write thinkpieces on why all polling is pointless. Trusting what some random journalist thinks is much more reliable, after all. Parties and politicians can commission all the polls they like, but people’s actual actions are impossible to get down precisely until they happen.

Of course, that doesn’t explain why politicians seem to get obvious things wrong. Here, something more interesting is at play. Voters, as I’ve said, don’t always know what they want, or what they’re thinking. Nobody does. However, they will happily strategically change how they say what they think or what they say depending on who’s asking - pollsters whom they think it’s funny to troll, politicians whom they think they ought not to offend, or mates whom they don’t mind letting loose around. We all have very good intuitive senses of what voters are thinking at any one point, because we talk to “voters” all the time. We just know them under different names - friends, neighbours, family members, acquaintances, and so on. While we don’t always know what we want, we often know what others are thinking and feeling in a tacit way.

This “tacit knowledge” is the sort of thing economists are very used to talking about. There’s a famous paper by Friedrich von Hayek, an excellently-named mid-century economist, called “The Uses of Knowledge in Society”. (Oh, to live in the days when one could get away with such simple names for papers!) He talked about how no central planner can ever be more efficient than a free market is, because the central planner can’t ever have enough information to make all the correct judgements they’d need to. After all, people sometimes go shopping without knowing everything they’ll buy when they get there. People know things they can’t articulate. I think that principle roughly applies here. Voters have decent functional senses of what they’re thinking as a group, but they may not be able to articulate it well or helpfully to politicians. I certainly wouldn’t talk to the Prime Minister as though he were my best friend, and I don’t think anyone else would. This makes collecting and relying on information about public attitudes rather difficult.

However, this is far from a complete explanation. While it’s hard to rely on polling data or individual interviews, it’s certainly not impossible. Individual voters’ impressions of the “atmosphere” are far from perfect, too: they’re heavily coloured by social circles and area, among other things. Revealed preference is an economic concept, defined as the preference people have when put to the test - rather than the preference they say they have. The implication is that sometimes, we don’t actually know what we’ll prefer until we have to choose between two things. We’ve all done it - flipped a coin on whether we’ll go out for the evening or stay in, and then decided that it’s best of 3 because we didn’t like the result. No, best of 5! No, uh, best of 7… However, this means that some political stuff is actually, strictly, unknowable. Unknowable to anyone at all. Yet mistakes emanating from this source of error - like, say, the winter fuel allowance cuts turning out to be the least popular thing since death was invented - don’t seem like the ones I was talking about above. We need something more.

How Do You Do, Fellow Kids? Or, Politicians’ Uniqueness

A recent - and simply outstandingly well-titled - paper called “Political practitioners poorly predict which messages persuade the public” in PNAS Political Sciences argued that politicians are actually quite bad at working out how to persuade the public. Of course, they think they can, but they can’t. They are no better than random members of the public at predicting how successful any given bit of messaging will be at actually persuading people. Data science and thorough analysis can fix this, but humans naturally work on instinct; it takes a lot of data, and an open mind, to change an initial impression. If a politician thinks a given policy or campaign focus will be successful because they personally find it persuasive, they’re both unlikely to be right and likely to act on that hunch. Given that the journalist class is so fond of talking about people’s “political instincts”, this is perhaps unsurprising. It’s natural for politicians to want to be seen as genius political operators with a great sense for the popular mood, and so it’s very hard to persuade them to think differently.

Worse, it seems that politicians have seriously wrong-headed ideas about public opinion, regardless of how many polls they’ve seen. After all, as I discussed above, polls are often wrong in ways that are absolutely small but relatively large. A 5% error in polling for support for a party isn’t a big deal in the grand scheme of things, but it absolutely matters when you’re talking about a marginal seat whose incumbent has a majority of 2.3%. One recent paper in the American Journal of Political Science shows that politicians, regardless of their information environment, seriously overestimate how much the public supports what they support. Of course, this is a common sin - we’ve all seen people ramble on about how everyone really supports such and such a policy or party deep down. What’s interesting about this is that it seems rather like an original sin of political action - it just doesn’t go away. People keep making the same misjudgement no matter how well-informed on the facts they are.

So why is all this? Data analysis gives us some interesting facts, but as yet it provides few answers. I do have a suggestion, however. Yes, politicians are human, and so are their staffers. Yet on top of that basic human fallibility, we might fairly assume that politicians are selected for self-confidence on at least some level. Someone who isn’t self-confident is unlikely to think they can guide a constituency or country! Surrounding them with data and analysis won’t change this fact: they are likely to be more obstinate and less open to new evidence than the average person. After all, there is no more dangerous a form of false confidence than that of the knowledgeable: a politician who spends all their time talking to other politicians, journalists, constituents, and the like about politics is going to be more confident that their pre-existing political views are correct than an average person.

Furthermore, politicians face perverse incentives here. Being seen to have your policy dictated by “wonks” or aides is a double danger for a politician: fellow politicians will see you as weak, and the general public will see you as listless. Keir Starmer’s current public image is evidence enough of this: though he often trumpets what is exactly the “average” view on a given issue, most people see him as gray and lacking in real beliefs. They don’t think he has a theory to answer the question of politics. Even within the world of politics, politicians want to build reputations as good judges and important sources of advice and aid. They have good reason not to follow the evidence in too much detail, and good reason to be particularly obstinate about their views. This perhaps dooms them to get things badly wrong every so often, even when they have all the time and information with which to think about things in the world.

By Oliver Haythorne