Last week, I got one of my favourite books out of the Trinity College Dublin library, having left my well-worn copy in Britain. That book is Things Can Only Get Better, comedian John O'Farrell's wonderfully self-deprecating account of his time as a Labour party activist between 1979 and 1997. Or in the words of his own subtitle: "Eighteen miserable years in the life of a Labour supporter."

You might have heard of it. It was a bestseller back in its day. More recently it has become notorious as the book where O'Farrell confessed to having been sincerely sorry that the IRA didn't manage to murder Margaret Thatcher with the Brighton bomb.

The storm generated by those comments is largely a matter of lack of context: he wrote the book from the perspective of a middle-aged man looking back with some bafflement at the extremes of youth, and he made it fairly clear that he matured beyond wishing even Lady Thatcher to an early grave. However, another insight from the book seems eerily prescient:

I got it into my head that I was something of a political jinx. Started out knocking doors on Labour's worst ever day, fighting my first ever successful campaign to elect someone to an authority which then got abolished, researching for one of the half-dozen Labour MPs who lost their job in 1987, and then standing in the only borough that saw a Tory landslide on the night of 3 May 1990. If there was a wrong place and a wrong time to be in politics, it seemed you could count on me to be there then.

Should O'Farrell ever take up his pen again, he can add: "Beaten into fourth place in a high-profile by-election by both government parties and Ukip." The hand of history appears to weigh rather heavily on his political fortunes.

Eastleigh was a strange by-election in that it was potentially momentous without being a truly great night for anybody. It isn't like Orpington 1962 or Crosby 1981 where the Liberals and SDP respectively smashed expectations with surprise wins. It isn't like Norwich North or Crewe and Nantwich in the last parliament, where the primary party of opposition won in strong government seats. Rather, even the winners cannot walk away wholly satisfied.

The Liberal Democrats made excellent use of a formidable base in local politics to hold the seat. But where in happier circumstances this might lead to great claims about the party's resurgence from the doldrums of coalition, it will just serve to brighten the torrid cloud of stories surrounding Lord Rennard and Nick Clegg. It is true that the Rennard scandal makes their by-election hold more impressive, but it won't dominate the news cycle as it might have.

Not because of the Liberal Democrats, at least. The big shock of Eastleigh was the UK Independence Party coming second. And not squeaking in by a handful of votes, either, but beating the Tories into third place by over 1,000 votes and coming less than 2,000 votes behind the victorious Liberal Democrat candidate. The purples have good reason to be pleased with their performance.

Black cloud in a sunny sky

Yet as with the Liberal Democrats, there's a "but", sitting like a black cloud in their sunny sky. One of the intriguing stories during the early days of the race was whether or not Nigel Farage, Ukip's charismatic and widely known leader, would stand as their candidate. He contested the last Eastleigh by-election, where the Liberal Democrats first took the seat, in 1994. In fact, that by-election was the first ever contested by Ukip. All of which would have provided some nice symmetry and an interesting narrative to the Ukip bid.

But he declined, and no matter what he says publicly he must surely be kicking himself for that now. Ukip didn't just do well in Eastleigh. It came within snatching distance of winning Eastleigh. It is entirely credible to think that Farage's profile, charisma and star power would have been enough to push the party over the top and give it its first MP.

As it is, it is an opportunity missed. For all the hype, political people do not reminisce about the momentous by-elections where so-and-so came in second place, and it is a long shot to think Ukip will face another non-European election with such favourable conditions for a long time. Small parties that are dominated by a single media personality need to exploit that asset ruthlessly, as the SNP have done with Alex Salmond. Small parties are not made into large parties by failures of nerve.

Moving down the list, we reach the Conservatives. The general consensus amongst commentators, and not one I'm minded to contradict, is that last night was a disaster for David Cameron's leadership of the party. Eastleigh is precisely the sort of seat that Tory 'modernisers' have been claiming their brand of socially liberal, fiscally rigorous politics was aimed at, and it sits in the low teens on the party's 2015 target seat list. The departing MP had been forced out by a criminal conviction and the defending party's leader and chief electoral strategist were engulfed in a sexual harassment scandal days before polling. This was a seat which, prior to 1994, the Tories had held for decades.

Ghost of Christmas Past

But they didn't take it. Much worse than that, they came third. Worst of all, they came third to Ukip, a party which stalks the Conservatives like the Ghost of Christmas Past. "Remember how good things used to be?" they croon to disaffected Tory voters, before going on to offer the sort of staunchly right-wing fare which was only capable of winning elections when, ironically enough, the political left was Balkanised between Labour and the Alliance. If you combine the Ukip and Conservative vote you can see what made Eastleigh a bastion of the centre-right for so long. But that vote has been split, with potentially disastrous consequences for the right.

UKIP represent a form of conservatism that Cameron's leadership has been built around repudiating. He justified this move, like Blair with Labour, by claiming he was taking party towards the British people and to power. But unlike Blair, he hasn't made a winner of himself. Blair's move to the centre ground was not marked by strong electoral challenges from the remnants of the hard left.

The road forward for the Tories now is not an easy one. First, questions over Cameron's future as leader will need to be resolved. But it is important that they don't just ditch him in a fit of panic: whoever wins has to have a joined-up alternative vision for the party and the country. That means at least giving potential challengers time to gear up. The second big risk is of a panic-driven drive to form some kind of "pact" with Ukip, the only certain consequences of which are cementing the party into British politics into the medium and long term and scaring off whatever middling voters Cameron has attracted back to the Liberal Democrats and Labour.

Speaking of Labour, they secured fourth place with a mighty 4,088 votes. To put that in perspective, in the 1994 Eastleigh by-election they came second, with 15,234 votes and 27.6 per cent of the vote. That was during Margaret Beckett's caretaker leadership after the death of John Smith, and before Tony Blair's New Labour machine stormed into the Tory south in 1997.

For Ed Miliband's claims of running a 'One Nation' Labour party, this was a vital test. New Labour's success lay in its ability to break out of Labour's northern and Celtic heartlands. He was given a by-election where both of the two strong southern parties were government parties at the same time, leaving Labour as the only major party in opposition. Yet they failed to take the seat, or even make an impression on the race. In the sort of southern seat where Labour were credible challengers even before Blair, they've almost vanished, and Labour is not in a good place when it is looking back on the glory days of 1994.

Author of Dilettante, Henry Hill is an award-winning rightwing political blogger. A liberal Conservative and Unionist Party member, he can be found on Twitter @Dilettante11.


Eastleigh By-Election: Lib Dems Retain Seat, Tories Pushed to Third Spot

Eastleigh Result: David Cameron Faces Growing Conservative Backlash

Nick Clegg Aims Swipe at David Cameron over Eastleigh By-Election

'Spirit of Margaret Thatcher' Summoned by Top Tory Michael Gove After Eastleigh Nightmare

Eastleigh: Ukip's Nigel Farage is Grinning Face of Doom for David Cameron