When Sheriff L.O. Davis begun his campaign for re-election as the head of the police force in St Augustine, Florida, he strapped on his gun. He might need it, he explained, to fire at civil rights workers.

One of his early campaign visits was to inform African-American inhabitants of America's oldest city that he didn't want their votes. He would rather do without the support of “niggers”, he said. It wasn't much of a political sacrifice, given that non-whites were beaten or jailed if they tried to register.

Davis promptly won an overwhelming 70 per cent of the vote and, emboldened, proceeded to appoint the local Ku Klux Klan as his deputies. The leader of Davis's gang was a convicted felon named “Hoss”, who told the press that his business was “raisin' pigs and shootin' niggers”.

The year was 1964 and the election that Davis won was a Democratic primary. Read Taylor Branch's magisterial history of the civil rights movement (America in the King Years) and one of its most striking features is that the segregationists were all Democrats.

If you want to understand what is happening in this presidential election, start here: Governor Ross Barnett, who believed that God made men black to punish them and who came to court to shake the hand of the man who, in 1963, murdered the civil rights hero Medgar Evers - Democrat. Governor George Wallace, who stood in the schoolhouse door to prevent black students from entering a “whites only” college - Democrat. Senator James Eastland, who dismissed the murder of civil rights workers in Mississippi as a publicity stunt by African-Americans and who personally blocked more than 100 civil rights measures in the Senate - Democrat.

There are many bright shining moments in the history of the Democratic Party, but there is also this terrible stain. Even read about now, more than 40 years later and on another continent, the conduct of these party stalwarts repels and angers. And illuminates.

Ask yourself this - how could the great heroes of the American Left - Roosevelt, Truman, Kennedy, Johnson - have been in alliance with such men? It is simple - they needed them, they needed the votes of the segregationists. The poor, the ignorant, the isolated and dispossessed, these were their voters. And these people were damned if they were going to share their lunch counters with the kith and kin of Martin Luther King.

The tension between liberal instinct and electoral base is a recurring feature of the politics of the Left. Historical myth tends to unite great acts of liberal reform with those of social amelioration in a seamless story of progressive change. Most of the time it isn't like that. The political liberals and the mass of poor voters fight each other.

In America in the 1960s the Democrats realised that supporting equality for African-Americans would cost them dearly and it has. As he put down his pen from signing the great Civil Rights Bill, President Lyndon Johnson sighed to an aide that “we have lost the South for a generation”. He was wrong. It has taken longer than that.

Civil rights split the Democratic Party. A part of the Democrat base, the poor South, split off to support the Republicans, who had been, more or less, non-combatants in the war over civil rights. Since the day of LBJ's lament on the White House lawn there has not been a successful Democratic presidential candidate from the North, and the Republicans have won almost twice as often as their opponents. Only once in the past ten elections has a Democrat polled more than 50 per cent of the vote (Georgia's Jimmy Carter received 50.08 per cent in 1976). The long period of ascendancy for the Right in America was an uncovenanted bonus from a bitter bout of political infighting.

And now the advantages of that bonus are coming to an end. America is changing and the Democratic liberals might be able (just) to win a national election without the overwhelming support of the alienated white working class. If such a victory doesn't happen this time, it will soon. America, like Britain, is seeing the rise of what one might call a mass chattering class.

The US census shows that a record number of Americans - more than 80 per cent - now complete high school or go to college. There is also, for the first time, a mass class of millionaires - 7.3 million Americans belong to this group, with more than a million dollars of assets. If you include their primary residence in the calculation there are many millions more. More than half of the country now considers itself middle class and is working less and enjoying more leisure time. Lyndon Johnson's voters in the poor part of Texas did not have electricity. Last year more than 80 per cent of Americans went online.

With this change in economic fortunes has come a revolution in social attitudes - a mass class that is more tolerant, broadminded, socially concerned. And Democratic.

It is common to write of the current strength of the Democrats as if it were a passing phase caused by the unpopularity of George W. Bush. But it is the rise of the mass chattering class, rather than the temporary problems of the present White House incumbent, that really threatens the Republicans.

Republican reformers note with dismay that the less educated you are, the more likely you are to vote conservative. Once upon a time this might have been an electoral advantage. Now it is anything but. The poorly educated are a declining market sector.

What is significant about Barack Obama's candidacy is not simply that he is an African-American, It is that as a Northern liberal, he is still a viable candidate. I doubt that L.O. Davis would like that very much.

daniel.finkelstein@thetimes.co.uk

Times Recommends

• It’s not just about Iraq, it’s about Blair’s style
• You can’t rank hospitals like football teams
• Minarets are not an essential part of Islam