In the chaotic, colourful, cathartic American primary campaign of the past few months, it has in the end come down to a clarifying choice.

In a completely open field – with no incumbent president or vice-president running and both Republicans and Democrats casting about in a newly fluid ideological world – two fundamental emotions have bubbled to the surface. In the final few days before the first critical contest in Iowa, the race is between hope and fear.

The reasons for fear are obvious. America is still adjusting to the impact of 9/11 and the gruelling wars in Afghanistan and Iraq. The country is also experiencing a wave of immigration – much of it illegal and uncontrollable – greater than anything since the beginning of the last century.

In the past few years, what were once heartland certainties have been shattered: America is immune from direct military attack; America’s public culture is overwhelmingly Christian; America does not torture prisoners; if the worst happens – a hurricane like Katrina – the federal government comes to the rescue. All these bedrock assumptions have been called into question. These are unnerving, unmoored times and the candidates who have based their campaigns on fear – and their ability to assuage and reassure – have propelled themselves to prominence.

Among the Republicans, Rudy Giuliani banked everything on his response to 9/11. Fear of Al-Qaeda resonated through every speech. The assassination of Benazir Bhutto might be seen as a boon to his campaign. But in the end, Giuliani’s utterly unnuanced commitment to fighting back any time, anywhere, did not reassure. It alarmed. His mercurial temperament, fiery egotism and willingness to make enemies of everyone have become liabilities. He has fallen consistently in the polls for the entire year.

Mitt Romney, at the start, pitched himself as an inveterate optimist. Alas, his set speeches often came off as robotic invocations of themes lifted from the 1980s. And so his pitch soon reverted to fear – especially of illegal immigrants, where he taunted even Giuliani for being soft on “illegals”. For evangelicals, suspicious of his Mormonism, he relied on another set of fears. He promised to fight to make abortion illegal and ban rights for gay couples in the constitution itself.

Mike Huckabee, Romney’s chief rival in Iowa this coming Thursday, has tried another tack. His credibility as a candidate came from his being the only real true-believing fundamentalist in the field. In a Republican party remade by George Bush and Karl Rove as a religious movement, he was “one of us”. His good humour and ready wit struck many as a strange confluence of Ronald Reagan and Bill Clinton.

However, it was his economic message that appealed to working-class Republicans. In a world where globalisation unsettles many, Huckabee is the first Republican candidate in a long time to attack unabashedly free trade and unfettered market capitalism. Railing against Wall Street, he deftly exploited populist themes that had special power in states like Iowa. But, in the end, fear also undid him. In a dangerous world, his total cluelessness in foreign policy remains a huge liability. In the wake of chaos in Pakistan he looks like a risky bet.

On the Democratic side, John Edwards shifted his uplifting message of the 2004 election into a populist screed against the moneyed and powerful. Declaring the tax system to be rigged for the wealthy, the healthcare system cruelly indifferent to working Americans and Washington controlled by corrupt, wealthy lobbyists, he insisted that he alone was able to fight the forces arrayed against the little guy. Using the skills he finessed as a trial lawyer, and focusing almost manically on Iowa, he enters this week with surprising strength. Neither Barack Obama nor Hillary Clinton has been able to consign him to the asterisk status that many expected. Most polls still show the race as a tight three-way tie.

No one has exploited the politics of fear as intuitively as Clinton. Her deepest fear has long been of Republicans. She believes deep down that they command a majority and has long practised a politics that seeks first to neutralise the enemy before attempting anything positive herself. This is the scar tissue of the Reagan and Newt Gingrich eras – with the biggest wounds her 1993 healthcare debacle and the impeachment nightmare of her husband’s second term. Her biggest appeal to her party is that she can withstand the attacks from the right. And as long as they fear the Rove Republicans more than they believe in themselves, she wins.

In the battle with her fellow Democrats, she also resorts to fear of the unknown. When Obama’s poll numbers equalled hers in Iowa and New Hampshire, her surrogates unleashed a torrent of negative attacks: the Republicans will eat the young Obama for breakfast; they will smear him as a former cocaine user, as a Muslim, as black.

The candidate herself, bereft of any serious policy differences with Obama, made her final pitch that she has the experience that Obama lacks. For those afraid of risk in a world at war, she is a surer bet than the young dreamer from Illinois. And if all else fails, Bill Clinton will be there – an insurance policy for the jittery.

This leaves one viable candidate on either side. They are the least afraid and the most hopeful. They are Obama and John McCain, the Republican senator and Vietnam war hero. Yes, McCain’s experience has emerged as a great strength in an unstable world. But what remains impressive about his candidacy is that he has taken positions that are more forward-looking than many of his younger rivals.

McCain is the only Republican eager to address climate change. Faced with a Republican base furious about illegal immigration, he stuck to his view that illegal immigrants needed to be assimilated and even defended a bill that he authored with Ted Kennedy, the Democrat senator, to achieve this. He also bravely said that America does not need to torture prisoners and that the war in Iraq can be won. As the candidate of honour, he also became a candidate of hope – especially in Iraq. He has seen his numbers surge recently in New Hampshire and, if he can prevent Romney getting momentum, he still has a chance to pull it off.

Obama, of course, based his entire candidacy on the title of his campaign book, The Audacity of Hope. The fearful have every reason to look elsewhere. If you do not believe that a black man can be president; if you do not believe that America can risk talking to Iran’s leadership or withdrawing from Iraq without losing the wider war; if you think it’s naive to hope that the polarising culture war of the past 40 years can ever end; if you doubt that a man with a name like Obama who once attended a secular madrasah in Indonesia can ever win a majority of US votes, you really should vote for Clinton.

Obama knows this and directly confronts it. In the final days his appeal is disarmingly simple. “The question is, do you believe in change?” he asks. “The question is, do you believe deep in your gut we can do better than we’re doing?”

There are real and powerful reasons to fear right now. It is not crazy to want the reassurance of a former president back in the White House; it is not mysterious that retrenchment is a powerful sentiment in a world of terror and globalisation and mass immigration. Americans have to make a gut decision – whether Republican or Democrat. Should they take a risk or stick to what they know? Should they dare to be optimists or rely on the pessimism that these past few years has been a good guide to a darkening world?

After following this race for an almost interminable preamble, all I can say is that I can’t imagine a more constructive race than one between Obama and McCain. The odds are still against it. But it is more imaginable now than at any time in the past year.

And it reminds me of something. In Tel Aviv, a while back, a slogan began appearing on walls in graffiti. In the depths of the Arab-Israeli conflict, as optimism seemed like a delusion, it spread the way memes do. It’s a simple slogan and, as this new year beckons, worth holding on to, as a few Americans in a wintry state decide in which direction to take their country.

Know hope.

Times Recommends

• Care in old age: we won’t pay, the State can’t
• Viewers’ choice: our talentless Everyman
• Memo to the poor: go forth and don’t multiply