The Gillingham public address announcer welcomed Aston Villa to the Priestfield Stadium and could not resist adding that they had never won there. It was meant in jest — Villa had made only one previous visit, in 1971 — but you still suspected that he would regret tempting fate. In the end he probably did, but yesterday’s match was a close-run thing that turned on a moment of controversy.

James Milner’s first two goals since joining Villa permanently in August, the second a highly contentious penalty, sent Villa through to a fourth-round tie away to Cheltenham Town or Doncaster Rovers, and if that does not sound too challenging, then neither did their task against Coca-Cola League Two opponents — until the match got under way.

Gillingham, two points off the League Two play-off places and unbeaten in 11 home matches but beset by injuries and suspensions, had feared a heavy defeat. Instead, a display of courage and ambition meant that it was the team in fourth place in the Barclays Premier League who were relieved to remain in a competition that they want to win, as Martin O’Neill, the manager, admitted.

“We never felt comfortable all afternoon,” he said. “They were always in the game, and after their [equalising] goal it was in the balance and I would gladly have accepted a replay at Villa Park. It was blood and thunder and I’m delighted to have got through. This was big.”

O’Neill had fielded almost his strongest available team, resting only Gareth Barry, but when Villa took the lead after 14 minutes, it did not look as though his absence or those of the injured Martin Laursen or Gabriel Agbonlahor, who had been taken ill, would be felt. Stiliyan Petrov pounced when Adam Miller was unable to control a pass from Stuart Lewis, exchanged passes with Milner, who celebrated his 22nd birthday by firing past Simon Royce from 18 yards.

Early appearances, though, were deceptive. Gillingham soon recovered from the shock and began to get at a vulnerable back four. Curtis Davies and Zat Knight were troubled by the pace of Simeon Jackson, the Gillingham top scorer, which makes the return of Laursen more urgent if Villa are to continue their pursuit of a —top-four finish.

After 20 minutes Jackson set up Gary Mulligan for a low cross from the right and Dennis Oli slid in unmarked only to put his shot into the side-netting, and Jackson would have been through after Milner’s ill-judged back-pass two minutes later but for Knight’s last-ditch, full-stretch tackle.

Oli then skipped past Knight before being felled by Nigel Reo-Coker just outside the penalty area and when Jackson stole the ball from Knight and set off for goal, the Villa defender was lucky that Keith Stroud, the referee, awarded a foul. But in the 57th minute, Villa’s luck ran out as Jackson turned Knight and beat Brad Friedel with a venomous shot, to the delight of the Rainham End.

“We got undone by trying to play out of defence on a bobbly pitch, which I ask them to do, but we got back into it with a fantastic goal,” Mark Stimson, the Gillingham manager, said.

The match could have gone either way after that, but it went Villa’s when Ashley Young tumbled extravagantly after a collision with Miller and the resulting penalty was converted by Milner with 13 minutes left. Half the floodlights went out in protest at an award that looked extremely soft.

“Looked soft? It was soft,” Miller said. “Referees are under pressure to do the right thing, but doing the right thing isn’t always in favour of the big teams. I wasn’t going to get the ball but neither was he, and for the referee to give that was so disappointing. I was already thinking of a replay in front of 50,000 people.”

Stimson was less sure. “It was harsh, but you do see them given,” he said. “There was a connection and the referee thought there was enough to give a penalty, but I don’t think it was a deliberate attempt [to cheat]. We have to take it on the chin, but we gave everything we had.”

Gillingham (4-4-2): S Royce 6 - S Lewis 6, M Bentley 6, S King 6, J Nutter 6 - D Oli 7, C Weston 6, A Miller 7, A Barcham 6 - G Mulligan 6, S Jackson 8. Substitutes not used: A Julian, A Pugh, C Essam, L Rooney, J Payne, D Rance, C Stimson.

Aston Villa (4-4-2): B Friedel 6 - N Reo-Coker 5, Z Knight 4, C Davies 5, N Shorey 4 - C Gardner 5, S Petrov 7, S Sidwell 5, A Young 5 - N Delfouneso 5, J Milner 7. Substitute: N Delfouneso 5 (sub: M Harewood, 73min). Not used: B Guzan, M Salifou, I Osbourne, B Bannan, C Clark, S Lowry.

Referee: K Stroud Attendance: 10,107

Explore Football

• Premier League

• Football League

• Scotland

• European Football

• International Football

• TheGame

• Live Match Tracker

• The Fink Tank

• Fantasy Football