Did we want to stay in a castle? A big one? With more battlements, turrets and drawbridges than you could lob a cannonball at?

As my three-year-old son Rory spends most days pretending he is Edmund in Narnia, while six-year-old Holly thinks she’s a princess and that living in a terraced house is all a hideous mistake, it was a done deal.

English Heritage has started renting renovated cottages in the grounds of six of its properties in locations ranging from Northumberland to Cornwall. The Sergeant Major’s House, sitting snugly within the walls of Dover Castle, was our choice and we were the first people to stay in it.

With Rory in full knight’s regalia, sword at the ready, and Holly in something pink and frilly, we swept up the drive to the castle gates and picked up the keys. We were met by the housekeeper, who said there was just one rule: after closing time we had to stick to the area around the cottage.

Not, it turned out, because of the nine ghosts in various states of distress who are supposed to haunt the castle’s nooks and crannies, but because the castle has its own microclimate, and the wind whipping round the battlements could knock us off our feet.

Holiday cottages can be such a gamble. One man’s quaint is another’s rusty and decrepit, and I’ve seen some stinkers, even when paying top dollar. Even when you stay in a historic property owned by the venerable Landmark Trust you have to make some sacrifices — not many have washing machines and none has a television.

But this was the business. A perfectly proportioned Georgian house, decorated in muted period colours and furnished with the best of modern English design, but sufficiently practical to ensure that rampaging kids could have a good time without leaving a trail of destruction.

It was fantastically well equipped, from the bespoke oak kitchen to the semi-basement kitted out with a huge flat-screen TV, wonderful selection of board games and a full-size ping-pong table.

Upstairs, the beds were made up with crisp white duvets, the main bathroom was all square sinks, trendy tiles and posh bubble bath, while the sitting room had stacks of new and interesting books.

It felt like a smart boutique hotel. But the best bit was outside: from one side of the house the view was of the keep, floodlit at night. On the other, we could see over Dover and across the sea to France. It was a perfect outlook and one I would have been happy to enjoy from the comfort of one of the steamer chairs in the garden, had the microclimate been a bit kinder.

But who cares about biting winds and lashing rain when there’s a castle to explore? And at Dover there’s an awful lot to discover. The 86-acre (35ha) site is a microcosm of English history and remarkably well preserved — partly no doubt, because apart from an annoyingly persistent French prince who laid siege to the castle in 1216, no invading army ever breached the ramparts.

We stormed the keep, rebuilt by Henry II in the 1180s to form part of the first concentric medieval fortification in Western Europe. Now it houses an exhibition about the visit by Henry VIII in 1539, which is a bit over the heads of small children, but what did fascinate my two was ye olde Tudor lavatory, a plank of wood with a hole in it over a deep shaft which still seemed to have an unmistakable aroma. “Good job I’ve got a blocked nose,” said Holly.

Times recommends

• The three best routes to the Med
• How to cash in on your holiday money
• 10 tips for the perfectly packed suitcase