If you have children or teenagers in your home - or, indeed, a man under the age of 40 - the chances are that your living room has a video games console in it. Even if you don't meet those criteria, perhaps you're among the 50 million households that picked up a Nintendo Wii recently to try out the sports and fitness games.

That seemingly innocuous box, however, is more than it seems. Giant corporations such as Microsoft and Sony don't sink billions of dollars into the video games market without a grander, long-term plan.

From the outset, Sony's PlayStation and Microsoft's Xbox have been Trojan Horses. The latest versions of these devices can serve as game consoles, but they're also powerful media systems. The companies want you to use them to rent movies, watch premium TV shows, buy music and access the internet. Sony already sells an accessory for the PlayStation 3 that allows users to watch and record Freeview; Microsoft has struck a deal with Sky to bring premium TV to the Xbox 360 without the need for a satellite dish. And this is only the beginning.

Their vision of the future can be seen clearly this week at the Electronic Entertainment Expo (or E3 for short), a giant annual trade fair in Los Angeles, where the world's biggest players in digital entertainment reveal the products they're betting on in the coming year- and their longer-term ambitions.

Ostensibly, it's a video game fair; if you were attending, you'd certainly not be short of stereotype-fulfilling games about space marines and busty heroines to try. The action games for boys of all ages, however, are jostling for position with many unexpected rivals.

Sony devoted a chunk of its pre-E3 conference to announcing a tie-in with Disney's Hannah Montana, an idol among young teenage girls. Giant game maker Electronic Arts led out with games such as The Littlest Pet Shop, aimed at pre-teen girls, and EA Sports Active, a fitness package likely to find a receptive audience among busy mothers.

Video games aren't only about boys any more. And as the appeal of video games widens, Sony and Microsoft's multibillion-dollar, decades-long plans to dominate how we receive and consume media face new challengers using cheaper hardware, better design and innovative business ideas.

The first challenger is Apple, whose iPhone has become one of the world's most successful portable media and gaming devices. Earlier this year Apple announced that the App Store, which sells games and other applications directly to iPhone and iPod users, had shifted a billion pieces of software. Its home device, the Apple TV, doesn't play games - yet - but this is one rival that neither Sony nor Microsoft had banked on. (Both will be anxiously watching Apple's conference in San Francisco next week, where new iPhone models are expected to appear.)

The other challenger they face is Nintendo. Long considered to be a spent force after almost a decade of being overshadowed by Sony's PlayStation, Nintendo is suddenly the most successful company in video games again. More than any other consoles, Nintendo's Wii and its DS handheld have managed to convince people who never played games before to give them a try. It has sold more than 50 million Wii consoles in just over two years. The handheld DS records even more impressive numbers - there are 100 million in pockets and handbags around the world.

This is partly the result of Nintendo's fantastic design ethic, which has allowed it to create games, such as Wii Fit and Brain Training, which are so far from the traditional view of a video game that they almost form a new entertainment category of their own. It's also partly the result of friendly, intuitive controls. The Xbox and PlayStation joypads are, quite simply, intimidating. Bristling with buttons and sticks, they present newcomers with a steep learning curve. In comparison, Nintendo's simple touch-screen and motion sensitive controls often don't require any buttons at all.

After the entry of Apple and Nintendo into the market, the game has changed: Sony and Microsoft are left looking as if they need to catch up.

To recapture the initiative from the upstarts, their chosen battleground is motion control - technology that allows you to control your equipment with simple gestures and movements. This week Sony unveiled a system whereby remote-control-sized sticks with glowing points at the top could be tracked by a camera on the TV. It works so accurately that it's possible to write as though you were holding a pen in mid-air. One demonstration involved a handwritten note being created on a TV screen, which could presumably then be e-mailed around the world. Others showed how the controller could be a sword or gun in a game - or simply a way to point at the movie you want to see or music you want to hear.

Microsoft dropped the controller entirely for its eye-opening demonstration of a technology called Project Natal. Instead, it used a camera that can track the movements of everyone in a room, allowing you to control everything with a combination of gestures and voice commands. You could wave your hand at the TV to flip through a virtual “shelf” of your content or speak the name of the movie or game you want. To play a quiz game with the whole family, you clap your hands to “buzz in” and answer a question. The console could even carefully monitor and regulate an exercise and fitness regime.

Unlike Nintendo's existing Wii motion controller, this technology is at least a year from launch and probably won't become widespread for several more years. Here lies the difficulty for Microsoft and Sony. With expensive, powerful consoles on the market, they need to stay relevant now; exciting demonstrations of technology that even early adopters won't get their hands on for years may not be enough to stop the mass market slipping from their grasp.

Although video games will be the first to use this technology, it's a tantalising glimpse of how all our living rooms will work in the future - regardless of which company's name ends up on the box. The days of the remote control may be numbered. Instead, we'll control our media, play our games, do our exercises by learning to use gestures and voice. Forget complex buttons and flashing lights - the most advanced technology, it seems, is the technology that you can't see at all.

Times Recommends

• Bank charges hit us for being human
• Sorry, Father, thou still shalt not steal
• The shed where Santa’s failures go to die