The Financial Services Authority has levied a record £900,000 fine on one of the country's biggest independent financial adviser groups for serious failures in selling sub-prime mortgages.

The watchdog said that Thinc Group, the mortgage broker, was guilty of poor record-keeping and failing to prove that the sub-prime loans it had sold were right for the customers who took them out.

The fine is the biggest ever imposed for sub-prime failings by the FSA since it began mortgage regulation in 2004. It comes almost a year after the FSA concluded its original investigation into the sub-prime market, when it referred five companies in total for enforcement.

The other four who have either faced smaller fines or who no longer have the right to sell sub-prime loans are The Loan Company, Next Generation, Homebuyers Security Ltd and Aidan Mortgage Consultants.

The clampdown follows warnings that the UK is in danger of its own US-style sub-prime crisis. Many borrowers with poor credit histories who took out sub-prime loans before they virtually disappeared following the credit crunch could now face much higher repayments when they come to remortgage, as the market has almost completely dried-up.

In the last few months, some brokers and lenders, who typically make more profit from the sale of sub-prime rather than prime mortgages, have been accused of selling sub-prime deals to borrowers who didn't need them.

While the FSA stopped short of accusing Thinc of mis-selling, it said that it was guilty of "failure to demonstrate" why customers' credit histories merited the sale of a sub prime mortgage; why a subprime deal matched those customers' needs and circumstances; and that it had taken taken into account whether the customer could afford the mortgage it recommended.

Margaret Cole, director of enforcement at the FSA, said: "The level of fine shows that we are determined to impose higher fines for serious failings in the retail market and that poor record keeping is a serious failing even where, as in this case, the FSA has not determined that the firm mis-sold sub prime mortgages and there have been few complaints."

Thinc, which will continue to trade, but will be under review by an independent third party, said that it "regrets" its failures and is undergoing a "comprehensive remedial plan addressing record-keeping processes, adviser training and vetting procedures for non-prime mortgages."